Turn websites into a Google Sheets database with MrScraper and Gmail

Turn Internet Into Database — n8n Workflow Description This n8n template automates the entire process of turning any website into a structured database — no manual scraping required. It uses MrScraper's AI-powered agents to crawl a domain, extract listing pages, scrape detail pages, and export everything into Google Sheets with an email notification via Gmail. Whether you're building a real estate database, product catalog, job board aggregator, or competitor price tracker, this workflow handles the full pipeline end-to-end. --- How It Works Phase 1 – Discover URLs (Crawling): The Map Agent crawls your target domain and discovers all relevant URLs based on your include/exclude patterns. It returns a clean list of listing/search page URLs. Phase 2 – Scrape Listing Pages: The workflow loops through each discovered listing URL and runs the Listing Agent to extract all detail page URLs. Duplicates are automatically removed. Phase 3 – Scrape Detail Pages: Each detail URL is looped through the General Agent, which extracts structured fields (title, price, location, description, etc.). Nested JSON is automatically flattened into clean, spreadsheet-ready rows. Phase 4 – Export & Notify: Scraped records are appended or upserted into Google Sheets using a unique key. Once complete, a Gmail notification is sent with a run summary. --- How to Set Up 1. Create 3 scrapers in your MrScraper account: Map Agent Scraper (for crawling/URL discovery) Listing Agent Scraper (for extracting detail URLs from listing pages) General Agent Scraper (for extracting structured data from detail pages) Copy the scraperId for each — you'll need these in n8n. 2. Enable AI Scraper API access in your MrScraper account settings. 3. Add your credentials in n8n: MrScraper API token Google Sheets OAuth2 Gmail OAuth2 4. Configure the Map Agent node: Set your target domain URL (e.g. https://example.com) Set includePatterns to match listing pages (e.g. /category/) Adjust maxDepth, maxPages, and limit as needed 5. Configure the Listing Agent node: Enter the Listing scraperId Set maxPages based on how many pages per listing URL to scrape 6. Configure the General Agent node: Enter the General scraperId 7. Connect Google Sheets: Enter your spreadsheet and sheet tab URL Choose append or upsert strategy (recommended: upsert by url) 8. Configure Gmail: Set recipient email, subject line, and message body --- Requirements MrScraper account with API access enabled Google Sheets (OAuth2 connected) Gmail (OAuth2 connected) --- Good to Know The workflow uses batch looping, so large sites with hundreds of pages are handled gracefully without overloading. The Flatten Object node automatically normalizes nested JSON — no manual field mapping needed for most sites. Set a unique match key (e.g. url) in the Google Sheets upsert step to avoid duplicate rows on re-runs. Scraping speed and cost will depend on MrScraper's pricing plan and the number of pages processed. --- Customising This Workflow Different site types: Works for real estate listings, job boards, e-commerce catalogs, directory sites, and more — just adjust your URL patterns. Add filtering: Insert a Code or Filter node after Phase 3 to drop incomplete records before saving. Schedule it: Replace the manual trigger with a Schedule Trigger to run daily or weekly and keep your database fresh automatically. Multi-site: Duplicate Phase 1–3 branches to scrape multiple domains in a single workflow run.