Web crawler: Convert websites to AI-ready markdown in Google Sheets

by Daniel Nkencho ✓ · 2,387 views · created 2025-10-13 · Document Extraction, AI RAG

What this workflow does

This workflow has 16 nodes and runs manually, when you click “Execute workflow”. Along the way it connects Google Sheets.

How it runs

Trigger: Manually, when you click “Execute workflow”.

1. Set Website (Edit Fields (Set)): sets 1 field: website_url
2. Scrape Homepage (HTTP Request): calls an HTTP API
3. Extract Links from HTML (HTML): parses or builds HTML
4. Split Links (Split Out): splits a list into separate items
5. Remove Duplicate Links (Remove Duplicates): removes duplicate items
6. Filter: keeps only the items that match a condition
7. Separate Images and Links (Switch): routes items into 2 branches
8. Aggregate: combines items into one
9. Aggregate: combines items into one
10. Scrape Content Links (HTTP Request): calls an HTTP API

…plus 5 more steps. See the full node list below.

Services it uses

Google Sheets

Scraping pages by hand? Firecrawl is a simpler option (10% off for new users).

Before you upgrade to n8n 3.0

Nodes used (11)

• Edit Fields (Set)n8n-nodes-base.set
• Manual Triggern8n-nodes-base.manualTrigger
• HTTP Requestn8n-nodes-base.httpRequest
• HTMLn8n-nodes-base.html
• Split Outn8n-nodes-base.splitOut
• Remove Duplicatesn8n-nodes-base.removeDuplicates
• Filtern8n-nodes-base.filter
• Switchn8n-nodes-base.switch
• Aggregaten8n-nodes-base.aggregate
• Markdownn8n-nodes-base.markdown
• Google Sheetsn8n-nodes-base.googleSheets

From the author

Transform any website into a structured knowledge repository with this intelligent crawler that extracts hyperlinks from the homepage, intelligently filters images and content pages, and aggregates full Markdown-formatted content—perfect for fueling AI agents…

View the original on n8n.io → Check your own copy