Scrape multi-page websites recursively with Google Sheets storage
Ready for 3.0What this workflow does
This workflow has 12 nodes and runs manually, when you click “Execute workflow”. Along the way it connects Google Sheets.
How it runs
Trigger: Manually, when you click “Execute workflow”.
- Input (Edit Fields (Set)): sets or maps fields
- Get Start URL (HTTP Request): calls an HTTP API
- Split Out: splits a list into separate items
- Merge: merges the data from its input branches
- Scrape Next Page Link (HTML): parses or builds HTML
- Scrape Fields (HTML): parses or builds HTML
- If: checks a condition and branches into true / false
- Aggregate: combines items into one
- Next Page Input (Edit Fields (Set)): sets 3 fields: startUrl, nextPageSelector, fields
- Split Out: splits a list into separate items
…plus 1 more step. See the full node list below.
Services it uses
Before you upgrade to n8n 3.0
Nothing in this workflow is removed in n8n 3.0, so it should import and run after the upgrade.
Nodes used (9)
- Manual Trigger
n8n-nodes-base.manualTrigger - Edit Fields (Set)
n8n-nodes-base.set - HTTP Request
n8n-nodes-base.httpRequest - Split Out
n8n-nodes-base.splitOut - Merge
n8n-nodes-base.merge - HTML
n8n-nodes-base.html - If
n8n-nodes-base.if - Aggregate
n8n-nodes-base.aggregate - Google Sheets
n8n-nodes-base.googleSheets
From the author
Configurable Multi-Page Web Scraper Introduction This n8n workflow provides a robust and highly reusable solution for scraping data from paginated websites. Instead of building a complex series of nodes for every new site, you only need to update a simple…