Domain-specific web content crawler with depth control & text extraction
Ready for 3.0Ready for 3.0. This template uses no nodes that n8n 3.0 removes. No removed nodes, versions or helpers found. Static check, last run 2026-10-19.
What this workflow does
This workflow has 16 nodes and runs when an HTTP POST request hits its webhook URL. It uses only n8n's built-in nodes.
How it runs
Trigger: When an HTTP POST request hits its webhook URL.
- Init Crawl Params (Edit Fields (Set)): sets or maps fields
- Init Globals (Code): runs custom JavaScript code
- Seed Root Crawl Item (Merge): merges the data from its input branches
- Merge: merges the data from its input branches
- Fetch HTML Page (HTTP Request): calls an HTTP API
- Combine & Chunk (Code): runs custom JavaScript code
- Attach URL/Depth to HTML (Code): runs custom JavaScript code
- Respond to Webhook: sends the response back to the webhook caller
- Extract Body & Links (HTML): parses or builds HTML
- Queue & Dedup Links (Code): runs custom JavaScript code
…plus 5 more steps. See the full node list below.
Before you upgrade to n8n 3.0
Nothing in this workflow is removed in n8n 3.0, so it should import and run after the upgrade.
Nodes used (9)
- Webhook
n8n-nodes-base.webhook - Loop Over Items (Split in Batches)
n8n-nodes-base.splitInBatches - If
n8n-nodes-base.if - HTML
n8n-nodes-base.html - Code
n8n-nodes-base.code - HTTP Request
n8n-nodes-base.httpRequest - Merge
n8n-nodes-base.merge - Edit Fields (Set)
n8n-nodes-base.set - Respond to Webhook
n8n-nodes-base.respondToWebhook
From the author
This template implements a recursive web crawler inside n8n. Starting from a given URL, it crawls linked pages up to a maximum depth (default: 3), extracts text and links, and returns the collected content via webhook. --- 🚀 How It Works 1) Webhook Trigger…