Orchestrate web crawls with Scrapyd and automated data enrichment
Ready for 3.0Ready for 3.0. This template uses no nodes that n8n 3.0 removes. No removed nodes, versions or helpers found. Static check, last run 2026-10-19.
What this workflow does
This workflow has 21 nodes and runs manually, when you click “Execute workflow”. It uses only n8n's built-in nodes.
How it runs
Trigger: Manually, when you click “Execute workflow”.
- run_job (HTTP Request): calls an HTTP API (POST)
- job_list (HTTP Request): calls an HTTP API
- filter_job (Code): runs custom JavaScript code
- check_job_status (If): checks a condition and branches into true / false
- check_items (HTTP Request): calls an HTTP API
- job_log (HTTP Request): calls an HTTP API
- Wait: pauses before continuing
- Filter-result (Code): runs custom JavaScript code
- If: checks a condition and branches into true / false
- Respond to Webhook: sends the response back to the webhook caller
…plus 10 more steps. See the full node list below.
Before you upgrade to n8n 3.0
Nothing in this workflow is removed in n8n 3.0, so it should import and run after the upgrade.
Nodes used (9)
- Manual Trigger
n8n-nodes-base.manualTrigger - HTTP Request
n8n-nodes-base.httpRequest - Code
n8n-nodes-base.code - If
n8n-nodes-base.if - Wait
n8n-nodes-base.wait - Loop Over Items (Split in Batches)
n8n-nodes-base.splitInBatches - Aggregate
n8n-nodes-base.aggregate - Split Out
n8n-nodes-base.splitOut - Respond to Webhook
n8n-nodes-base.respondToWebhook
From the author
How it works This workflow runs a spider job in the background via Scrapyd, using a YAML config that defines selectors and parsing rules. When triggered, it schedules the spider with parameters (query, project ID, page limits, etc.). The workflow polls…