Start ScraperAPI crawler jobs from webhooks and store results in data tables
UnknownWhat this workflow does
This workflow has 9 nodes and runs manually, when you click “Execute workflow” (plus 2 other triggers). Along the way it connects Scraperapi Official (community).
How it runs
Trigger: Manually, when you click “Execute workflow” (plus 2 other triggers).
- Create crawled-pages table (Data table): create (table)
- Build crawl parameters (Code): runs custom JavaScript code
- Parse crawled pages (Code): runs custom JavaScript code
- Start crawler job (Scraper Api): crawler job create
- Store crawled pages (Data table): create or update (row)
- Respond with crawler job (Respond to Webhook): sends the response back to the webhook caller
Services it uses
Before you upgrade to n8n 3.0
We can't fully verify this one: it relies on 1 community package, and n8n 3.0 turns unverified community packages off by default (N8N_UNVERIFIED_PACKAGES_ENABLED=false).
- Check that n8n-nodes-scraperapi-official is verified or supports 3.0 (used in Start crawler job).
Nodes used (6)
- Manual Trigger
n8n-nodes-base.manualTrigger - Data table
n8n-nodes-base.dataTable - Webhook
n8n-nodes-base.webhook - Code
n8n-nodes-base.code - Scraper Api
n8n-nodes-scraperapi-official.scraperApi - Respond to Webhook
n8n-nodes-base.respondToWebhook
From the author
Quick Overview This workflow starts a ScraperAPI crawler job from an incoming webhook request and stores each crawled page returned via a callback webhook into an n8n Data Table. How it works 1. Receives a POST request on a webhook with crawl inputs like…