Sync website crawl deltas into Qdrant with Apify and OpenAI embeddings
UnknownWhat this workflow does
This workflow has 7 nodes and runs on Apify events. Along the way it connects Apify (community). It also calls api.apify.com over HTTP.
How it runs
Trigger: On Apify events.
- Fetch Qdrant Vectors Pre-Sync (HTTP Request): calls an HTTP API
- Post Dataset to Qdrant (HTTP Request): calls api.apify.com (POST)
- Wait: waits 30 seconds
- Fetch Qdrant Vectors Post-Sync (HTTP Request): calls an HTTP API
- Retrieve Sync Log (HTTP Request): calls an HTTP API
- Analyze Freshness Report (Code): runs custom JavaScript code
Services it uses
HTTP calls to: api.apify.com
Before you upgrade to n8n 3.0
We can't fully verify this one: it relies on 1 community package, and n8n 3.0 turns unverified community packages off by default (N8N_UNVERIFIED_PACKAGES_ENABLED=false).
- Check that @apify/n8n-nodes-apify is verified or supports 3.0 (used in When Apify Job Completes).
Nodes used (4)
- Apify Trigger
@apify/n8n-nodes-apify.apifyTrigger - HTTP Request
n8n-nodes-base.httpRequest - Wait
n8n-nodes-base.wait - Code
n8n-nodes-base.code
From the author
Quick overview This workflow triggers on a completed Apify Website Content Crawler run, syncs the crawl dataset into a Qdrant collection using OpenAI embeddings, then compares vector counts and parses the Apify run logs to report what chunks were added,…