Audit AI and search bot access using robots.txt checks and Gmail reports
Ready for 3.0Ready for 3.0. This template uses no nodes that n8n 3.0 removes. No removed nodes, versions or helpers found. Static check, last run 2026-10-19.
What this workflow does
This workflow has 14 nodes and runs on a schedule (every month at 08:00). Along the way it connects Gmail.
How it runs
Trigger: On a schedule (every month at 08:00).
- Set Audit Parameters (Edit Fields (Set)): sets 4 fields: testUrl, bots, browserUserAgent…
- Fetch robots.txt (HTTP Request): calls an HTTP API
- Normalise Response (Code): runs custom JavaScript code
- Resolve robots.txt Rules (Code): runs custom JavaScript code
- Request Page as Each Bot (HTTP Request): calls an HTTP API
- Attach Bot Responses (Code): runs custom JavaScript code
- Is This the Browser Control Run? (If): checks a condition and branches into true / false
- Request Page as a Browser (HTTP Request): calls an HTTP API
- Merge: merges the data from its input branches
- Measure Browser Response (Code): runs custom JavaScript code
…plus 3 more steps. See the full node list below.
Services it uses
Before you upgrade to n8n 3.0
Nothing in this workflow is removed in n8n 3.0, so it should import and run after the upgrade.
Nodes used (8)
- Schedule Trigger
n8n-nodes-base.scheduleTrigger - Edit Fields (Set)
n8n-nodes-base.set - HTTP Request
n8n-nodes-base.httpRequest - Code
n8n-nodes-base.code - If
n8n-nodes-base.if - Merge
n8n-nodes-base.merge - Aggregate
n8n-nodes-base.aggregate - Gmail
n8n-nodes-base.gmail
From the author
Quick overview This workflow runs monthly to audit whether popular AI and search crawlers can access a specific page by comparing robots.txt rules with real HTTP responses, then emails a verdict report via Gmail. How it works 1. Runs on a monthly schedule…