Evaluate AI prompts with OpenRouter LLM-as-judge and Notion reports
Ready for 3.0Ready for 3.0. This template uses no nodes that n8n 3.0 removes. No removed nodes, versions or helpers found. Static check, last run 2026-10-19.
What this workflow does
This AI workflow has 17 nodes and runs when someone submits an n8n form (“Evaluate AI prompt quality with LLM-as-Judge and save result”). Along the way it connects OpenRouter Chat Model and Notion.
How it runs
Trigger: When someone submits an n8n form (“Evaluate AI prompt quality with LLM-as-Judge and save result”).
- Set Notion IDs (Edit Fields (Set)): sets 2 fields: NOTION_PARENT_PAGE_ID, NOTION_DATABASE_ID
- Dataset Generator (AI Agent): runs an AI agent, with model: OpenAI Dataset Generator
- Flatten Dataset Code (Code): runs custom JavaScript code
- Batch Process Items (Loop Over Items (Split in Batches)): loops over the items in batches
- Collate Evaluation Results (Code): runs custom JavaScript code
- Upload Item to Notion (Notion): work with database page
- Prepare Evaluation Item (Code): runs custom JavaScript code
- Synthesis Agent (AI Agent): runs an AI agent, with model: OpenAI Synthesis Agent
- Answer Agent (AI Agent): runs an AI agent, with model: OpenAI Answer Agent
- Upload Report to Notion (Notion): uses Notion
…plus 2 more steps. See the full node list below.
Services it uses
Before you upgrade to n8n 3.0
Nothing in this workflow is removed in n8n 3.0, so it should import and run after the upgrade.
Nodes used (7)
- AI Agent
@n8n/n8n-nodes-langchain.agent - n8n Form Trigger
n8n-nodes-base.formTrigger - Edit Fields (Set)
n8n-nodes-base.set - OpenRouter Chat Model
@n8n/n8n-nodes-langchain.lmChatOpenRouter - Code
n8n-nodes-base.code - Notion
n8n-nodes-base.notion - Loop Over Items (Split in Batches)
n8n-nodes-base.splitInBatches
From the author
Quick overview This workflow collects a prompt and model choice via an n8n Form, uses OpenRouter LLMs to generate an evaluation dataset, run the prompt against each test case, and grade the responses, then saves per-item results and a synthesized one-line…