Evaluation metric example: Correctness (judged by AI)
Needs changesNeeds changes. This template uses 1 thing(s) that n8n 3.0 removes: AI Agent v1 (old agent modes). 1 breaking item(s) found. Static check, last run 2026-10-19.
What to replace before upgrading
| Removed in 3.0 | Replace with |
|---|---|
| AI Agent v1 (old agent modes) AI Agent (v1.9) | Latest AI Agent node (Tools Agent). For SQL Agent, use a Postgres or MySQL tool with a recent AI Agent. |
Nodes used (8)
- OpenAI Chat Model
@n8n/n8n-nodes-langchain.lmChatOpenAi - Evaluation Trigger
n8n-nodes-base.evaluationTrigger - Evaluation
n8n-nodes-base.evaluation - AI Agent
@n8n/n8n-nodes-langchain.agent - OpenAI
@n8n/n8n-nodes-langchain.openAi - Chat Trigger
@n8n/n8n-nodes-langchain.chatTrigger - Edit Fields (Set)
n8n-nodes-base.set - No Operation, do nothing
n8n-nodes-base.noOp
What it does
AI evaluation in n8n This is a template for n8n's evaluation feature. Evaluation is a technique for getting confidence that your AI workflow performs reliably, by running a test dataset containing different inputs through the workflow. By calculating a metric (score) for each input, you can see where the workflow is performing well and where it isn't. How it works This template shows how to calculate a workflow evaluation metric: whether an output matches an expected output (i.e. has the same meaning). The workflow takes questions about the causes of historical events and compares them with the reference answers in the dataset. - We use an evaluation trigger to read in our dataset - It is wired up in parallel with the regular chat trigger so that the workflow can be started from either one. More info - If we're evaluating (i.e. the execution started from the evaluation trigger), we calculate the correctness metric using AI - We pass this information back to n8n as a metric - If we're not evaluating we avoid calculating the metric, to reduce cost