Evaluate AI agent response correctness with OpenAI and RAGAS methodology
Ready for 3.0Ready for 3.0. This template uses no nodes that n8n 3.0 removes. No removed nodes, versions or helpers found. Static check, last run 2026-10-19.
What this workflow does
This AI workflow has 24 nodes and runs on Evaluation events (plus 1 other trigger). Along the way it connects OpenAI Chat Model and Evaluation. It also calls api.openai.com over HTTP.
How it runs
Trigger: On Evaluation events (plus 1 other trigger).
- Remap Input (Edit Fields (Set)): sets 1 field: chatInput
- AI Agent: runs an AI agent, with model: OpenAI Chat Model
- Evaluation: check if evaluating
- Set Input Fields (Edit Fields (Set)): sets 3 fields: question, answer, groundTruth
- Correctness Classifier (Basic LLM Chain): sends a prompt to an LLM, with model: OpenAI Chat Model; output parser: Examples
- Get Embeddings (HTTP Request): calls api.openai.com (POST)
- GroundTruth to Items (Split Out): splits a list into separate items
- Calculate F1 Score (Code): runs custom JavaScript code
- Remap Embeddings (Edit Fields (Set)): sets 1 field: answer
- Get Embeddings (HTTP Request): calls api.openai.com (POST)
…plus 8 more steps. See the full node list below.
Services it uses
HTTP calls to: api.openai.com
Before you upgrade to n8n 3.0
Nothing in this workflow is removed in n8n 3.0, so it should import and run after the upgrade.
Nodes used (14)
- Basic LLM Chain
@n8n/n8n-nodes-langchain.chainLlm - Structured Output Parser
@n8n/n8n-nodes-langchain.outputParserStructured - OpenAI Chat Model
@n8n/n8n-nodes-langchain.lmChatOpenAi - Evaluation Trigger
n8n-nodes-base.evaluationTrigger - Edit Fields (Set)
n8n-nodes-base.set - Evaluation
n8n-nodes-base.evaluation - No Operation, do nothing
n8n-nodes-base.noOp - AI Agent
@n8n/n8n-nodes-langchain.agent - Chat Trigger
@n8n/n8n-nodes-langchain.chatTrigger - Merge
n8n-nodes-base.merge - Code
n8n-nodes-base.code - HTTP Request
n8n-nodes-base.httpRequest - Split Out
n8n-nodes-base.splitOut - Aggregate
n8n-nodes-base.aggregate
From the author
This n8n template demonstrates how to calculate the evaluation metric "Correctness" which in this scenario, measures the compares and classifies the agent's response against a set of ground truths. The scoring approach is adapted from the open-source…