If you build automations in n8n, the popular workflow tool that lets you wire apps and AI together without much code, the July 2026 release has a feature worth stopping for. n8n added built-in evaluations for AI workflows. In plain terms: you can now test whether your AI automation actually works before it goes live and starts making decisions on your behalf.
The problem evals solve
Anyone who has put a language model inside an automation knows the sick feeling. It worked in the three examples you tried. Then a real customer sends a slightly weird message and the thing confidently does the wrong thing. Ordinary automations are predictable. AI ones are not, because the model can respond differently to inputs that look almost identical. You cannot eyeball your way to confidence.
Evaluations fix that by turning ‘I hope this works’ into a number. You give the workflow a set of real inputs paired with the answers you expect, run them through, and score how close the output lands. n8n now includes a set of built-in metrics so you are not inventing the scoring maths yourself, and there is an ‘add evaluation trigger from setup’ option in the editor so wiring it in takes a couple of clicks.
What you can actually do with it
Three things, all useful to a small team.
One, catch regressions. Change a prompt, rerun the evaluation, and see whether your score went up or down before you push the change to customers. No more finding out from an angry email.
Two, choose a model with evidence. Run the same test set through two different models and compare the scores. Now ‘which model should I use’ is a data question, not a vibe.
Three, watch for drift. Run the evaluation on a schedule and review the scores as a trend, so you notice when performance slips instead of discovering it after a bad week.
How to start without over-engineering
You do not need a machine-learning team or a hundred test cases. Start with twenty to fifty examples pulled from real failures: the messages that tripped your workflow up, each with the answer it should have given. Add the evaluation trigger, run it, and treat the score as a baseline. Every time you change a prompt or a model, rerun and compare.
The wider n8n July release also brought a new AI builder, a workflow review feature, MCP and API improvements and admin-managed credentials, part of the 2.0 line that isolated workflow execution in task runners for safety. But evaluations are the one that changes how you work. It moves n8n from ‘automation you cross your fingers over’ to ‘automation you can test like software’, which is the difference between a hobby and something you can put your name on.
The class angle: for years, testing AI properly was something only companies with data scientists did. Now it is a trigger in a tool a solo operator can self-host for nothing. The gap between the big players and the rest just got a little smaller.
Did you know: n8n began as a self-hostable side project, and the name is pronounced ‘n-eight-n’, short for ‘nodemation’, nodes plus automation.