AI Tools·Standings
AI Agent Evaluation Tools
AI agent evaluation tools check how agents perform, with evaluation methods and tool-call checks helping decide which fits your needs.
The board compares trace ingestion, safety evaluations, regression runs, SDK language support and dataset limits. Read the signal score out of 10 alongside the API, open-source and free-tier marks.
- 28 platforms
- 24 with a free tier
- Checked 23 Sept 2026
- 1 D
DeepEvaldeepeval.comAn open-source tool for evaluating AI systems, with a free plan and web, desktop, and self-hosted options.
- 2 FA
Future AGI AI Evaluation SDKfutureagi.comFuture AGI is an AI reliability platform for simulating, evaluating, protecting, observing, optimizing and routing AI agents.
- 3 P
Promptfoopromptfoo.devAn open-source prompt evaluator with security checks for AI apps; free plan, plus web and desktop access.
- 4 WB
W&B Weavesite.wandb.aiW&B Weave provides observability, evaluation and continuous-improvement tooling for production AI agents and LLM applications.
- 5 O
Opikcomet.comOpik records agent interactions, retrieval, tool calls and model responses, then evaluates and monitors them across development, testing and production.
- 6 L
LangWatchlangwatch.aiAn open-source platform for simulating agent conversations and tool use; free plan, paid from €29 per user monthly.
- 7 G
Giskardgiskard.aiGiskard tests AI agents for security vulnerabilities, quality failures and compliance risks before and after deployment.
- 8 TTruLenstrulens.org
An open-source tool that evaluates AI agents and records steps, inputs, costs, and response quality.
- 9 L
LangSmithlangchain.comA platform that scores saved examples and live agent conversations; free plan or paid plans from $39 per user monthly.
- 10 B
Braintrustbraintrust.devA platform for reviewing agent traces and scoring outputs; free plan or paid plans from $249/month.
- 11 A
Arklexarklex.aiAn open-source tool that tests agents with simulated conversations and checks responses against quality standards.
- 12 OEOpenAgent Evalopenagenthq.github.io
A free local tool with Python and command-line options, plus reports in Markdown, HTML, and JSON.
- 13 SEStrands Evalsstrandsagents.com
A free, open-source toolkit for scoring agent outputs and sessions, with checks for quality, safety, and tool use.
- 14 G
Galileogalileo.aiGalileo is an observability, evaluation, and guardrail platform for GenAI and agentic applications.
- 15 MA
Maxim AIgetmaxim.aiA platform for agent evaluation and prompt experiments; free plan, with prompt versioning and no-code deployment.
- 16 TTangletangle.tools
Tangle runs AI agents in isolated sandboxes and records their work for review before a person approves changes.
- 17 N
Noveumnoveum.aiA platform that traces and scores live chat, voice, and workflow agents; free plan, paid from $69/month.
- 18 H
HoneyHivehoneyhive.aiA platform for inspecting and evaluating production agents, with a free plan and automated or human reviews.
- 19 A
AgentClashagentclash.devAn open-source tool that runs agents on tasks, scores results, and turns failures into repeatable tests.
- 20 VVRUNAIvrunai.com
A web tool for testing agent paths, tool calls, outcomes, and costs across multiple AI providers.
- 21 MGMLflow GenAI Evaluationmlflow.org
A free tool for evaluating AI applications and agents with test cases, human feedback, and custom scorers.
- 22 PAParea AIparea.ai
A platform for testing prompts on samples or datasets and reviewing production logs; free plan.
- 23 SSenseisensei.sh
Sensei is an open-source qualification engine for testing, evaluating, and certifying AI agents across professional skills.
- 24 GCGoogle Cloud Agent Evaluationdocs.cloud.google.com
A Google Cloud service that evaluates agent performance and can create multi-turn test scenarios.
- 25 IA
Inspect AIinspect.aisi.org.ukA free model testing toolkit with 200+ evaluations, a web results viewer, and a VS Code extension.
Places #1, #2, #3, #4, #5 on this board are open to sponsors. Take #1 · Other ways to advertise
How to choose
Which evaluation methods does it support?
Start with the evaluation methods you need, then check whether the tool also supports tool-call checks and safety evaluations.
DeepEval, Promptfoo and TruLens are examples on the board; compare their listed evaluation methods against the work you need to assess.
Can it ingest traces and run regression checks?
Check trace ingestion and regression runs if you need to review agent activity or compare results over time.
The board lists these as separate comparison fields, so confirm both rather than assuming one implies the other.
Where does the tool need to run?
Match the supported platform to your setup: the board covers API, web, self-hosted, Linux, macOS and Windows options.
If you need a tool to run on your own infrastructure, check its self-hosted support; for a service connection, check for an API.
Does it have a free plan, and what does paid access cost?
Compare the free-plan mark and the paid-from price before choosing. A free plan is listed for DeepEval, Promptfoo and TruLens; Braintrust lists paid access from $249 per month, while LangSmith lists it from $39 per user per month.
Pricing notes can distinguish a free plan from a free trial. Noveum lists a 7-day trial, and Maxim AI lists a 14-day trial.
Which SDK languages and dataset limits should I check?
Check SDK language support against the languages your team uses, and compare dataset limits with the evaluations you plan to run.
These are separate fields on the board.
Questions
What are AI agent evaluation tools used for?
They assess AI agent performance. The board compares evaluation methods, tool-call checks, trace ingestion, safety evaluations and regression runs.
Which AI agent evaluation tools have a free plan?
The board marks free plans, and examples include DeepEval, Promptfoo and TruLens. Check the free-plan mark and pricing note for the tool you are considering.
Do AI agent evaluation tools support self-hosting?
Self-hosted support is one of the board's platform fields. Check the platform details for a tool before choosing it for your setup.
What should I compare besides evaluation methods?
Compare tool-call checks, trace ingestion, safety evaluations, regression runs, SDK language support and dataset limits, along with pricing and platform support.
How do I tell a free plan from a free trial?
The board lists free-plan and free-trial information separately. For example, Noveum lists a 7-day trial and Maxim AI a 14-day trial.
Some top positions on this board can be paid placements, standing where they were sold. The facts on each platform are not for sale. How we rank · Disclosure