AI Tools·Standings

AI Agent Evaluation Tools

AI agent evaluation tools check how agents perform, with evaluation methods and tool-call checks helping decide which fits your needs.

The board compares trace ingestion, safety evaluations, regression runs, SDK language support and dataset limits. Read the signal score out of 10 alongside the API, open-source and free-tier marks.

  • 28 platforms
  • 24 with a free tier
  • Checked 23 Sept 2026
  1. 1
    D
    DeepEvaldeepeval.com

    An open-source tool for evaluating AI systems, with a free plan and web, desktop, and self-hosted options.

    7.7 APIOpenFree Free tier · from $200/mo
  2. 2
    FA
    Future AGI AI Evaluation SDKfutureagi.com

    Future AGI is an AI reliability platform for simulating, evaluating, protecting, observing, optimizing and routing AI agents.

    7.7 APIOpenFree Free tier · from $250/mo
  3. 3
    P
    Promptfoopromptfoo.dev

    An open-source prompt evaluator with security checks for AI apps; free plan, plus web and desktop access.

    7.7 APIOpenFree Free tier
  4. 4
    WB
    W&B Weavesite.wandb.ai

    W&B Weave provides observability, evaluation and continuous-improvement tooling for production AI agents and LLM applications.

    7.7 APIOpenFree Free tier · from $60/mo
  5. 5
    O
    Opikcomet.com

    Opik records agent interactions, retrieval, tool calls and model responses, then evaluates and monitors them across development, testing and production.

    7.7 APIOpenFree Free tier · from $19/mo
  6. 6
    L
    LangWatchlangwatch.ai

    An open-source platform for simulating agent conversations and tool use; free plan, paid from €29 per user monthly.

    7.6 APIOpenFree Free tier · from €29/mo
  7. 7
    G
    Giskardgiskard.ai

    Giskard tests AI agents for security vulnerabilities, quality failures and compliance risks before and after deployment.

    7.4 APIOpenFree Free tier
  8. 8
    T
    TruLenstrulens.org

    An open-source tool that evaluates AI agents and records steps, inputs, costs, and response quality.

    7.1 APIOpenFree Free tier
  9. 9
    L
    LangSmithlangchain.com

    A platform that scores saved examples and live agent conversations; free plan or paid plans from $39 per user monthly.

    6.7 APIOpenFree Free tier · $39 usage-based
  10. 10
    B
    Braintrustbraintrust.dev

    A platform for reviewing agent traces and scoring outputs; free plan or paid plans from $249/month.

    6.3 APIOpenFree Free tier · from $249/mo
  11. 11
    A
    Arklexarklex.ai

    An open-source tool that tests agents with simulated conversations and checks responses against quality standards.

    6.3 APIOpenFree Free tier
  12. 12
    OE
    OpenAgent Evalopenagenthq.github.io

    A free local tool with Python and command-line options, plus reports in Markdown, HTML, and JSON.

    6.3 APIOpenFree Free tier
  13. 13
    SE
    Strands Evalsstrandsagents.com

    A free, open-source toolkit for scoring agent outputs and sessions, with checks for quality, safety, and tool use.

    6.2 APIOpenFree Free tier
  14. 14
    G
    Galileogalileo.ai

    Galileo is an observability, evaluation, and guardrail platform for GenAI and agentic applications.

    6.0 APIOpenFree Free tier · from $100/mo
  15. 15
    MA
    Maxim AIgetmaxim.ai

    A platform for agent evaluation and prompt experiments; free plan, with prompt versioning and no-code deployment.

    6.0 APIOpenFree Free tier · from $29/mo
  16. 16
    T
    Tangletangle.tools

    Tangle runs AI agents in isolated sandboxes and records their work for review before a person approves changes.

    6.0 APIOpenFree Free tier
  17. 17
    N
    Noveumnoveum.ai

    A platform that traces and scores live chat, voice, and workflow agents; free plan, paid from $69/month.

    6.0 APIOpenFree Free tier · from $69/mo
  18. 18
    H
    HoneyHivehoneyhive.ai

    A platform for inspecting and evaluating production agents, with a free plan and automated or human reviews.

    5.9 APIOpenFree Free tier
  19. 19
    A
    AgentClashagentclash.dev

    An open-source tool that runs agents on tasks, scores results, and turns failures into repeatable tests.

    5.8 APIOpenFree Free tier · from $49/mo
  20. 20
    V
    VRUNAIvrunai.com

    A web tool for testing agent paths, tool calls, outcomes, and costs across multiple AI providers.

    5.8 APIOpenFree Free tier
  21. 21
    MG
    MLflow GenAI Evaluationmlflow.org

    A free tool for evaluating AI applications and agents with test cases, human feedback, and custom scorers.

    5.8 APIOpenFree Free tier
  22. 22
    PA
    Parea AIparea.ai

    A platform for testing prompts on samples or datasets and reviewing production logs; free plan.

    5.8 APIOpenFree Free tier · from $150/mo
  23. 23
    S
    Senseisensei.sh

    Sensei is an open-source qualification engine for testing, evaluating, and certifying AI agents across professional skills.

    5.7 APIOpenFree Free tier
  24. 24
    GC
    Google Cloud Agent Evaluationdocs.cloud.google.com

    A Google Cloud service that evaluates agent performance and can create multi-turn test scenarios.

    5.6 APIOpenFree —
  25. 25
    IA
    Inspect AIinspect.aisi.org.uk

    A free model testing toolkit with 200+ evaluations, a web results viewer, and a VS Code extension.

    5.6 APIOpenFree Free tier

Places #1, #2, #3, #4, #5 on this board are open to sponsors. Take #1 · Other ways to advertise

How to choose

Which evaluation methods does it support?

Start with the evaluation methods you need, then check whether the tool also supports tool-call checks and safety evaluations.

DeepEval, Promptfoo and TruLens are examples on the board; compare their listed evaluation methods against the work you need to assess.

Can it ingest traces and run regression checks?

Check trace ingestion and regression runs if you need to review agent activity or compare results over time.

The board lists these as separate comparison fields, so confirm both rather than assuming one implies the other.

Where does the tool need to run?

Match the supported platform to your setup: the board covers API, web, self-hosted, Linux, macOS and Windows options.

If you need a tool to run on your own infrastructure, check its self-hosted support; for a service connection, check for an API.

Does it have a free plan, and what does paid access cost?

Compare the free-plan mark and the paid-from price before choosing. A free plan is listed for DeepEval, Promptfoo and TruLens; Braintrust lists paid access from $249 per month, while LangSmith lists it from $39 per user per month.

Pricing notes can distinguish a free plan from a free trial. Noveum lists a 7-day trial, and Maxim AI lists a 14-day trial.

Which SDK languages and dataset limits should I check?

Check SDK language support against the languages your team uses, and compare dataset limits with the evaluations you plan to run.

These are separate fields on the board.

Questions

What are AI agent evaluation tools used for?

They assess AI agent performance. The board compares evaluation methods, tool-call checks, trace ingestion, safety evaluations and regression runs.

Which AI agent evaluation tools have a free plan?

The board marks free plans, and examples include DeepEval, Promptfoo and TruLens. Check the free-plan mark and pricing note for the tool you are considering.

Do AI agent evaluation tools support self-hosting?

Self-hosted support is one of the board's platform fields. Check the platform details for a tool before choosing it for your setup.

What should I compare besides evaluation methods?

Compare tool-call checks, trace ingestion, safety evaluations, regression runs, SDK language support and dataset limits, along with pricing and platform support.

How do I tell a free plan from a free trial?

The board lists free-plan and free-trial information separately. For example, Noveum lists a 7-day trial and Maxim AI a 14-day trial.

Some top positions on this board can be paid placements, standing where they were sold. The facts on each platform are not for sale. How we rank · Disclosure