AI Tools·Standings

Compare AI LLM Observability Tools

AI LLM observability tools track what happens when an AI app runs; compare trace capture, token costs, evaluations, latency and failure tracking.

Use the signal score out of 10 as one comparison point, then check the API, open-source and free-tier marks. Compare free-plan retention and paid starting prices before choosing.

  • 23 platforms
  • 1 with a public API on this page
  • 23 with a free tier
  • Checked 23 Sept 2026
  1. 1
    WB
    W&B Weavesite.wandb.ai

    An open-source tool for tracking and refining production AI agents, with a free plan.

    7.7 APIOpenFree Free tier · from $60/mo
  2. 2
    O
    OpenLITopenlit.io

    A free, open-source tool for tracing AI activity, scoring results, and tracking costs.

    7.7 APIOpenFree Free tier
  3. 3
    O
    Opikcomet.com

    An open-source tool that logs agent actions and evaluates AI apps from development through production.

    7.7 APIOpenFree Free tier · from $19/mo
  4. 4
    L
    LangWatchlangwatch.ai

    An open-source platform with simulations for multi-turn agents, tool use, and recovery.

    7.6 APIOpenFree Free tier · from €29/mo
  5. 5
    L
    Langfuselangfuse.com

    An open-source platform that traces AI app activity and manages prompts, with a free plan.

    7.4 APIOpenFree Free tier · from $29/mo
  6. 6
    MT
    MLflow Tracingmlflow.org

    An open-source tracing tool that records LLM inputs, outputs, and request steps to help find problems.

    7.4 APIOpenFree Free tier
  7. 7
    L
    Lunarylunary.ai

    A web and self-hosted tool for tracking AI activity, errors, and search.

    7.3 APIOpenFree Free tier · from $20/mo
  8. 8
    A
    Agentaagenta.ai

    A web and self-hosted tool for AI coworkers, with Slack and Telegram support.

    7.3 APIOpenFree Free tier · from $29/mo
  9. 9
    T
    TruLenstrulens.org

    An open-source tool that traces AI steps and evaluates agents, retrieval apps, and summaries.

    7.1 APIOpenFree Free tier
  10. 10
    L
    LangSmithlangchain.com

    A platform for building and monitoring agents, with evaluations of test data and live use.

    6.7 APIOpenFree Free tier · $39 usage-based
  11. 11
    B
    Braintrustbraintrust.dev

    A platform for reviewing agent activity and measuring latency, cost, and quality.

    6.3 APIOpenFree Free tier · from $249/mo
  12. 12
    P
    Portkeyportkey.ai

    A platform with one API for AI models and configurable routing, retries, and failovers.

    6.0 APIOpenFree Free tier · from $49/mo
  13. 13
    FA
    Fiddler AIfiddler.ai

    A platform for evaluating AI in development and monitoring models and agents in production.

    6.0 APIOpenFree Free tier
  14. 14
    G
    Galileogalileo.ai

    A platform for evaluating and monitoring GenAI apps, with tools for experiments and guardrails.

    6.0 APIOpenFree Free tier · from $100/mo
  15. 15
    MA
    Maxim AIgetmaxim.ai

    Maxim is an end-to-end evaluation and observability platform for simulating, evaluating, and monitoring AI agents.

    6.0 APIOpenFree Free tier · from $29/mo
  16. 16
    AP
    Arize Phoenixarize.com

    A free tool for tracing and evaluating AI apps, with prompt testing and experiment comparisons.

    5.9 APIOpenFree Free tier
  17. 17
    CA
    Confident AIconfident-ai.com

    A platform for evaluating AI apps and tracing production activity, with alerts when quality drops.

    5.9 APIOpenFree Free tier · from $200/mo
  18. 18
    H
    HoneyHivehoneyhive.ai

    A platform for tracing agents and evaluating live traffic, with drift detection and alerts.

    5.9 APIOpenFree Free tier
  19. 19
    H
    Heliconehelicone.ai

    A platform that logs and analyzes AI requests, with model routing, failovers, and caching.

    5.8 APIOpenFree Free tier · from $79/mo
  20. 20
    RF
    Respan (formerly Keywords AI)respan.ai

    An AI gateway with tracing, evaluations, and security testing that routes requests to over 1,000 models.

    5.8 APIOpenFree Free tier · from $199/yr
  21. 21
    AA
    Arize AXarize.com

    A platform for improving agents with trace investigations, evaluations, and swarm monitoring.

    5.8 APIOpenFree Free tier · from $50/mo
  22. 22
    P
    PromptLayerpromptlayer.com

    A prompt editor and registry with versioning, evaluations, observability, and agent tracing.

    5.8 APIOpenFree Free tier · from $49/mo
  23. 23
    O
    OpenLLMetrytraceloop.com

    An open-source monitor that traces LLM app activity and exports it to Traceloop or another observability stack.

    5.5 APIOpenFree Free tier

Places #1, #2, #3, #4, #5 on this board are open to sponsors. Take #1 · Other ways to advertise

How to choose

Which platforms does it need to run on?

Check whether you need an API, a self-hosted setup or a web app. The platform marks also cover Linux, macOS and Windows.

An API lets software connect through a defined interface. Self-hosting means running the tool on infrastructure you manage.

What should it track in my AI app?

Match the tool to what you need to see: LLM traces, token costs, output evaluations, latency or failures.

A trace records steps in an AI app's request and response. Output evaluations help assess the responses; latency metrics show response time.

How can I compare free and paid plans?

Check the free-plan mark, any trial, the paid starting price and free-plan retention. Retention tells you how long free-plan records remain available.

Published examples include OpenLIT, listed as free, and Agenta, with a free plan and paid pricing from $29 per month. Prices and plan terms can differ, so check the current offer before deciding.

Do I need an open-source tool or public API?

Check the open-source and API marks against how you plan to run and connect the tool. If you need to read or change the code, review the tool's license.

A public API can connect the observability tool to other software. Confirm the details for the specific tool before choosing.

Questions

What does an AI LLM observability tool do?

It helps track AI app activity, including traces, token costs, output evaluations, latency and failures.

How do I compare LLM observability tools?

Compare the tracking features you need, platform support, free-plan retention, paid starting price, and API, open-source and free-tier marks.

Can I self-host an LLM observability tool?

Self-hosted is a listed platform to compare. Check the individual tool's platform details to confirm its setup.

Are there free LLM observability tools?

The board includes free-plan and free-trial information. Check a tool's current plan terms, retention and paid starting price.

What is an LLM trace?

A trace records steps in an AI app's request and response, helping you see what happened during a run.

Some top positions on this board can be paid placements, standing where they were sold. The facts on each platform are not for sale. How we rank · Disclosure