AI Tools·Standings
Compare AI LLM Observability Tools
AI LLM observability tools track what happens when an AI app runs; compare trace capture, token costs, evaluations, latency and failure tracking.
Use the signal score out of 10 as one comparison point, then check the API, open-source and free-tier marks. Compare free-plan retention and paid starting prices before choosing.
- 23 platforms
- 1 with a public API on this page
- 23 with a free tier
- Checked 23 Sept 2026
- 1 WB
W&B Weavesite.wandb.aiAn open-source tool for tracking and refining production AI agents, with a free plan.
- 2 OOpenLITopenlit.io
A free, open-source tool for tracing AI activity, scoring results, and tracking costs.
- 3 O
Opikcomet.comAn open-source tool that logs agent actions and evaluates AI apps from development through production.
- 4 L
LangWatchlangwatch.aiAn open-source platform with simulations for multi-turn agents, tool use, and recovery.
- 5 L
Langfuselangfuse.comAn open-source platform that traces AI app activity and manages prompts, with a free plan.
- 6 MTMLflow Tracingmlflow.org
An open-source tracing tool that records LLM inputs, outputs, and request steps to help find problems.
- 7 LLunarylunary.ai
A web and self-hosted tool for tracking AI activity, errors, and search.
- 8 A
Agentaagenta.aiA web and self-hosted tool for AI coworkers, with Slack and Telegram support.
- 9 TTruLenstrulens.org
An open-source tool that traces AI steps and evaluates agents, retrieval apps, and summaries.
- 10 L
LangSmithlangchain.comA platform for building and monitoring agents, with evaluations of test data and live use.
- 11 B
Braintrustbraintrust.devA platform for reviewing agent activity and measuring latency, cost, and quality.
- 12 PPortkeyportkey.ai
A platform with one API for AI models and configurable routing, retries, and failovers.
- 13 FA
Fiddler AIfiddler.aiA platform for evaluating AI in development and monitoring models and agents in production.
- 14 G
Galileogalileo.aiA platform for evaluating and monitoring GenAI apps, with tools for experiments and guardrails.
- 15 MA
Maxim AIgetmaxim.aiMaxim is an end-to-end evaluation and observability platform for simulating, evaluating, and monitoring AI agents.
- 16 APArize Phoenixarize.com
A free tool for tracing and evaluating AI apps, with prompt testing and experiment comparisons.
- 17 CA
Confident AIconfident-ai.comA platform for evaluating AI apps and tracing production activity, with alerts when quality drops.
- 18 H
HoneyHivehoneyhive.aiA platform for tracing agents and evaluating live traffic, with drift detection and alerts.
- 19 H
Heliconehelicone.aiA platform that logs and analyzes AI requests, with model routing, failovers, and caching.
- 20 RFRespan (formerly Keywords AI)respan.ai
An AI gateway with tracing, evaluations, and security testing that routes requests to over 1,000 models.
- 21 AAArize AXarize.com
A platform for improving agents with trace investigations, evaluations, and swarm monitoring.
- 22 P
PromptLayerpromptlayer.comA prompt editor and registry with versioning, evaluations, observability, and agent tracing.
- 23 OOpenLLMetrytraceloop.com
An open-source monitor that traces LLM app activity and exports it to Traceloop or another observability stack.
Places #1, #2, #3, #4, #5 on this board are open to sponsors. Take #1 · Other ways to advertise
How to choose
Which platforms does it need to run on?
Check whether you need an API, a self-hosted setup or a web app. The platform marks also cover Linux, macOS and Windows.
An API lets software connect through a defined interface. Self-hosting means running the tool on infrastructure you manage.
What should it track in my AI app?
Match the tool to what you need to see: LLM traces, token costs, output evaluations, latency or failures.
A trace records steps in an AI app's request and response. Output evaluations help assess the responses; latency metrics show response time.
How can I compare free and paid plans?
Check the free-plan mark, any trial, the paid starting price and free-plan retention. Retention tells you how long free-plan records remain available.
Published examples include OpenLIT, listed as free, and Agenta, with a free plan and paid pricing from $29 per month. Prices and plan terms can differ, so check the current offer before deciding.
Do I need an open-source tool or public API?
Check the open-source and API marks against how you plan to run and connect the tool. If you need to read or change the code, review the tool's license.
A public API can connect the observability tool to other software. Confirm the details for the specific tool before choosing.
Questions
What does an AI LLM observability tool do?
It helps track AI app activity, including traces, token costs, output evaluations, latency and failures.
How do I compare LLM observability tools?
Compare the tracking features you need, platform support, free-plan retention, paid starting price, and API, open-source and free-tier marks.
Can I self-host an LLM observability tool?
Self-hosted is a listed platform to compare. Check the individual tool's platform details to confirm its setup.
Are there free LLM observability tools?
The board includes free-plan and free-trial information. Check a tool's current plan terms, retention and paid starting price.
What is an LLM trace?
A trace records steps in an AI app's request and response, helping you see what happened during a run.
Some top positions on this board can be paid placements, standing where they were sold. The facts on each platform are not for sale. How we rank · Disclosure