L

Prompting Tool

LangSmith: AI Agent Testing and Monitoring

langchain.com · checked 15 Oct 2026

LangSmith is a commercial platform for building, evaluating, deploying, and monitoring AI agents.

How it plugs in

—Public API
NoOpen source
YesFree tier

Signal on Prompting Tool6.7#2 of 25

LangSmith helps teams trace agent runs, test prompts and models, evaluate production interactions, and deploy agent servers. It accepts data through an SDK or REST API and supports Python, TypeScript, Go, and Java SDKs. A free Developer plan is available; paid use starts at $39 a month, with usage billed separately.

LangSmith home page

Review

In a line

LangSmith is a commercial, framework-agnostic platform for building, testing, deploying, monitoring, and improving AI agents.

What it does

Tracing records agent conversations step by step, including prompts, model calls, tools, errors, response times, and token use. Monitoring dashboards show usage, latency, errors, costs, and feedback scores, with webhook and PagerDuty alerts.

Teams can run offline evaluations on curated datasets or score live production interactions. Evaluators can be human reviewers, rule-based checks, an LLM acting as a judge, pairwise comparisons, or custom Python and TypeScript code.

The Playground compares prompt versions and model providers, while LangSmith chat can improve prompts automatically. Annotation queues assign runs to subject-matter experts for feedback, prompt changes, or dataset updates. Deployment offers managed or self-hosted agent servers with streaming, scheduled jobs, authentication, MCP support, and more than 30 API endpoints.

How it connects

LangSmith accepts traces through its SDK or API from LangGraph, custom Python, and other application stacks. It connects evaluation workflows with pytest, Vitest, and GitHub workflows for automated checks in CI/CD pipelines.

SDKs are available for Python, TypeScript, Go, and Java, with native tracing for popular agent frameworks and OpenTelemetry. A REST API and command-line tool manage projects, traces, runs, datasets, evaluators, experiments, threads, and sandboxes.

The public SDK repository uses the MIT license, but the LangSmith platform is proprietary. It runs on web, macOS, Linux, and Windows, with cloud, hybrid, or self-hosted hosting depending on the plan. Enterprise self-hosting or hybrid installations run on Kubernetes in AWS, Google Cloud, or Azure.

Good fit

LangSmith suits teams building AI agents that need detailed run records, repeatable evaluations, prompt comparisons, cost tracking, and production monitoring.

It can fit organizations using different agent frameworks because traces can come through the SDK or API. Teams that need private deployment can choose Enterprise hosting options, with cloud data in GCP US or EU regions and self-hosted data kept in the customer's virtual private cloud.

Watch for

The hosted and self-hosted platform is proprietary, and self-hosted use requires a LangChain license key. Enterprise is the plan with self-hosted or hybrid deployment, custom SSO, access controls, and a support service-level agreement.

Base traces are kept for 14 days. Extended traces last 180 days and cost extra. Agent Server deployments require PostgreSQL and Redis; self-hosted installations may use customer-managed versions.

Check whether your team needs the listed compliance coverage—SOC 2 Type II, ISO 27001:2022, GDPR, or HIPAA—and whether usage-based charges fit your budget.

Pricing

The Developer plan is free, includes one seat and 5,000 base traces a month, and has community support. Usage is billed separately on a pay-as-you-go basis.

Plus costs $39 a month per seat, includes 10,000 base traces a month, unlimited seats, and one free small serverless deployment. Usage is also pay-as-you-go.

Enterprise pricing is on request and is invoiced annually upfront. It adds self-hosted or hybrid deployment, custom SSO and access controls, and a support service-level agreement, with usage billed separately.

Verdict

Choose LangSmith if your team needs one place to inspect agent behavior, compare prompts, run structured evaluations, and watch costs after launch. Its API, SDKs, framework-neutral tracing, and CI workflow support make it suitable for mixed application stacks.

The free plan offers a way to start with limited trace volume. Paid plans make more sense when you need more traces, multiple seats, deployment choices, or formal support. Avoid committing before checking retention fees, database requirements, proprietary licensing, and whether Enterprise is needed for your hosting or access-control requirements.

Our take

LangSmith is a practical choice for teams that treat AI agents as software requiring repeatable tests and ongoing monitoring. It combines tracing, evaluation, prompt work, feedback review, and deployment in one service, while accepting data from several languages and frameworks. The free plan covers a small starting workload, and Plus adds seats and trace capacity for a monthly fee. Larger organizations should review Enterprise pricing, hosting, retention charges, database requirements, and the proprietary license before choosing it.

  • Step-by-step traces include prompts, tools, errors, latency, and token use.
  • Offline and live evaluations support human, rule-based, judge, pairwise, and custom checks.
  • SDKs, REST API, CLI, and CI links support several application stacks.
  • Managed, hybrid, and self-hosted deployment options are available.
  • Free Developer plan and a $39-a-month Plus plan are offered.
  • The platform is proprietary and self-hosting requires a LangChain license key.
  • Base traces stay for 14 days; 180-day retention costs extra.
  • Agent Server deployments require PostgreSQL and Redis.
  • Enterprise pricing is on request and billed annually upfront.
  • Usage charges are separate from plan fees.
  • Pick it ifTeams building and monitoring AI agents; Projects needing evaluations and prompt comparisons; Organizations requiring hybrid or self-hosted deployment
  • Skip it ifYou need a fully open-source platform.; You cannot support PostgreSQL and Redis for agent servers.; You need predictable pricing without separate usage charges.

Pricing

Developer is free with one seat and 5,000 base traces per month. Plus is $39 a month per seat with 10,000 base traces, unlimited seats, and one free small serverless deployment. Enterprise is price on request. All plans use pay-as-you-go usage billing.

  • Developer$0 per seat per month; usage pay-as-you-go · 1 seat · 5,000 base traces/month · community supportFree
  • Plus$39 per seat per month; usage pay-as-you-go · 10,000 base traces/month · unlimited seats · 1 free small serverless deployment$39/mo
  • Enterpriseinvoiced annually upfront; usage pay-as-you-go · self-hosted or hybrid deployment · custom SSO, ABAC and RBAC · support SLAOn request

As langchain.com lists them, checked 15 Oct 2026.

Questions

Does LangSmith have a free plan?

Yes. Developer is free, with one seat, 5,000 base traces per month, and community support. Usage is billed pay-as-you-go.

What does LangSmith cost after the free plan?

Plus costs $39 a month per seat and includes 10,000 base traces per month, unlimited seats, and one free small serverless deployment. Usage costs extra.

Can LangSmith be self-hosted?

Enterprise offers cloud, hybrid, or self-hosted deployment. Self-hosted installations run on Kubernetes in AWS, Google Cloud, or Azure and require a LangChain license key.

Which programming languages have LangSmith SDKs?

The listed SDKs are for Python, TypeScript, Go, and Java. Traces can also arrive through the REST API from LangGraph, custom Python, or other stacks.

What does LangSmith track?

Tracing records prompts, model calls, tools, errors, latency, and token use. Dashboards track usage, latency percentiles, error rates, cost, and feedback scores.

How long are traces retained?

Base traces are retained for 14 days. Extended traces are retained for 180 days for an additional charge.

Is LangSmith open source?

The public SDK repository is MIT-licensed, but the hosted and self-hosted LangSmith platform is proprietary.