AgentClash

Input—per 1M tokens

Output—per 1M tokens

Context—tokens

WeightsClosed

About

AgentClash is ranked #3 of 29 in AI agent evaluation tools on Inferse. It runs on API, Self-hosted, Web. There is a free plan. Paid plans start at $49/mo.

Compared on AI agent evaluation tools

Free plan: Yesagentclash.dev
Paid from: $39/moagentclash.dev
Evaluation methods: hybridagentclash.dev
Tool-call checks: Yesagentclash.dev
Trace ingestion: Yesagentclash.dev
Safety evaluations: Yesagentclash.dev
Regression runs: Yesagentclash.dev

Facts

Purpose: AgentClash is an open-source AI-agent evaluation platform that runs agents on real tasks, scores outcomes, replays steps, and turns failures into regression tests.agentclash.dev · 1 Oct 2026
Agent evaluation: It evaluates multi-turn agents that take actions in a real sandbox and scores tool choices, cost, latency, recovery, and the final result.agentclash.dev · 1 Oct 2026
Sandboxing: Each agent runs in a fresh Firecracker microVM with an isolated filesystem and network, and the sandbox is torn down after the run.agentclash.dev · 1 Oct 2026
Providers: First-class adapters support OpenAI, Anthropic, Gemini, xAI, Mistral, and OpenRouter, with more than 300 models available through OpenRouter.agentclash.dev · 1 Oct 2026
Tools: Agents can use file I/O, data queries, HTTP, shell, and test runners, with declarative YAML challenge packs defining tools, policy, scoring, and starting state.agentclash.dev · 1 Oct 2026
Scoring: Runs combine deterministic, mathematical, behavioural, and LLM-based judges with configurable consensus aggregation and weights.agentclash.dev · 1 Oct 2026
Regression loop: When a model fails a challenge, AgentClash freezes the failing trace into a permanent test that future evaluations replay.agentclash.dev · 1 Oct 2026
Integrations: CI/CD integrations can run regression tests from GitHub Actions, a webhook, or the CLI and fail builds when correctness, cost, latency, or required evidence regresses.agentclash.dev · 1 Oct 2026
Security: API keys, database credentials, and OAuth tokens are stored in a scoped secret vault and injected at tool-call time without appearing in prompts, traces, or replays.agentclash.dev · 1 Oct 2026
Knowledge sources: Knowledge sources include PDFs, wikis, Notion, codebases, and custom APIs, with provenance attached to retrieved facts.agentclash.dev · 1 Oct 2026
Workloads: The product is positioned for coding, research, SRE, multi-step operations, codebase question answering, and support workloads.agentclash.dev · 1 Oct 2026
Open source and hosting: AgentClash is MIT licensed, can be self-hosted as a full stack, or used against the hosted backend; its CLI installs from npm as the agentclash package.agentclash.dev · 1 Oct 2026
Documentation: The public documentation covers the CLI, local stack, Fleet eval sets, datasets, regression gates, multi-turn human takeover, security stress harnesses, and runtime components.agentclash.dev · 1 Oct 2026