Envoy AI Gateway

Overview

Previously called Envoy AI Gateway, Agent Router keeps the same code, maintainers, and APIs after becoming an Agentic AI Foundation project. Its OpenAI-compatible interface fronts hosted and self-hosted models together with MCP servers, giving developers one access pattern while infrastructure teams manage providers behind it.

Integrations cover OpenAI, Azure OpenAI, Google Gemini, Vertex AI, AWS Bedrock, Mistral, Cohere, Groq, Together AI, DeepInfra, DeepSeek, Hunyuan, SambaNova, Grok, and Anthropic. That breadth is useful for multi-provider architectures and fallback strategies; it does not make Agent Router an inference engine or model vendor. Founded in 2024 and licensed under Apache 2.0, the project directs support to its community Discord and GitHub issues.

Key features

Traffic and policy controls

Model virtualization presents a consistent model-facing layer, while provider failover can keep requests moving when an upstream is unavailable. Token quotas and usage-based rate limits help teams govern shared consumption, and budget controls put a practical boundary around spending. Usage analytics supports attribution and operational review. These controls are valuable when several products or groups use the same gateway; a small application needing only a thin proxy may find the additional control-plane scope unnecessary.

MCP and inference routing

The MCP gateway multiplexes servers, routes tools, handles OAuth authentication, and supplies observability. That makes it a stronger fit for agent platforms than a gateway limited to model completion calls. InferencePool support adds intelligent routing and load balancing for inference endpoints, giving teams a way to distribute traffic across available backends.

Credentials and observability

Upstream authentication secures connections to AI services while keeping provider credentials out of application workloads. Central management includes short-lived credential flows for AWS Bedrock, Azure OpenAI, and GCP Vertex AI, reducing the need to distribute long-lived secrets. Metrics collection and monitoring cover AI workloads, complementing the gateway's quotas and analytics.

Pricing

Open-source project — free. This is self-managed software, with no paid tier described. The local CLI is included, and Kubernetes deployments remain part of the same open-source offering. The price advantage is clearest for teams willing to operate the gateway themselves; infrastructure, Kubernetes, Envoy Gateway, and engineering time are still your responsibility. There are no published seat, request, or renewal charges attached to this plan.

Platforms

Agent Router is available as an API and self-hosted deployment. The aigw run CLI currently supports Linux and macOS and can run as a standalone local proxy. Production deployment uses Kubernetes with Envoy Gateway. The compatibility matrix calls for Agent Router v1.2.x with Kubernetes v1.32 or later, Envoy Gateway v1.9.2 or later, and Gateway API v1.6.1. Hosted services are linked as alternatives from other providers rather than being this project's managed plan.

Who it's for

This is aimed at platform and infrastructure teams serving multiple applications, model providers, or MCP tools. Choose it when centralized credentials, quotas, usage attribution, budget limits, failover, and provider portability are requirements. It is less suitable for teams that cannot operate Kubernetes in production, need a managed gateway with vendor support, or only require a simple single-provider endpoint.

Pros and cons

• Pros: OpenAI-compatible access spans hosted, self-hosted, and MCP workloads.
• Pros: Broad provider integrations and failover support reduce dependence on one upstream.
• Pros: Quotas, rate limits, budgets, analytics, and centralized credentials support accountable shared usage.
• Pros: Apache 2.0 licensing and self-hosting preserve deployment control.
• Cons: Production requires Kubernetes, Envoy Gateway, and specified dependency versions.
• Cons: Linux and macOS are the only supported local CLI platforms.
• Cons: Operating the control plane and upstream integrations remains the customer's job.

Alternatives

Browse the list of LLM gateways for a wider comparison. Pick Kong Gateway instead when you want a paid gateway with a free trial, web support, and a 30-day trial period. TensorZero is a self-hosted, open-source LLMOps option for teams prioritizing that platform's workflow. Choose Eden AI for a web API gateway with no subscription and provider charges plus a 5.5% platform fee on purchased credits. Helicone suits teams wanting an open-source, freemium service with a hosted hobby tier. LangSmith is the better fit for tracing-oriented teams comfortable with per-seat pricing. LiteLLM offers a self-hosted open-source gateway with 100+ providers, virtual keys, spend tracking, budgets, and rate limits. LLM Gateway provides a freemium hosted route across 200+ models and 40+ providers. Manifest is an alternative for agent routing with a free cloud allowance and unlimited self-hosted use.

Verdict

Choose Agent Router if your team needs an open, self-managed front door for many model providers and MCP tools, with credentials, failover, quotas, budgets, and attribution under central control. Look elsewhere if you want a fully managed service or cannot meet its Kubernetes and Envoy Gateway production requirements.