Inference Gateway

About

Inference Gateway is ranked #26 of 33 in LLM gateways on Inferse. It runs on API, Linux, macOS, Self-hosted, Web, Windows.

Compared on LLM gateways

Free plan

Yesdocs.inference-gateway.com

Fallback routing

Nodocs.inference-gateway.com

Usage analytics

Yesdocs.inference-gateway.com

Self-hosted deployment

Yesdocs.inference-gateway.com

Facts

Product

Inference Gateway is an open-source, cloud-native proxy that presents supported hosted and local LLM providers through one OpenAI-compatible API.docs.inference-gateway.com · 9 Oct 2026

Provider coverage

The supported providers page lists OpenAI, Anthropic, Groq, Cohere, Ollama, Ollama Cloud, DeepSeek, Cloudflare, Google, Mistral, MiniMax, Moonshot, NVIDIA, llama.cpp, Z-AI, and ElevenLabs.docs.inference-gateway.com · 9 Oct 2026

Provider switching

The gateway lets applications switch providers by changing configuration, without an application redeploy, and can route model aliases to pools of upstream deployments.docs.inference-gateway.com · 9 Oct 2026

MCP integration

MCP support discovers tools from connected servers, can connect to multiple servers, and executes tool calls through the gateway.docs.inference-gateway.com · 9 Oct 2026

Agent workflows

The product supports Agent-to-Agent coordination and describes defining agents in YAML with a CLI that generates Go, Rust, or TypeScript projects.docs.inference-gateway.com · 9 Oct 2026

Kubernetes

A Kubernetes Operator provides custom resources for managing gateways, agents, MCP servers, and chat orchestrators.docs.inference-gateway.com · 9 Oct 2026

Authentication

OIDC authentication validates JWTs from a configured identity provider, and the gateway documentation says the gateway needs no client secret for token validation.docs.inference-gateway.com · 9 Oct 2026

Observability

The gateway documents Prometheus-compatible metrics, OpenTelemetry instrumentation, structured JSON logs, and Grafana, Loki, and Jaeger examples.docs.inference-gateway.com · 9 Oct 2026

Privacy

The product page states that it has no analytics or telemetry phoning home and can be self-hosted on-premises, in the cloud, or air-gapped.docs.inference-gateway.com · 9 Oct 2026

Deployment

The documentation provides a Docker image and describes running the gateway in Docker or Kubernetes.docs.inference-gateway.com · 9 Oct 2026

Desktop platforms

The desktop app page lists Linux, macOS, and Windows assets for amd64 and arm64; it also says releases are not signed with Apple Developer or Windows code-signing certificates.docs.inference-gateway.com · 9 Oct 2026

License

The documentation states that Inference Gateway is released under the Apache-2.0 license.docs.inference-gateway.com · 9 Oct 2026

Purpose

Inference Gateway is an open-source, cloud-native proxy that provides one OpenAI-compatible API for multiple hosted and local LLM providers.docs.inference-gateway.com · 10 Oct 2026

Routing

The gateway supports model routing so a stable model alias can point to a pool of upstream deployments.docs.inference-gateway.com · 10 Oct 2026

Agent coordination

Its Agent-to-Agent support lets agents discover capabilities, delegate tasks, and stream results within a conversation.docs.inference-gateway.com · 10 Oct 2026

Footprint

The gateway is described as an approximately 10.8 MB static binary designed to scale horizontally with Kubernetes HPA.docs.inference-gateway.com · 10 Oct 2026

SDKs

Official SDKs are available for Python, TypeScript, Go, and Rust, with streaming support over Server-Sent Events.docs.inference-gateway.com · 10 Oct 2026

License and support

The project is released under Apache-2.0, with GitHub Discussions for questions and an issue tracker for bugs and feature requests.docs.inference-gateway.com · 10 Oct 2026

Where it ranks on Inferse

• Best LLM Gateways in 2026#26 of 33

Sources

• docs.inference-gateway.com· checked 9 Oct 2026
• docs.inference-gateway.com/supported-providers/· checked 9 Oct 2026
• docs.inference-gateway.com/mcp/· checked 9 Oct 2026
• docs.inference-gateway.com/authentication/· checked 9 Oct 2026
• docs.inference-gateway.com/observability/· checked 9 Oct 2026
• docs.inference-gateway.com/desktop/· checked 9 Oct 2026
• docs.inference-gateway.com/sdks/· checked 10 Oct 2026