vLLM Semantic Router

About

vLLM Semantic Router is ranked #29 of 33 in LLM gateways on Inferse. It runs on API, Linux, macOS, Self-hosted.

Compared on LLM gateways

Free plan

Yesvllm-sr.ai

Fallback routing

Novllm-sr.ai

Usage analytics

Yesvllm-sr.ai

Self-hosted deployment

Yesvllm-sr.ai

Facts

Purpose

vLLM Semantic Router is an open, programmable decision layer that selects or combines models for each call according to policy.github.com · 10 Oct 2026

Model routing

It can route requests by task, capability, modality, context length, and tool requirements across configured model backends.vllm-sr.ai · 10 Oct 2026

Routing priorities

Operators can set quality, latency, and cost priorities for model selection.vllm-sr.ai · 10 Oct 2026

Deployment choices

The documentation describes cloud, data center, edge, and enterprise hybrid deployment environments.vllm-sr.ai · 10 Oct 2026

Local and remote models

It can keep eligible sensitive or offline traffic on local models and allow selected requests to escalate to approved remote models.vllm-sr.ai · 10 Oct 2026

API compatibility

The Quickstart demonstrates requests through an OpenAI-style /v1/chat/completions endpoint.vllm-sr.ai · 10 Oct 2026

Agent use

Agent harnesses connect to the Router inference endpoint while retaining responsibility for the agent loop, tools, and task state.github.com · 10 Oct 2026

Installation

The Quickstart offers installation with curl, pip, uv, or an agent.vllm-sr.ai · 10 Oct 2026

Requirements

The Quickstart lists Linux, macOS, or WSL2, Python 3.10 or newer, and Docker, with Podman available as a Linux fallback.vllm-sr.ai · 10 Oct 2026

Open source license

The public GitHub repository lists an Apache-2.0 license.github.com · 10 Oct 2026

Security boundary

The documentation says the Router complements identity, network segmentation, secrets management, retention policy, and provider governance rather than replacing them.vllm-sr.ai · 10 Oct 2026

Deployment limits

The Router chooses a model or model pool, while the inference platform handles replica placement, batching, autoscaling, cache locality, and accelerator scheduling.vllm-sr.ai · 10 Oct 2026

Support and community

The project links to GitHub Discussions and describes maintainers as responsible for technical escalations and releases.vllm-sr.ai · 10 Oct 2026

Playground

The project provides an online Playground at app.vllm-sr.ai/playground.github.com · 10 Oct 2026

Product

vLLM Semantic Router describes itself as an open, programmable decision layer for models and compute.vllm-sr.ai · 11 Oct 2026

Routing

It selects a model backend per request using signals that can include context, intent, safety, preferences, and system state.vllm-sr.ai · 11 Oct 2026

Routing algorithms

The site lists 16 algorithms: 11 selection algorithms and 5 loopers.vllm-sr.ai · 11 Oct 2026

Deployment locations

The site says users configure where inference runs, including cloud, private, or edge environments.vllm-sr.ai · 11 Oct 2026

Model backends

The site names vLLM, SGLang, and OpenAI API among the model backends it can route to.vllm-sr.ai · 11 Oct 2026

Routing modes

It offers a Router mode for routing chat to model backends and an Engine mode that disables routing so the application owns the next step.vllm-sr.ai · 11 Oct 2026

Privacy policy

The site describes routing private domain knowledge to open models on on-premises GPUs while allowing policy-eligible requests to use external frontier models.vllm-sr.ai · 11 Oct 2026

Client protocols

The documentation lists OpenAI Chat Completions, OpenAI Responses, and Anthropic Messages as client-facing protocols.vllm-sr.ai · 11 Oct 2026

Protocol limitations

The compatibility matrix says custom free-form tools are not supported for Anthropic Messages, and audio input is supported only for Chat Completions.vllm-sr.ai · 11 Oct 2026

Security guidance

The security guide advises authenticating public inference endpoints, removing internal control and identity headers at a trusted proxy, and keeping management and storage ports private.vllm-sr.ai · 11 Oct 2026

Data retention

The security guide says replay, response cache, memory, response history, service logs, and provider logs can retain request-derived data, and recommends setting retention and deletion policies for enabled stores.vllm-sr.ai · 11 Oct 2026

Support

The site links to GitHub Discussions as a community resource.vllm-sr.ai · 11 Oct 2026

Project team

The project team page lists five maintainers responsible for project-wide technical and operational stewardship.vllm-sr.ai · 11 Oct 2026

Where it ranks on Inferse

• Best LLM Gateways in 2026#29 of 33

Sources

• github.com/vllm-project/semantic-router· checked 10 Oct 2026
• vllm-sr.ai/docs/overview/use-cases/· checked 10 Oct 2026
• vllm-sr.ai· checked 10 Oct 2026
• vllm-sr.ai/docs/installation/· checked 10 Oct 2026
• vllm-sr.ai/community/team/· checked 10 Oct 2026
• vllm-sr.ai/docs/installation/protocol-compatibilit· checked 11 Oct 2026
• vllm-sr.ai/docs/installation/security-hardening/· checked 11 Oct 2026