The sovereign LLM gateway for every model call.
routeplane is an OpenAI-compatible gateway in front of 14 providers. It redacts PII and leaked secrets, blocks prompt injection, locks regulated data to in-region providers on every request, and attributes each token to a team, project, and currency — while a savings flywheel makes every request cheaper than the last. Adoption is a base-URL change. Neutral across clouds, sovereign by default.
Every team stitches the same four tools together.
Gateways handle traffic. Observability tools handle traces. Hyperscalers handle compliance. You're left wiring them into the request path yourself, and none of them answer "where did this data go, and what did it cost?"
"Deploy in a region" isn't enough
GDPR, DPDP, and HIPAA care about where each request goes, not where your cluster runs. Per-request enforcement is the actual requirement, and the thing nobody ships.
Token counts aren't FinOps
Finance needs chargeback by team, project, and currency, not a dashboard of token totals. Especially when the bill lands in INR but providers quote in USD.
Cost hides inside agent runs
A single agentic workflow fans out across dozens of model and tool calls. Attributing that back to one business action is a problem current tools don't even attempt.
What happens between your app and the model.
Every call flows through one pipeline. Each stage is policy-defined, and the whole path adds single-digit milliseconds.
Authenticate
A scoped virtual key resolves to a workspace, its budgets, its routing policy, and its residency rules. Provider keys never leave the vault.
Classify & guard
Deterministic, sub-microsecond guardrails redact PII and leaked secrets, block prompt injection, and tag regulated data, on the request and response.
Route
If the request carries regulated data, routing is locked to in-region providers. Otherwise it picks the fastest or cheapest model that meets your bar, with fallback.
Account & prove
Every call is attributed to a team, project, and currency, and written to a tamper-evident log, so the data-flow inventory your auditor asks for generates itself.
The open-source AI gateway that saves you tokens today and passes your CISO audit tomorrow.
Point any OpenAI SDK, Cursor, or Claude Code at it — zero code change. 14 providers, one Rust binary, no database, Apache-2.0.
Trim repeated tool output before it reaches the model
Coding agents re-send the same git diffs, grep output, and build logs every turn. Routeplane's deterministic token compression collapses that repeated tool output on the way to the provider — pure string processing, sub-millisecond, and fail-safe (a request never grows). It helps most on tool-heavy agent traces; traffic with little repeated tool output sees little change. These are input-token reductions to measure on your own workload, not a guarantee your agent's output is unchanged — so keep it on where quality holds and opt out where it doesn't.
See the method + replay harness →The Enterprise tier goes further with RTC — quality-certified context compression that is evaluated against your own traffic before it is allowed to touch it, and disabled anywhere the quality gate says no. Talk to us about RTC →
| Capability | Community Edition | Enterprise |
|---|---|---|
| OpenAI-compatible API — chat, embeddings, SSE streaming | ✓ | ✓ |
| 14 provider adapters, incl. self-hosted / local | ✓ | ✓ |
| Routing strategies, fallback chains, retries, circuit breaker | ✓ | ✓ |
| Named combos + multi-account key pools | ✓ | ✓ |
| Token compression | ✓ | ✓ |
| Exact-match response cache | ✓ | ✓ |
| Basic rate & spend limits + PII masking | ✓ | ✓ |
| Self-host — single binary, no database | ✓ | ✓ |
| Sovereign residency routing + signed audit ledger | ✗ | ✓ |
| MCP agentic-security gateway + agent governance | ✗ | ✓ |
| Advanced guardrails (webhook + ML detectors) | ✗ | ✓ |
| Semantic cache | ✗ | ✓ |
| Versioned prompt registry | ✗ | ✓ |
| FinOps export + durable telemetry retention | ✗ | ✓ |
| Multi-tenant control plane (SSO, RBAC, key issuance) | ✗ | ✓ |
Neutral and sovereign. The combination nobody else ships.
Gateways win on traffic. Hyperscalers win on compliance, but only inside their own cloud. The column that matters is the one where everyone else has a gap.
| Capability | routeplane | Portkey | LiteLLM | Bedrock / Foundry |
|---|---|---|---|---|
| Per-request residency enforcement | ✓ | ✗ | ✗ | ◐ single cloud |
| Regulated-data classification per call | ✓ | ✗ | ✗ | ◐ |
| Cross-cloud, vendor-neutral routing | ✓ | ✓ | ✓ | ✗ |
| Auto-generated audit / data-flow inventory | ✓ | ✗ | ✗ | ◐ |
| Chargeback in local currency | ✓ | ✗ | ✗ | ◐ |
| Business-process cost attribution for agents | ✓ | ✗ | ✗ | ✗ |
| Pluggable regulatory packs (GDPR · DPDP · HIPAA) | ✓ | ✗ | ✗ | ◐ |
Every request makes the next one cheaper.
Five optimizations compound automatically. No configuration, no threshold tuning. Your LLM bill drops week over week.
-
1
Route smart
Easy prompts go to cheap models. Hard ones go to the best. Per-request, automatic.
Up to 60% cost reduction on eligible traffic -
2
Compress tokens
RTK strips redundant
20–40% input token savingstool_resultblocks before they hit the meter. Deterministic and fail-safe — a request never grows. -
3
Cache hits
Identical prompts return instantly from cache. Semantic cache catches near-matches too.
$0 per cache hit -
4
Evaluate quality
LLM-as-judge scores every response. Low scores re-route to better models; high scores validate cheap-model eligibility.
Quality-gated cost optimization -
5
Compound
Each rung feeds the next. Routing data improves compression; cache hits reduce evaluation cost. The loop tightens every week.
Savings grow with usage
Without Routeplane
- Every prompt hits GPT-4o (~$10/M output tokens)
- No caching — identical prompts billed twice
- No visibility into which calls need expensive models
- Compliance blocks your launch
With Routeplane
- Easy prompts → Groq (~$0.08/M) automatically
- Cache hits → $0
- 20–40% fewer tokens via compression
- Per-request sovereign routing — ship without CISO approval
The flywheel starts on your first request. No configuration needed.
Built for developers, not dashboards.
Install the SDK, CLI, or MCP server. Ship in 30 seconds, not 30 minutes.
Two lines to production
Change base_url in any OpenAI SDK. Or install routeplane for typed headers, response metadata, and 10 resource namespaces. Works with LangChain, LlamaIndex, CrewAI, and the Vercel AI SDK.
Your gateway from the terminal
Stream completions, check logs, view costs, manage prompts — all from rp. Zero-install via npx.
AI assistants meet your gateway
33 curated tools for Claude Code, Cursor, and VS Code. Send completions, check costs, manage keys — from your AI assistant.
One sovereign control plane.
Four capabilities, one request pipeline. Each is policy-defined and works across every cloud you route to.
Sovereign routing
Define data-residency policy once; enforce it per request across any cloud. Personal data stays in-region, automatically, with the proof attached.
GDPR · DPDP · HIPAA packsMulti-currency FinOps
Budgets, hard caps, and chargeback by team and project, denominated in the currency your finance team actually reports in.
USD · INR · extensibleInline guardrails
Deterministic, pure-Rust PII & secret redaction, prompt-injection detection, and denylists on every request and response, microseconds on the hot path.
PII · secrets · injectionAgentic governance
An MCP agent gateway authorizes tool calls, traces a full agent run as one graph, detects agentic threats, and breaks the circuit before a runaway run drains a budget.
MCP gateway · approvalsSRE-grade reliability
A purpose-built data plane with predictable, low tail latency, circuit breakers per provider, and fallback that keeps serving when one goes down.
lock-free hot pathSovereignty you can prove, not just promise.
Anyone can deploy in a region. routeplane enforces residency on every individual request and generates the evidence as a byproduct, which is why a US-built gateway can't simply add it later. It's an architectural choice, made from the first request.
Data-flow inventory, on demand
Show exactly which data reached which provider and region, how much carried personal data, and how much was kept in-region, exported for audit in a click.
Tamper-evident audit log
Every configuration change and privileged action is hash-chained, so the record can be proven intact, a baseline for SOC 2 and sectoral review.
Start free, or talk to us about Enterprise.
Self-host the Community Edition in minutes — Apache-2.0, no signup. When you need sovereign residency routing, a signed audit ledger, or the MCP agent gateway, that's the Enterprise conversation.