Routeplane vs TensorZero
TensorZero was the most technically ambitious open-source LLMOps platform. Its repository was archived in June 2026, which changes the decision.
Credit first: TensorZero set a high bar. A Rust gateway with sub-millisecond published overhead, a unique in-process embedded mode (compile the gateway into your Python process via PyO3, no HTTP hop at all), and a genuinely differentiated optimization flywheel, adaptive A/B testing, dynamic in-context learning, and fine-tuning pipelines fed by a ClickHouse observability store. Nothing else in open source matched that experimentation depth.
In June 2026 the company wound down: the GitHub repository was archived on June 12, 2026 and is read-only, no further updates, issue resolution, or security patches from the team. The code remains available under Apache-2.0. For teams that bet on it, the practical question is what a maintained path forward looks like. Routeplane’s answer: an actively developed Rust, no-GC data plane that starts from a single container with no ClickHouse or Postgres to operate, plus the sovereignty, guardrails, and MCP governance TensorZero never targeted.
On the flywheel itself we are further along than a quick look suggests, and it is worth being precise rather than vague. Shipped today: durable feedback keyed to an episode, named metrics, and offline evaluation runs with deterministic evaluators over your own saved traffic. Built but granted per tenant: best-of-N inference-time selection, which multiplies spend by N and so is a deliberate operator decision rather than a plan inclusion. Planned: adaptive bandit allocation. Dropped on purpose: dynamic in-context learning. We are not claiming parity — TensorZero’s adaptive experimentation went further than ours does today, and the table below marks every row where they are ahead.
Feature-by-feature
| Capability | routeplane | TensorZero |
|---|---|---|
| Actively maintained | ✓ | ✗ repo archived Jun 2026 |
| OpenAI-compatible API | ✓ | ✓ |
| Rust, no-GC data plane | ✓ | ✓ |
| In-process embedded mode (no HTTP hop) | ✗ network proxy | ✓ PyO3 |
| Durable feedback keyed to an episode | ✓ | ✓ |
| Offline evaluation runs over saved traffic | ✓ deterministic evaluators | ✓ |
| Best-of-N inference-time selection | ◐ built, enterprise grant only | ✓ |
| Adaptive A/B (bandit) allocation | ✗ planned | ✓ |
| Dynamic in-context learning | ✗ dropped by choice | ✓ DICL |
| Infrastructure needed to start | ✓ single container, scale-to-zero | ◐ ClickHouse (+Postgres) |
| Your prompts and outputs stay in your estate | ✓ no analytical cluster | ◐ your DB, but ClickHouse required |
| Per-request regulated-data classification | ✓ | ✗ |
| Region-locked routing enforced per request | ✓ | ✗ |
| PII / secret redaction in the data plane | ✓ | ✗ |
| MCP / agent tool-call governance | ✓ | ✗ |
| Multi-currency FinOps & chargeback | ✓ | ✗ USD only |
Which should you choose?
Choose routeplane when
- You need a vendor that ships updates and security patches, the archival is the deciding fact for most teams.
- You want a feedback-and-savings flywheel without standing up ClickHouse and Postgres first.
- Sovereignty, guardrails, or agent governance are requirements, TensorZero never targeted them.
- Scale-to-zero economics matter: one small container, near-zero idle cost.
Choose TensorZero when
- You already run TensorZero, it works, and you’re prepared to maintain an archived Apache-2.0 codebase yourself.
- The embedded PyO3 in-process mode is essential and a network proxy is unacceptable.
- You need adaptive A/B allocation or dynamic in-context learning, and can operate ClickHouse. We plan the first and have deliberately dropped the second.
- You want fine-tuning driven from the gateway itself. We export curated datasets to your MLOps stack instead, and do not send your data to a provider for training.
Migrating to routeplane
TensorZero exposes an OpenAI-compatible endpoint, so apps using it that way migrate with a base-URL change to https://api.routeplane.ai/v1. Apps using the embedded client or inference-level experimentation APIs will need a small refactor to the OpenAI shape, that’s the honest cost of leaving the embedded mode.
Frequently asked questions
Is TensorZero discontinued?
The tensorzero/tensorzero GitHub repository was archived on June 12, 2026 and is read-only; the founders announced the company was winding down. The code remains available under Apache-2.0, but there is no maintaining team shipping updates or security patches.
Does Routeplane match TensorZero’s optimization flywheel?
On the middle of it, yes; at the top, not yet. Shipped: durable feedback keyed to an episode, named metrics, and offline evaluation runs with deterministic evaluators over your saved traffic — none of which need a ClickHouse deployment. Best-of-N inference-time selection is built but granted per tenant rather than included in a plan, because fanning out to N candidates multiplies spend by N. Not shipped: adaptive bandit allocation (planned) and dynamic in-context learning (dropped on purpose — retrieving one tenant’s content into another’s prompt is the wrong default for a regulated buyer). An LLM-as-judge scorer is built and calibrated but is not running in production yet: it re-transmits stored content to a model provider, so it waits on our data-custody review rather than on the code being finished.
What happens to my data in Routeplane’s flywheel?
It stays in storage you control. TensorZero already told operators to keep inferences and feedback in their own database, so this is not an idea we invented — but TensorZero additionally required a ClickHouse cluster to query it, alongside PostgreSQL, Valkey and S3. Routeplane carries the same instinct one step further: your data, and no analytical cluster. The honest trade is that batch cohort selection over file manifests is materially worse than interactive OLAP for ad-hoc analysis. Dataset curation and eval runs are batch work that tolerates seconds, so we took that trade deliberately.
Why does Routeplane not offer an embedded in-process mode?
Routeplane is deliberately a network data plane: governance, residency, and audit have to sit at a boundary all clients pass through, in every language. An in-process library trades that away for latency. TensorZero’s PyO3 mode was a real innovation for the latency-first case, a concession we make plainly.
See the residency header come back true.
Point your existing OpenAI-compatible client at routeplane and route your first sovereign request this week.