Provider guide

Groq through one OpenAI-compatible gateway

Route Groq’s fast inference through the same OpenAI-compatible endpoint as everything else — with fallback, guardrails, and residency, plus Whisper transcription on the same surface.

Groq serves an OpenAI-compatible API, so Routeplane’s adapter is a faithful passthrough with the gateway’s reliability and governance wrapped around it. Groq’s speed makes it a natural primary in a latency-first routing strategy.

Put Groq at the head of a fallback chain and let the gateway spill over to a slower-but-broader provider when Groq is saturated.

Drop in your OpenAI client

Point your existing OpenAI SDK at the gateway and add two headers — your virtual key and the provider. Nothing else about your code changes.

bashcurl

curl https://api.routeplane.ai/v1/chat/completions \
  -H "content-type: application/json" \
  -H "x-routeplane-api-key: rp_your_gateway_key" \
  -H "x-routeplane-provider: groq" \
  -d '{"model":"llama-3.3-70b","messages":[{"role":"user","content":"Hello!"}]}'

pythonopenai SDK

import openai

client = openai.OpenAI(
    api_key="rp_your_gateway_key",
    base_url="https://api.routeplane.ai/v1",
    default_headers={
        "x-routeplane-api-key": "rp_your_gateway_key",
        "x-routeplane-provider": "groq",   # route to Groq
    },
)

resp = client.chat.completions.create(
    model="llama-3.3-70b",
    messages=[{"role": "user", "content": "Hello!"}],
)
print(resp.choices[0].message.content)

typescriptopenai SDK

import OpenAI from 'openai';

const client = new OpenAI({
  apiKey: 'rp_your_gateway_key',
  baseURL: 'https://api.routeplane.ai/v1',
  defaultHeaders: {
    'x-routeplane-api-key': 'rp_your_gateway_key',
    'x-routeplane-provider': 'groq', // route to Groq
  },
});

const completion = await client.chat.completions.create({
  model: 'llama-3.3-70b',
  messages: [{ role: 'user', content: 'Hello!' }],
});
console.log(completion.choices[0]?.message.content);

Or use the Routeplane SDK

The Routeplane SDKs subclass the official OpenAI clients, wire up the x-routeplane-* headers for you, and add typed access to routing metadata and the non-OpenAI surfaces.

bashinstall

pip install routeplane

pythonrouteplane SDK

from routeplane import Routeplane

client = Routeplane(
    api_key="rp_your_gateway_key",
    provider="groq",   # route to Groq
)

resp = client.chat.completions.create(
    model="llama-3.3-70b",
    messages=[{"role": "user", "content": "Hello!"}],
)
print(resp.choices[0].message.content)

bashinstall

npm i @routeplane/sdk

typescript@routeplane/sdk

import { Routeplane } from '@routeplane/sdk';

const client = new Routeplane({
  apiKey: process.env.ROUTEPLANE_API_KEY!, // rp_...
  provider: 'groq',                 // route to Groq
});

const completion = await client.chat.completions.create({
  model: 'llama-3.3-70b',
  messages: [{ role: 'user', content: 'Hello!' }],
});
console.log(completion.choices[0]?.message.content);

Add a fallback chain

Try Groq first for its low latency; fall back to OpenAI. Make the provider header a comma-separated list and the gateway walks it in order, skipping any provider whose circuit is open.

bashcurl

curl https://api.routeplane.ai/v1/chat/completions \
  -H "content-type: application/json" \
  -H "x-routeplane-api-key: rp_your_gateway_key" \
  -H "x-routeplane-provider: groq,openai" \
  -d '{"model":"llama-3.3-70b","messages":[{"role":"user","content":"Hello!"}]}'

Groq on the gateway

The adapter supports chat completions, native streaming, and Whisper audio transcription and translation. Groq has no first-party embeddings endpoint, so embeddings requests return a clean 422 — route them to a provider that offers them (e.g. OpenAI or Cohere).

Whisper transcription

Groq hosts Whisper, so /v1/audio/transcriptions and translations route through this adapter — fast speech-to-text on the same gateway as your chat models.

Latency-first routing

Pair Groq with x-routeplane-strategy: latency and the gateway orders candidates by observed latency, keeping Groq first while it stays fast.

Groq model pricing

List prices for the Groq models you can call through the gateway — click any model for its per-token cost page.

llama-3.3-70b llama-3.1-8b-instant gemma2-9b-it

Sovereign routing

Sovereign routing works with Groq the same way it works everywhere: add x-routeplane-residency to a request, and when the gateway classifies regulated personal data in it, only providers resident in the requested region stay eligible — a hard constraint that overrides the provider header. Declare where Groq runs by setting GROQ_REGION.

Keep reading

Quickstart — send your first request in a few minutes.
Providers & routing strategy — how eligibility, fallback, and strategies work.
Python SDK and TypeScript SDK — the full typed clients.
All providers — every model provider behind the gateway.

Frequently asked questions

Is Routeplane a drop-in replacement for calling Groq directly?

Yes. The gateway is OpenAI-compatible, so switching is a base-URL change — point your existing OpenAI SDK at https://api.routeplane.ai/v1 and set the x-routeplane-provider header to groq. You keep your code and gain automatic fallback, in-data-plane PII guardrails, sovereign routing, and per-team cost attribution.

Which Groq models can I use through Routeplane?

Any Groq model the provider serves. See the list prices linked above for the ones we publish.

Can I enforce data residency on Groq requests?

Yes. Add the x-routeplane-residency header (for example IN) and, when a request carries regulated personal data, the gateway restricts routing to providers eligible in that region — overriding the provider header if it has to. The Groq adapter reads GROQ_REGION to declare where it is resident.

Route your first request this week.

Point your existing OpenAI-compatible client at Routeplane, set one header, and get fallback, guardrails, and sovereign routing across every provider.

Request access → Read the quickstart