> ## Documentation Index
> Fetch the complete documentation index at: https://warpllm.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

# Model routing

> How provider/model strings resolve, what the registry guarantees, and why unknown names are errors.

Every request names a model as `provider/model`. That string is the whole
routing decision: it picks the provider, its base URL, the environment variable
holding its key, and the name sent upstream.

```json theme={null}
{ "model": "openai/gpt-5-nano" }
```

<CardGroup cols={3}>
  <Card title="Provider" icon="building">
    `openai` — a host, its base URL, and one API-key variable.
  </Card>

  <Card title="Model" icon="cube">
    `gpt-5-nano` — a routable name under that provider.
  </Card>

  <Card title="Upstream name" icon="paper-plane">
    What actually goes on the wire, which may differ from warpllm's alias.
  </Card>
</CardGroup>

## The prefix is required

warpllm matches the **whole string** against its registry. A bare `gpt-5-nano`
does not resolve, and neither does a name the registry has never heard of.

<Warning>
  An unlisted model is an error, not a guess. warpllm will not forward an
  unknown name to some upstream default and let you discover the routing on
  your bill.
</Warning>

```python theme={null}
client.chat_completions({"model": "gpt-5-nano", ...})
# warpllm.BadRequestError: invalid model string 'gpt-5-nano':
# no registered model spec
```

There is no wildcard either. A registry entry of `openai/*` would register a
model literally named `*`.

## Switching providers

Because the request shape does not change, moving between providers is a string
edit plus the key that provider needs.

<CodeGroup>
  ```python Python theme={null}
  # OpenAI — needs OPENAI_API_KEY
  client.chat_completions({"model": "openai/gpt-5-nano", "messages": messages})

  # DeepSeek — needs DEEPSEEK_API_KEY
  client.chat_completions({"model": "deepseek/deepseek-v4-flash", "messages": messages})

  # Anthropic via OpenRouter — needs OPENROUTER_API_KEY
  client.chat_completions({"model": "openrouter/anthropic/claude-sonnet-4", "messages": messages})
  ```

  ```ts TypeScript theme={null}
  // OpenAI — needs OPENAI_API_KEY
  await client.chatCompletions({ model: 'openai/gpt-5-nano', messages })

  // DeepSeek — needs DEEPSEEK_API_KEY
  await client.chatCompletions({ model: 'deepseek/deepseek-v4-flash', messages })

  // Anthropic via OpenRouter — needs OPENROUTER_API_KEY
  await client.chatCompletions({ model: 'openrouter/anthropic/claude-sonnet-4', messages })
  ```
</CodeGroup>

A model name may contain further slashes after the provider prefix —
`openrouter/anthropic/claude-sonnet-4` is the OpenRouter provider serving the
`anthropic/claude-sonnet-4` slug.

## Claude over Anthropic's own protocol

`anthropic/` models go to `api.anthropic.com` directly, and they are the first
on the roster reached over a protocol that is not chat completions. Nothing
changes in your code: you send the same request shape and read the same reply
shape, and warpllm translates both ways.

```python Python theme={null}
# Claude direct — needs ANTHROPIC_API_KEY
client.chat_completions({"model": "anthropic/claude-opus-5", "messages": messages})

# The same model through OpenRouter — needs OPENROUTER_API_KEY
client.chat_completions({"model": "openrouter/anthropic/claude-sonnet-4", "messages": messages})
```

Both routes work. Direct is one hop fewer and bills through Anthropic; the
OpenRouter entry stays registered and unchanged.

Two differences from a chat-completions provider are worth knowing:

<AccordionGroup>
  <Accordion title="max_tokens is always sent" icon="ruler">
    Anthropic requires it and chat completions does not, so warpllm falls back
    to the model's published output ceiling when you name none. Your own value
    always wins. A model whose ceiling is undocumented and a request naming no
    `max_tokens` is refused rather than given an invented default — truncating
    a reply at a length nobody asked for looks like the model stopping early.
  </Accordion>

  <Accordion title="temperature, top_p and top_k are rejected by newer models" icon="triangle-exclamation">
    Anthropic deprecated all three on Claude Opus 4.7 and later: a non-default
    value returns a 400. warpllm forwards what you write rather than dropping
    it, because dropping a parameter would change your request instead of
    translating it — so the refusal is Anthropic's, with Anthropic's message.
    Omit them for those models and steer with the prompt.
  </Accordion>

  <Accordion title="Controls Anthropic cannot express are refused, not dropped" icon="ban">
    Everywhere else warpllm forwards what you write and lets the provider judge
    it. On a translated route the provider never receives the field, so it
    cannot judge it — and a silent drop would answer a different request than
    the one you sent. `n: 2` would come back as a single choice, which looks
    exactly like a model that ignored you.

    So these are refused with an `InvalidInput` error naming the field:

    | Field | Refused when |
    | - | - |
    | `n` | greater than 1 |
    | `frequency_penalty`, `presence_penalty` | non-zero |
    | `logprobs` | `true` |
    | `top_logprobs` | greater than 0 |
    | `logit_bias` | non-empty |
    | `seed` | any value |

    The value your SDK fills in when you asked for nothing — `n: 1`, zero
    penalties, `logprobs: false`, an empty bias map — is not refused, so an
    ordinary request carrying them still routes.

    `top_k` and `parallel_tool_calls` are NOT on this list: Anthropic has an
    equivalent for each, so warpllm translates them instead of refusing them.
  </Accordion>

  <Accordion title="Extended thinking cannot be combined with tool results" icon="brain">
    Anthropic requires the assistant turn holding a tool call to open with its
    own signed `thinking` block, and a chat-completions reply has nowhere to
    carry one — so the request that sends your tool results back cannot be
    built. warpllm refuses it with a `NotImplemented` error rather than letting
    Anthropic answer it with a 400.

    A single-turn tool call with thinking works, and so does
    thinking-without-tools; it is sending the results back that stops. Turn
    thinking off for a tool loop, or run the loop on a chat-completions
    provider. Tracked in
    [#101](https://github.com/warpllm/warpllm/issues/101).

    The thinking itself does reach you, streamed or whole, under
    `reasoning_content` — the same field DeepSeek and vLLM use. Its signature
    does not, because chat completions has no field for one, and that missing
    signature is exactly what the tool-result turn would need.
  </Accordion>
</AccordionGroup>

## What the registry records

The registry lives at
[`crates/warpllm/src/registry/specs.yaml`](https://github.com/warpllm/warpllm/blob/main/crates/warpllm/src/registry/specs.yaml)
and is compiled into the binary. Each model entry carries:

<ResponseField name="supported_apis" type="list" required>
  Every surface the model serves. A model serves exactly what it lists — one it
  does not list is refused before a request goes out.
</ResponseField>

<ResponseField name="model" type="string">
  The name sent upstream. Defaults to the key's last segment; set explicitly
  when warpllm's alias differs from the provider's own name.
</ResponseField>

<ResponseField name="capabilities.max_input_tokens" type="integer">
  The provider's published context window, recorded verbatim.
</ResponseField>

<ResponseField name="capabilities.max_output_tokens" type="integer">
  The provider's published output ceiling, where one exists.
</ResponseField>

<ResponseField name="capabilities.max_concurrent_requests" type="integer">
  A documented per-model concurrency ceiling, where the provider publishes one.
</ResponseField>

<ResponseField name="deprecation_date" type="string">
  `YYYY-MM-DD`, the day the provider stops serving the model, when one has been
  announced. Nothing acts on it — an entry keeps resolving past its date.
</ResponseField>

<Note>
  Every optional field is genuinely optional, and an absent limit means the
  provider does not publish one — not that the limit is unlimited. warpllm
  omits a number rather than guessing it.
</Note>

## Reading capabilities at runtime

Rust callers can look a spec up directly, which is useful for sizing prompts
against the routed model's window.

```rust theme={null}
use warpllm::fetch_model;

let (provider, spec) = fetch_model("openai/gpt-5-nano")?;

println!("{} sends this upstream as {}", provider.name(), spec.model());
if let Some(window) = spec.capabilities().max_input_tokens() {
    println!("context window: {window}");
}
```

`fetch_model` returns the provider spec and the model spec as a pair, or
[`Error::InvalidModel`](/concepts/errors) if the string does not resolve.

## Surfaces

A surface name is `<protocol>_<endpoint>`, which is how the registry records
the wire format a model speaks.

| Surface | Meaning | Status |
| - | - | - |
| `openai_compat_chat_completions` | One request, one whole reply | Implemented |
| `openai_compat_chat_completions_stream` | The same, as server-sent events | Implemented |
| `openai_compat_responses` | OpenAI's Responses API | Not implemented |

## Spreading load, retrying, and failing over

A request naming one `model` goes to exactly that model, and a failure comes
back to you to decide about. There is no retry loop yet, so the error taxonomy
is what lets you write your own policy — see [Errors](/concepts/errors).

**Failover** is the other shape: name an ordered list in `models` instead of
`model`, and warpllm tries each in turn until one answers.

```python theme={null}
client.chat_completions({
    "models": ["anthropic/claude-sonnet-5-5", "openai/gpt-6-sol"],
    "messages": [{"role": "user", "content": "Hello"}],
})
```

* Up to 8 candidates, each one validated before anything is sent. Duplicates
  are dropped, and an empty list is an error. `model` and `models` are
  mutually exclusive.
* A failure scoped to one provider moves on: network, rate limit, overload, a
  5xx, model not found, and a rejected or exhausted key — a dead key on one
  provider should not block a healthy one on another. A failure about the
  request itself, such as `InvalidInput`, stops the chain. So does `Decode`:
  the provider already answered 200 and billed, and retrying would buy a second
  completion.
* The whole chain shares one deadline, the client's timeout. Running out is
  `DeadlineExceeded` (504); every candidate failing is `CandidatesExhausted`
  (503), whose body lists each attempt and its error.
* A stream fails over until its first chunk arrives. After that it is
  committed, and a failure is the stream's to report.
* `BalancedClient` refuses a request carrying `models`, rather than balancing
  over one list and ignoring the other.

**Load balancing** shipped in 0.5.0. `BalancedClient`
spreads requests across a weighted set of `provider/model` pairs using smooth
weighted round-robin, rewriting the request's `model` to the candidate it
picked — so the name you pass acts as a group, and each candidate is a concrete
route within it.

```rust theme={null}
use warpllm::{BalancedClient, Client, ClientConfig};

let client = Client::new(ClientConfig::default())?;
let balanced = BalancedClient::new(&client, &[
    ("openai/gpt-5.6", 3),
    ("deepseek/deepseek-v4-pro", 1),
])?;
```

The candidate set is fixed at construction, and each selected candidate still
goes through the same admission checks a direct call does. It is Rust-only for
now — neither binding exposes it, and the gateway has no flag for it.

Load balancing is tracked in
[issue #28](https://github.com/warpllm/warpllm/issues/28).

## Next

<CardGroup cols={2}>
  <Card title="Model roster" icon="cube" href="/models/overview">
    Every routable model and its limits.
  </Card>

  <Card title="Add a model" icon="plus" href="/models/add-a-model">
    The registry edit, and the rules its tests enforce.
  </Card>
</CardGroup>


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.