Connecting a client

The HTTP gateway has landed on main but is not part of a published release. To run it today you build it from source or from the Dockerfile in the repository root. Flags, endpoints, and the error envelope may still change before it ships.

The gateway speaks the OpenAI chat-completions API, so any client that can take a base URL works. Two things change: the base URL, and the provider/ prefix on the model string.

Official OpenAI SDKs

from openai import OpenAI

client = OpenAI(base_url="http://localhost:8080/v1")

completion = client.chat.completions.create(
    model="openai/gpt-5-nano",
    messages=[{"role": "user", "content": "Hello!"}],
)

print(completion.choices[0].message.content)

import OpenAI from 'openai'

const client = new OpenAI({ baseURL: 'http://localhost:8080/v1' })

const completion = await client.chat.completions.create({
  model: 'openai/gpt-5-nano',
  messages: [{ role: 'user', content: 'Hello!' }],
})

console.log(completion.choices[0].message.content)

curl http://localhost:8080/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "openai/gpt-5-nano",
    "messages": [{"role": "user", "content": "Hello!"}]
  }'

The gateway ignores the caller’s Authorization header entirely — it is never forwarded, and the gateway authenticates upstream with its own provider keys. Whatever key your SDK sends is simply unused. It also performs no caller authentication of its own — see Gateway overview.

Streaming

Server-sent events, terminated by data: [DONE] — the shape an OpenAI SDK already expects.

stream = client.chat.completions.create(
    model="openai/gpt-5-nano",
    messages=[{"role": "user", "content": "Count to five."}],
    stream=True,
)

for chunk in stream:
    print(chunk.choices[0].delta.content or "", end="", flush=True)

const stream = await client.chat.completions.create({
  model: 'openai/gpt-5-nano',
  messages: [{ role: 'user', content: 'Count to five.' }],
  stream: true,
})

for await (const chunk of stream) {
  process.stdout.write(chunk.choices[0]?.delta?.content ?? '')
}

curl -N http://localhost:8080/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "openai/gpt-5-nano",
    "messages": [{"role": "user", "content": "Count to five."}],
    "stream": true
  }'

How a streamed failure arrives

The status is decided once, before a byte of the body exists.

When it fails	What you get
Before the response commits	An ordinary error response — real status, `Retry-After`, the provider’s message
After the 200 is committed	The error envelope as the last event, and no `[DONE]`

The missing sentinel is the point: [DONE] means the model finished, and sending one after a failure would tell a client the answer is complete when it is truncated. OpenAI does not send one either.

Other OpenAI-compatible tools

Anything that accepts an OpenAI base URL should work — LangChain, LlamaIndex, Continue, and similar. Two rules:

Base URL includes /v1

http://localhost:8080/v1. Tools that append /v1 themselves take http://localhost:8080.

Model strings keep the provider prefix

openai/gpt-5-nano, not gpt-5-nano. A tool that hardcodes a bare model name will get a 400.

There is no /v1/models endpoint, so tools that discover models by listing them will find nothing. Configure the model string by hand; the roster is the list.

Errors

The response body is the standard OpenAI envelope, so your SDK raises its own typed exception. warpllm adds fields alongside it that an HTTP caller can read and an SDK will ignore:

{
  "error": {
    "message": "deepseek: insufficient balance",
    "type": "insufficient_quota",
    "code": "insufficient_quota",
    "origin": "provider",
    "warpllm_code": "quota_exceeded",
    "provider": "deepseek",
    "provider_status": 402,
    "provider_code": "insufficient_balance"
  }
}

An upstream Retry-After is re-emitted as a real HTTP header, not only a body field — standard clients and proxies back off on the header and never read the JSON. See Errors for the full field list.

HTTP API reference

Schemas and an interactive playground.

Self-hosted gateway

HTTP API

Connecting a client

Official OpenAI SDKs

Streaming

How a streamed failure arrives

Other OpenAI-compatible tools

Errors

Next

HTTP API reference

​Official OpenAI SDKs

​Streaming

​How a streamed failure arrives

​Other OpenAI-compatible tools

​Errors

​Next

HTTP API reference

Official OpenAI SDKs

Streaming

How a streamed failure arrives

Other OpenAI-compatible tools

Errors

Next