OpenAI-Compatible API Gateway: The One-Line base_url Migration

If your code already calls GPT models through the official openai SDK, plugging in an OpenAI-compatible gateway is, in theory, a one-line change: point baseURL (or base_url) at the gateway and swap in the sk-… key it issued you. In practice, three things trip people up — streaming responses, error codes, and which host actually takes the auth header. This guide walks through the real request shapes so none of them surprise you.

Why "OpenAI-compatible" means a one-line change

"OpenAI-compatible" means the gateway mirrors the OpenAI wire protocol exactly: the /chat/completions path, the model/messages/stream fields in the request body, the choices/usage shape in the response, and the {"error": {"message", "type", "code"}} shape on failure. Anything that already "speaks OpenAI" — the official SDK, LangChain, LlamaIndex, or a bare fetch/requests call — doesn't need new call logic. It needs two config values swapped:

• Base URL: from https://api.openai.com/v1 to the gateway's address
• API key: from your OpenAI key to the key the gateway issued

One distinction matters here: the "gateway" is the data plane — the layer that actually processes /chat/completions requests. That's a different host from the control-plane site where you sign up, manage keys, and view billing. The control-plane site does not proxy /v1 traffic; every request should go straight to the gateway host.

Three steps: create a key, swap the base URL, call a model

1. Sign up and create an API key. Once you're logged in, go to the API Keys page in the console and create one. The full key is shown exactly once, at creation time — save it immediately, since it can't be retrieved in plaintext afterward. New accounts get trial credit automatically on signup, no card required, so you can run the full flow before deciding anything.
2. Point the base URL at the gateway. The JavaScript SDK uses baseURL; the Python SDK uses base_url; a bare HTTP call just appends every path to the gateway address.
3. Call any model. Swap the model field for a real id from the catalog. Which models are available and what their ids look like is whatever the model catalog currently shows — don't hardcode ids from memory.

A bare HTTP call looks like this (swap {GATEWAY_BASE_URL} for your actual deployment; the example below uses the placeholder domain the docs use):

curl https://api.apiko.example/v1/chat/completions \
  -H "Authorization: Bearer $APIKO_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "your-model-id",
    "messages": [{"role": "user", "content": "Hello"}]
  }'

JavaScript, using the official openai package — the only change is baseURL:

import OpenAI from 'openai';

const client = new OpenAI({
  apiKey: process.env.APIKO_API_KEY,
  baseURL: 'https://api.apiko.example/v1',
});

const completion = await client.chat.completions.create({
  model: 'your-model-id',
  messages: [{ role: 'user', content: 'Hello' }],
});

Python is the same idea — pass base_url when you instantiate the client:

from openai import OpenAI

client = OpenAI(
    api_key="APIKO_API_KEY",
    base_url="https://api.apiko.example/v1",
)

completion = client.chat.completions.create(
    model="your-model-id",
    messages=[{"role": "user", "content": "Hello"}],
)

Notice what didn't change across all three: the business logic — building messages, reading choices[0].message — is identical. That's what "OpenAI-compatible" actually buys you: you don't rewrite integration code per provider, and you don't maintain a different SDK dependency for each one.

Streaming: stream_options.include_usage is the part people miss

Set stream to true in the request body and the response becomes server-sent events: each line starts with data: and carries a JSON chunk, ending with data: [DONE]. That part matches the standard OpenAI streaming protocol, and it's rarely where people get stuck. The real gap is getting real token counts out of a streaming call.

A plain streaming response doesn't include a usage field in the intermediate chunks — if your billing or logging logic depends on token counts, you'll find they're just missing in the streaming path. Fix it by adding this explicitly to the request body:

{
  "model": "your-model-id",
  "messages": [{ "role": "user", "content": "Hello" }],
  "stream": true,
  "stream_options": { "include_usage": true }
}

With that set, the final chunk of the stream carries a usage object with the real prompt/completion token counts for that request — the exact same numbers the gateway bills against. So whatever usage you read client-side matches what gets deducted from your balance. If you're tracking usage or rate-limiting by token count, parse that last chunk instead of estimating with a local tokenizer.

The curl version of a streaming request:

curl https://api.apiko.example/v1/chat/completions \
  -H "Authorization: Bearer $APIKO_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "your-model-id",
    "messages": [{"role": "user", "content": "Hello"}],
    "stream": true,
    "stream_options": {"include_usage": true}
  }'

Error codes: check this table before you go digging through code

Non-200 responses use the same error body shape as OpenAI, so you can usually diagnose by status code alone:

| Status | Common cause | What to do | |---|---|---| | 401 | Missing or invalid API key | Check the Authorization: Bearer header carries the key, and confirm it hasn't been revoked | | 402 / 403 | Insufficient balance (insufficient_user_quota) | Top up in the console's Billing page and retry | | 404 | Model or path not found (model_not_found) | Cross-check the model id against the model catalog, and confirm the key isn't scoped to a model allowlist that excludes it | | 429 | Rate limited | Retry with exponential backoff so you don't immediately re-trigger the limit | | 5xx | Upstream provider failure | The gateway retries across channels automatically; if it still fails, the request is not billed — the pre-hold is refunded |

That last row is worth unpacking. When a request goes out, the gateway takes an estimated pre-hold, then settles the difference once the upstream provider returns a real result — billed on actual token usage. If the upstream call ultimately fails (after the gateway has already retried alternate channels behind the scenes), the pre-hold is refunded in full — you don't pay for a request that never produced a result. None of this requires extra handling on your end; refunds happen automatically, and you can verify the exact charge and refund for any request in the console's usage log.

What's next

Migrating existing code amounts to changing baseURL/base_url, swapping in a new key, and adding stream_options.include_usage where you need real usage numbers. If you want to prove out the flow before committing, new accounts get trial credit automatically — no card required — so you can just sign up, grab a key, and run the code above. For current per-model rates, the pricing page reflects exactly what billing uses.