API reference

One OpenAI-compatible API in front of 20 LLM providers. Cailos routes each request to the endpoint that fits it, falls back automatically when one fails, and records what happened.

Authentication

Every request requires an API key in the Authorization header:

Authorization: Bearer cai_your_key_here

Create keys in the dashboard (per-team). Keys are Argon2-hashed at rest; the full key is shown only once, when it's created.

Teams can also add their own provider keys in the dashboard and choose whether they're used first, as a fallback, or exclusively. cailos.key_source on each response says which key served it.

Base URL

Set this as base_url in any OpenAI SDK. All paths below are relative to this.

Quickstart

Pythonfrom openai import OpenAI

client = OpenAI(
    base_url=,
    api_key="cai_your_key_here",
)

response = client.chat.completions.create(
    model="auto",
    messages=[{"role": "user", "content": "Hello"}],
)
print(response.choices[0].message.content)

cURLcurl /chat/completions \
  -H "Authorization: Bearer cai_your_key_here" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "auto",
    "messages": [{"role": "user", "content": "Hello"}]
  }'

List models

Returns the model codenames your team can use (after its trust level and excluded providers and models), plus your team's custom models. Each entry carries a capabilities object describing what the model can serve for your team. Models whose endpoints are all temporarily out of rotation are left out.

Response{
  "object": "list",
  "data": [
    { "id": "auto", "object": "model", "capabilities": { "vision": true } },
    { "id": "claude-sonnet-5-5", "object": "model", "capabilities": { "vision": true } },
    { "id": "deepseek-4-1-flash", "object": "model", "capabilities": { "vision": false } }
  ]
}

Retrieve a model

Returns one model and its capabilities. Accepts any model string a chat request would, including strategy and mode suffixes (auto:speed, gpt-5-6-luna:cost) and custom model names, so you can check exactly what you're about to send. A retired model's codename returns the capabilities of the model now serving it. Returns 404 if the model is unknown to your team.

Response{
  "id": "auto:speed",
  "object": "model",
  "created": 0,
  "owned_by": "cailos",
  "capabilities": { "vision": true }
}

Chat completions

Send a conversation, get a completion. Requests and responses use the OpenAI Chat Completions format. Parameters not listed below (for example reasoning_effort, stream_options, parallel_tool_calls) are ignored.

Request body

Response format

Standard OpenAI chat completion format, plus a cailos metadata object:

Response{
  "id": "550e8400-e29b-41d4-a716-446655440000",
  "object": "chat.completion",
  "model": "gpt-5.4-mini-2026-03-17",
  "choices": [{
    "message": {
      "role": "assistant",
      "content": "Hello! How can I help you?",
      "reasoning_content": null,     // present only for reasoning models
      "tool_calls": null              // present only when tools are invoked
    },
    "finish_reason": "stop"
  }],
  "usage": {
    "prompt_tokens": 9,
    "completion_tokens": 12,
    "total_tokens": 21,
    "prompt_tokens_details": { "cached_tokens": 0 }
  },
  "cailos": {
    "optimise": "quality",
    "provider": "openai",
    "endpoint": "gpt-5.4-mini-2026-03-17",
    "trust_level": 2,
    "detected_languages": ["en"],
    "tier": "paid",
    "key_source": "cailos",
    "redirected_to": null,
    "compression": { "original_input_tokens": 11, "input_tokens_post_substitute": 9, "total_messages": 1, "compressed_messages": 1 }
  }
}

Examples

Streaming

Set stream: true to receive Server-Sent Events. Each event is a ChatCompletionChunk with incremental deltas, ending with data: [DONE]. The first chunk carries the cailos object, and the last chunk before [DONE] has an empty choices array and the usage totals, so check choices before indexing it.

Pythonstream = client.chat.completions.create(
    model="auto:speed",
    messages=[{"role": "user", "content": "Explain quantum computing"}],
    stream=True,
)

for chunk in stream:
    if not chunk.choices:   # the final chunk carries usage only
        continue
    delta = chunk.choices[0].delta.content
    if delta:
        print(delta, end="")

Responsedata: {"choices": [{"delta": {"content": "Hello"}}]}

data: {"choices": [{"delta": {"content": "!"}}]}

data: {"choices": [{"delta": {}, "finish_reason": "stop"}]}

data: {"choices": [], "usage": {"prompt_tokens": 9, "completion_tokens": 2, "total_tokens": 11}}

data: [DONE]

Tool calls

Define tools in the request. When the model invokes one, finish_reason is "tool_calls" and content is null:

Pythonresponse = client.chat.completions.create(
    model="auto:quality",
    messages=[{"role": "user", "content": "What's the weather in Amsterdam?"}],
    tools=[{
        "type": "function",
        "function": {
            "name": "get_weather",
            "parameters": {
                "type": "object",
                "properties": {"city": {"type": "string"}},
                "required": ["city"]
            }
        }
    }],
)

Response{
  "choices": [{
    "message": {
      "role": "assistant",
      "content": null,
      "tool_calls": [{
        "id": "call_abc123",
        "type": "function",
        "function": {
          "name": "get_weather",
          "arguments": "{\"city\": \"Amsterdam\"}"
        }
      }]
    },
    "finish_reason": "tool_calls"
  }]
}

To continue, append the assistant message (with tool_calls) and a tool message (with the result) back into messages.

Structured output

Force JSON with response_format: {"type": "json_object"}, or {"type": "json_schema", "json_schema": {...}} to follow a schema. Only endpoints that support structured output are considered.

Pythonresponse = client.chat.completions.create(
    model="auto:quality",
    messages=[{"role": "user", "content": "List 3 planets as JSON"}],
    response_format={"type": "json_object"},
)

data = json.loads(response.choices[0].message.content)
# {"planets": ["Mercury", "Venus", "Earth"]}

Vision

Send images as multipart content in the messages array:

Pythonresponse = client.chat.completions.create(
    model="auto:quality",
    messages=[{
        "role": "user",
        "content": [
            {"type": "text", "text": "What's in this image?"},
            {"type": "image_url", "image_url": {
                "url": "https://example.com/photo.jpg"
            }}
        ]
    }],
)

Both URLs and base64 data URIs (data:image/png;base64,...) are supported, and a message can carry several image_url parts. Cailos auto-detects vision requirements and routes to a capable model.

Reasoning

When a reasoning model answers (DeepSeek R1, Qwen3, Claude with extended thinking, Gemini thinking models, OpenAI GPT-5 and o-series), its reasoning is returned in reasoning_content, never as raw tags in content. For OpenAI models this is OpenAI's reasoning summary.

Response{
  "choices": [{
    "message": {
      "role": "assistant",
      "content": "The answer is 42.",
      "reasoning_content": "Let me think through this step by step..."
    }
  }],
  "usage": {
    "prompt_tokens": 15,
    "completion_tokens": 120,
    "reasoning_tokens": 98
  }
}

Pythonmsg = response.choices[0].message
print(msg.content)             # The actual answer

if hasattr(msg, "reasoning_content") and msg.reasoning_content:
    print(msg.reasoning_content)   # Chain-of-thought

• reasoning_content is absent for non-reasoning models.
• reasoning_tokens, when the provider reports them, are included in completion_tokens.
• Thinking tags (<think>, <thinking> and similar) are extracted automatically, so they never appear in content.

Routing

The model field takes a codename (gpt-5-6-luna, claude-sonnet-5-5). The same codename can be served by several providers, and Cailos ranks their endpoints by the strategy. When model is "auto", every model is a candidate.

Append a strategy suffix to any model value with : to control how providers are ranked:

Where model is auto, a codename or a custom model. Examples: auto:speed, gpt-5-6-luna:cost, claude-sonnet-5-5:quality.

Candidates are compared with each other, not against a fixed scale, and by how far apart they are: in a pool of a $3 and a $6 model, the $3 one is the cheap end. Differences too small to matter count as near-ties, so a model 3% dearer but much more capable can win a :cost request.

Choosing a strategy

On the free plan every request is routed for cost, whatever the suffix says. Strategy suffixes and the optimise hint take effect on paid plans.

Fallback

Each request gets a short ranked list of endpoints, usually three and from at least two providers when possible. If one fails or times out, the next is tried. If a stream fails after it started answering, the next endpoint able to continue a partial answer (Cailos tests each one) carries on from the text you already have, so the response stays one answer. Each recent failure, rate limits included, moves an endpoint down the ranking; the effect fades within the hour. After three failures in a row, or when half of its calls in the last two minutes failed, an endpoint is taken out of rotation until probes show it has recovered.

When a model is retired, its codename keeps working: requests go to the closest replacement and cailos.redirected_to names it. Switch to that codename to stop being redirected.

Execution modes

Routing sends each request to one endpoint. Add an execution mode to the model string to put more than one model on the question. A mode combines with a strategy, in any order (auto:cost:race, auto:race:cost), and a request can name one mode.

Pythonresponse = client.chat.completions.create(
    model="auto:quality:judge",
    messages=[{"role": "user", "content": "Review this contract clause for indemnity risk"}],
)

print(response.cailos.get("winner"))   # the endpoint whose answer you received; absent if the mode couldn't run

The models taking part come from the same ranked list a plain request would use, so the strategy and your trust level and exclusions still apply. The cailos object says what happened: mode names the mode and contenders lists the endpoints that took part. For race and judge, winner is the endpoint whose answer you received; for wizard, the model that wrote the reply is wizard.endpoint. When there aren't enough models for the mode to run, winner and contenders are absent and the mode's own object carries fallback. If the judge or writer can't finish, the top-ranked candidate's reply is served and the mode's object names the reason in fallback. A race with only one eligible endpoint runs as a normal request.

Custom models

Teams can create custom virtual models in their dashboard that operate in one of two routing modes:

• Dynamic pool: Defines an allowed candidate subset of models (for example gpt-5-6-luna, claude-sonnet-5-5, gemini-3-8-flash). For each request, Cailos ranks every endpoint of those models and routes to the best one for the active strategy: fastest for speed, cheapest for cost, most capable for quality.
• Routing instructions (dynamic pools): Describe in your own words which models should take which requests, for example “code to Claude, everything else to GPT; the harder the question, the more quality matters”. Cailos translates this into a routing plan. For each request, Jev, TypeSafe's classification model, reads the end of the latest user message (its last 800 characters, with PII cloaked; earlier turns are not sent) and the plan either narrows the pool to the right models or slides its ranking between two strategies (for example from cost towards quality). A strategy suffix or optimise hint, or the free plan's cost routing, overrides the slide; narrowing still applies. If Jev can't answer, the pool routes as usual.
• Fallback chain: Evaluates models in a strict sequential precedence tier (for example gpt-5-6-luna → claude-sonnet-5-5 → auto:speed). Cailos tries the first model's endpoints first and moves down the chain when they fail (errors, timeouts, rate limits, invalid JSON). Models that can't serve the request, for example without vision, are skipped.

Pythonresponse = client.chat.completions.create(
    model="my-custom-model",
    messages=[{"role": "user", "content": "Summarize this document"}],
)

# You can also override the optimisation strategy dynamically:
fast_response = client.chat.completions.create(
    model="my-custom-model:speed",
    messages=[{"role": "user", "content": "Ping"}],
)

Custom models work with the rest of Cailos:

• Strategy overrides: Configure a default strategy on the custom model (e.g. quality), or override per request via suffix (e.g. my-custom-model:cost).
• Model discovery: Active custom models for your team appear in GET /v1/models when authenticated with your team API key.
• Playground dry-run: Custom models in both dynamic pool and fallback modes get a real-time routing preview in the Playground, showing which underlying model and provider would be picked. The preview leaves routing instructions out; the custom model editor's “Try it” shows what they decide.

Routing hints

Fine-tune routing with the cailos object. Most fields are detected from the request, so set them only to override.

JSON{
  "model": "auto",
  "messages": [...],
  "cailos": {
    "trust_level": 2,
    "language": "en"
  }
}

Privacy and compression

PII protection

Before a request leaves Cailos, email addresses, phone numbers, credit card numbers and IP addresses in messages are replaced with placeholders by llmshield. The placeholders are swapped back in the reply, so you see the original values and the provider never does. Names, places and organisations are left as they are, so code and documentation aren't mangled.

What the provider seesRefund card <CREDIT_CARD_2>, then email <EMAIL_0> or call <PHONE_1>.

Cailos doesn't store the text of your prompts. It keeps the routing decision, latency, token counts and cost for each request.

Prompt compression

Cailos shortens prompts before sending them, with fixed word-level substitutions: filler phrases are removed and wordy ones shortened (“in order to” becomes “to”; articles such as “the” are dropped). Code blocks, inline code, XML and HTML elements and tool messages are left untouched. The response's cailos.compression says how much changed.

Compression is on unless your team turns it off in the dashboard. Set cailos.compression to false or true to override it for one request.

Prompt caching

Multi-turn conversations tend to stay on the endpoint that served the previous turn when its provider caches prompts, which makes later turns cheaper and faster. Anthropic prompt caching is turned on automatically. Cached input shows in usage.prompt_tokens_details.cached_tokens.

Feedback

Submit human feedback on any routed request. Satisfaction ratings per endpoint feed into model selection over time.

The {id} in the URL is the id field returned in every chat completion response; pass it through unchanged. No authentication is needed: the id is unguessable, and feedback is accepted for 1 week after the request. After that the endpoint returns 410 Gone. An unknown id returns 404.

Request body

cURLcurl -X POST /accounts/requests/550e8400-e29b-41d4-a716-446655440000/feedback/ \
  -H "Content-Type: application/x-www-form-urlencoded" \
  -d 'positive=0&comment=wrong language in response'

The endpoint returns an HTML partial (designed for htmx). The feedback is also visible on the request detail page in the dashboard, and aggregated satisfaction ratings appear per endpoint on the model browse page.

Errors

All errors follow the OpenAI error envelope. For validation errors, param names the field.

{
  "error": {
    "message": "Model 'nonexistent' not found or not active.",
    "type": "not_found_error",
    "param": null,
    "code": null
  }
}

Once a stream has started, errors arrive as an event instead of a status code: data: {"error": {"message": "...", "type": "provider_error"}} followed by data: [DONE]. Types are provider_error, provider_timeout, model_unavailable and server_error. A stream cut off mid-answer that no other endpoint could continue ends with finish_reason: "error".

Rate limits

Limits are per team, across all its keys, and set by plan. Each returns 429 when reached.

The per-minute window starts with the first request and lasts 60 seconds. Monthly budgets set on a team, or on its owner across all their teams, also return 429 once spent, and reset on the 1st.

Providers

You never talk to providers directly; send a codename and Cailos does the rest. For transparency, this is how each one is called: