> This page is also available as Markdown: append .md to its URL. The full index of Sikaru docs is at https://docs.sikaru.ai/llms.txt. Code examples use only operations from the public API reference at https://docs.sikaru.ai/api-reference.

# Inference

Operations on this page:

1. `POST /v1/chat/completions`, `POST /v1/completions` and `POST /v1/responses` accept the OpenAI request shape. `stream: true` returns server-sent events.
2. Model IDs take the `lab/model` form, such as `z-ai/glm-5.3-flash`. `GET /v1/models` lists available models; `GET /v1/models/{model}` returns one model with `display_name`, `pricing` (USD per million tokens) and `specs` (context window, max output, modalities, reasoning levels, supported parameters, recommended sampling defaults).
3. Python: `from sikaru_sdk import Sikaru`; `client.chat.completions.create(..., stream=True | False)`, `client.completions.create(...)`, `client.responses.create(...)`, `client.models.list()`, `client.models.retrieve(model)`.
4. TypeScript: `import { Sikaru } from "@aadi-labs/sikaru-sdk/inference"`; the same methods, with `stream: true` in the request object.
5. A field the model does not declare returns `400` with `error.param` naming the field. A field that is not part of the surface returns `400` with code `unknown_parameter`. An unknown model returns `404` with code `model_not_found` and `param: "model"`.
6. CLI: `sikaru inference models MODEL -o json` prints one model's card; `sikaru inference chat MODEL "prompt" -o json` prints the chat completion. The message can come from stdin; `--stream` streams the answer to stdout and reasoning to stderr.

Errors are `{"error": {"message", "type", "param", "code"}}`; branch on `code` and read `param`.

Sikaru serves its model catalog through OpenAI-compatible endpoints. Existing OpenAI SDK code works by changing the base URL, API key and model. Every request either behaves as the OpenAI contract describes or fails with an error that names the field. Sikaru never silently ignores a request field.

| Endpoint                    | Request shape                            | Streaming                                                                                  |
| --------------------------- | ---------------------------------------- | ------------------------------------------------------------------------------------------ |
| `POST /v1/chat/completions` | OpenAI chat completions                  | Chunks, ending with `data: [DONE]`                                                         |
| `POST /v1/completions`      | OpenAI completions with raw text prompts | Completion chunks, ending with `data: [DONE]`                                              |
| `POST /v1/responses`        | OpenAI Responses, stateless              | Typed events, ending with `response.completed`, `response.incomplete` or `response.failed` |
| `GET /v1/models`            | List models                              | No                                                                                         |
| `GET /v1/models/{model}`    | One model with its pricing and specs     | No                                                                                         |

Authenticate with a Sikaru API key that has the `inference:invoke` scope, sent as `Authorization: Bearer <key>`. The CLI can instead use your `sikaru login` sign-in: it sends the sign-in with the saved project in the `Sikaru-Project` header, and usage is billed to that project.

## Call a model

#### Python

`Sikaru` is the generated client with OpenAI-shaped `create(stream=...)` on its inference resources. Construct it exactly like `SikaruApi`; every other resource is unchanged.

```python
import os
from sikaru_sdk import Sikaru

client = Sikaru(api_key=os.environ["SIKARU_API_KEY"])
model = client.models.list().data[0].id

completion = client.chat.completions.create(
    model=model,
    messages=[{"role": "user", "content": "Summarize the release notes."}],
    top_k=40,
)
print(completion.choices[0].message.content)

for chunk in client.chat.completions.create(model=model, messages=[{"role": "user", "content": "Hi"}], stream=True):
    if chunk.choices:
        print(chunk.choices[0].delta.content or "", end="")
    if chunk.usage:
        print("\ntokens:", chunk.usage.total_tokens)
```

`AsyncSikaru` mirrors it: `await client.chat.completions.create(..., stream=True)` returns an async iterator.

#### TypeScript

```typescript
import { Sikaru } from "@aadi-labs/sikaru-sdk/inference";

const client = new Sikaru({ apiKey: process.env.SIKARU_API_KEY });
const { data } = await client.models.list();

const completion = await client.chat.completions.create({
  model: data[0].id,
  messages: [{ role: "user", content: "Summarize the release notes." }],
  top_k: 40,
});

const stream = await client.chat.completions.create({ model: data[0].id, messages: [{ role: "user", content: "Hi" }], stream: true });
for await (const chunk of stream) {
  process.stdout.write(chunk.choices?.[0]?.delta?.content ?? "");
}
```

#### OpenAI SDK

Point the official SDK at Sikaru. Sikaru extensions such as `top_k` go in `extra_body`.

```python
import os
from openai import OpenAI

client = OpenAI(base_url="https://api.sikaru.ai/v1", api_key=os.environ["SIKARU_API_KEY"])
completion = client.chat.completions.create(
    model="<model id>",
    messages=[{"role": "user", "content": "Hello"}],
    extra_body={"top_k": 40},
)
```

Streamed chat and completions requests always end with a usage chunk, so token counts are available without setting `stream_options`.

## How each request field is handled

Every documented OpenAI request parameter on the three endpoints is accepted and falls into exactly one class:

| Class              | Behavior                                                               | Examples                                                                                                                                                                                                                                                                                                      |
| ------------------ | ---------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Forwarded          | Sent to the model as supplied                                          | `messages`, `input`, `prompt`, `temperature`, `top_p`, `stop`, `seed`, penalties, output limits, function `tools`, `tool_choice`, `parallel_tool_calls`, `response_format`, `text`. A model whose `supported_parameters` omits a penalty accepts it only at `0` and otherwise returns `400` naming the field. |
| Handled by Sikaru  | Accepted and never sent to the model as supplied                       | `user`, `metadata` and `safety_identifier` are never forwarded. `prompt_cache_key` drives cache isolation within your organization and project. `stream_options` is managed by Sikaru.                                                                                                                        |
| Declared per model | Forwarded when the model declares it; otherwise `400` naming the field | `logprobs`, `top_logprobs`, `logit_bias`, `prediction`, reasoning controls, image and file inputs, and the extensions below                                                                                                                                                                                   |
| Rejected           | Always `400` naming the field                                          | `n` above 1, built-in tools such as `web_search_options`, stored prompts and conversations, `background: true`, references to stored files or items, and `service_tier`                                                                                                                                       |

A field outside the OpenAI surface and the documented extensions returns `400` with code `unknown_parameter`. Sikaru chooses where each request is served, so the routing fields `provider`, `transforms`, `models` and `route`, and `usage: {"include": true}`, are accepted for client compatibility and have no effect.

Requests are stateless for API keys: send history inline. `store: true`, and `previous_response_id` on Responses, return `400` with code `unsupported_continuation`.

## Extensions

These open-weight controls are first-class top-level fields. Each is forwarded on models that declare it and rejected with a named `400` otherwise.

| Field                                  | Endpoints                         | Meaning                                                                                               |
| -------------------------------------- | --------------------------------- | ----------------------------------------------------------------------------------------------------- |
| `top_k`                                | chat, completions                 | Sample from the `k` most likely tokens                                                                |
| `min_p`                                | chat, completions                 | Drop tokens below this fraction of the top token's probability                                        |
| `typical_p`                            | chat, completions                 | Locally typical sampling                                                                              |
| `repetition_penalty`                   | chat, completions                 | Penalize repeated tokens; values above 1 discourage repetition                                        |
| `reasoning_effort`                     | chat                              | A standard effort level, a boolean, or an integer token budget on models that accept budgets          |
| `thinking`                             | chat                              | `{"type": "enabled", "budget_tokens": N}` sets an explicit reasoning budget                           |
| `reasoning`                            | chat                              | `{"effort", "max_tokens", "exclude"}`; `exclude: true` keeps reasoning out of the response            |
| `include_reasoning`                    | chat                              | `false` is the same as `reasoning: {"exclude": true}`                                                 |
| `reasoning_history`                    | chat                              | `disabled`, `interleaved` or `preserved`: how prior `reasoning_content` in assistant messages is used |
| `response_format: {"type": "grammar"}` | chat (`text.format` on Responses) | Constrain output to a grammar                                                                         |

Send only one of `reasoning_effort`, `thinking` and `reasoning` per request.

## Look up a model

Model IDs take the `lab/model` form, such as `z-ai/glm-5.3-flash`. `GET /v1/models/{model}`, and each entry of `GET /v1/models`, describes one model:

```json
{
  "id": "z-ai/glm-5.3-flash",
  "object": "model",
  "created": 0,
  "owned_by": "sikaru",
  "display_name": "GLM 5.3 Flash",
  "pricing": {"input_usd_per_mtok": 0.1875, "output_usd_per_mtok": 0.625, "cache_read_usd_per_mtok": 0.0375},
  "specs": {
    "context_window": 1000000,
    "max_output_tokens": 65536,
    "modalities": {"input": ["text", "image"], "output": ["text"]},
    "endpoints": ["chat"],
    "supports_reasoning": true,
    "reasoning": {
      "efforts": ["low", "high", "max"],
      "default_effort": "max",
      "budget": false,
      "history": ["interleaved"],
      "default_history": "interleaved",
      "summary": false,
      "normalization": {"none": "low", "minimal": "low", "low": "low", "medium": "low",
                        "high": "high", "xhigh": "high", "max": "max", "true": "max", "false": null}
    },
    "supported_parameters": ["frequency_penalty", "include_reasoning", "max_tokens", "presence_penalty",
                             "reasoning", "reasoning_effort", "response_format", "seed", "stop",
                             "structured_outputs", "temperature", "tool_choice", "tools", "top_p"],
    "sampling_defaults": {"temperature": 1.0, "top_p": 0.95, "top_k": null, "min_p": null}
  }
}
```

* `pricing`: US dollars per million input, output and cached input tokens.
* `specs.modalities`: what the model reads and writes.
* `specs.endpoints`: the request endpoints the model serves.
* `specs.reasoning`: the reasoning controls and what each standard effort runs at, or `null` when the model has none.
* `specs.supported_parameters`: every request parameter the model accepts. `structured_outputs` means JSON-schema `response_format`.
* `specs.sampling_defaults`: the model lab's recommended sampling. A request that omits `temperature`, `top_p`, `top_k` or `min_p` runs at the value shown here; any value you send wins. `null` means the serving default applies.

## Reasoning effort normalization

Effort written in the standard vocabulary (`none`, `minimal`, `low`, `medium`, `high`, `xhigh`, `max`) runs at the nearest level the model declares. Ties resolve to the lower level. `true` selects the model's default effort, and `false` turns reasoning off where the model declares `none`. The `specs.reasoning.normalization` table shows exactly what each value runs at; a `null` entry means that value is rejected for the model. Only values outside the vocabulary are always rejected.

## Reasoning continuity

On chat, models that declare reasoning history accept prior `reasoning_content` in assistant messages, with `reasoning_history` choosing how it is used.

## Errors

Errors use the OpenAI envelope:

```json
{"error": {"message": "This model does not support top_k", "type": "invalid_request_error", "param": "top_k", "code": "unsupported_parameter"}}
```

| Status   | Code                                          | Meaning                                                                                                        |
| -------- | --------------------------------------------- | -------------------------------------------------------------------------------------------------------------- |
| 400      | `unknown_parameter`                           | The field is not part of the inference surface                                                                 |
| 400      | `unsupported_parameter`                       | The model does not declare this field, input type or value                                                     |
| 400      | `unsupported_endpoint`                        | The model does not serve this endpoint                                                                         |
| 400      | `unsupported_continuation`                    | API keys are stateless: `store` and `previous_response_id` are not available; send history inline              |
| 400      | `invalid_request`                             | A value is invalid; `param` names it                                                                           |
| 400      | `model_retired`                               | The model passed its deprecation date                                                                          |
| 401      | `invalid_api_key`                             | Missing or invalid key                                                                                         |
| 402      | `credits_exhausted`, `spend_limit_reached`    | Out of credits, or the organization's monthly spend limit or the agent's spending cap was reached; see Billing |
| 403      | `insufficient_scope`                          | The key lacks the `inference:invoke` scope                                                                     |
| 404      | `model_not_found`                             | Unknown or unavailable model; `param` is `model`                                                               |
| 413      | `request_too_large`, `invalid_request`        | The request is too large for the model or the API                                                              |
| 429      | `rate_limit_exceeded`                         | Capacity is full; retry after `Retry-After`                                                                    |
| 503, 504 | `billing_unavailable`, `upstream_timeout`     | Temporarily unavailable; retry after `Retry-After`                                                             |
| 502, 503 | `upstream_unavailable`, `service_unavailable` | Serving is unavailable; retry later with backoff                                                               |

A request rejected by Sikaru's validation never reaches a model and is not charged.