Skip to navigation

Inference

Call Sikaru catalog models with OpenAI-compatible chat completions, completions and Responses requests, look up what each model supports, and handle field-level errors.

View as Markdown

Sikaru serves its model catalog through OpenAI-compatible endpoints. Existing OpenAI SDK code works by changing the base URL, API key and model. Every request either behaves as the OpenAI contract describes or fails with an error that names the field. Sikaru never silently ignores a request field.

EndpointRequest shapeStreaming
POST /v1/chat/completionsOpenAI chat completionsChunks, ending with data: [DONE]
POST /v1/completionsOpenAI completions with raw text promptsCompletion chunks, ending with data: [DONE]
POST /v1/responsesOpenAI Responses, statelessTyped events, ending with response.completed, response.incomplete or response.failed
GET /v1/modelsList modelsNo
GET /v1/models/{model}One model with its pricing and specsNo

Authenticate with a Sikaru API key that has the inference:invoke scope, sent as Authorization: Bearer <key>. The CLI can instead use your sikaru login sign-in: it sends the sign-in with the saved project in the Sikaru-Project header, and usage is billed to that project.

Call a model

Sikaru is the generated client with OpenAI-shaped create(stream=...) on its inference resources. Construct it exactly like SikaruApi; every other resource is unchanged.

import os
from sikaru_sdk import Sikaru
client = Sikaru(api_key=os.environ["SIKARU_API_KEY"])
model = client.models.list().data[0].id
completion = client.chat.completions.create(
model=model,
messages=[{"role": "user", "content": "Summarize the release notes."}],
top_k=40,
)
print(completion.choices[0].message.content)
for chunk in client.chat.completions.create(model=model, messages=[{"role": "user", "content": "Hi"}], stream=True):
if chunk.choices:
print(chunk.choices[0].delta.content or "", end="")
if chunk.usage:
print("\ntokens:", chunk.usage.total_tokens)

AsyncSikaru mirrors it: await client.chat.completions.create(..., stream=True) returns an async iterator.

Streamed chat and completions requests always end with a usage chunk, so token counts are available without setting stream_options.

How each request field is handled

Every documented OpenAI request parameter on the three endpoints is accepted and falls into exactly one class:

ClassBehaviorExamples
ForwardedSent to the model as suppliedmessages, input, prompt, temperature, top_p, stop, seed, penalties, output limits, function tools, tool_choice, parallel_tool_calls, response_format, text. A model whose supported_parameters omits a penalty accepts it only at 0 and otherwise returns 400 naming the field.
Handled by SikaruAccepted and never sent to the model as supplieduser, metadata and safety_identifier are never forwarded. prompt_cache_key drives cache isolation within your organization and project. stream_options is managed by Sikaru.
Declared per modelForwarded when the model declares it; otherwise 400 naming the fieldlogprobs, top_logprobs, logit_bias, prediction, reasoning controls, image and file inputs, and the extensions below
RejectedAlways 400 naming the fieldn above 1, built-in tools such as web_search_options, stored prompts and conversations, background: true, references to stored files or items, and service_tier

A field outside the OpenAI surface and the documented extensions returns 400 with code unknown_parameter. Sikaru chooses where each request is served, so the routing fields provider, transforms, models and route, and usage: {"include": true}, are accepted for client compatibility and have no effect.

Requests are stateless for API keys: send history inline. store: true, and previous_response_id on Responses, return 400 with code unsupported_continuation.

Extensions

These open-weight controls are first-class top-level fields. Each is forwarded on models that declare it and rejected with a named 400 otherwise.

FieldEndpointsMeaning
top_kchat, completionsSample from the k most likely tokens
min_pchat, completionsDrop tokens below this fraction of the top token’s probability
typical_pchat, completionsLocally typical sampling
repetition_penaltychat, completionsPenalize repeated tokens; values above 1 discourage repetition
reasoning_effortchatA standard effort level, a boolean, or an integer token budget on models that accept budgets
thinkingchat{"type": "enabled", "budget_tokens": N} sets an explicit reasoning budget
reasoningchat{"effort", "max_tokens", "exclude"}; exclude: true keeps reasoning out of the response
include_reasoningchatfalse is the same as reasoning: {"exclude": true}
reasoning_historychatdisabled, interleaved or preserved: how prior reasoning_content in assistant messages is used
response_format: {"type": "grammar"}chat (text.format on Responses)Constrain output to a grammar

Send only one of reasoning_effort, thinking and reasoning per request.

Look up a model

Model IDs take the lab/model form, such as z-ai/glm-5.3-flash. GET /v1/models/{model}, and each entry of GET /v1/models, describes one model:

{
"id": "z-ai/glm-5.3-flash",
"object": "model",
"created": 0,
"owned_by": "sikaru",
"display_name": "GLM 5.3 Flash",
"pricing": {"input_usd_per_mtok": 0.1875, "output_usd_per_mtok": 0.625, "cache_read_usd_per_mtok": 0.0375},
"specs": {
"context_window": 1000000,
"max_output_tokens": 65536,
"modalities": {"input": ["text", "image"], "output": ["text"]},
"endpoints": ["chat"],
"supports_reasoning": true,
"reasoning": {
"efforts": ["low", "high", "max"],
"default_effort": "max",
"budget": false,
"history": ["interleaved"],
"default_history": "interleaved",
"summary": false,
"normalization": {"none": "low", "minimal": "low", "low": "low", "medium": "low",
"high": "high", "xhigh": "high", "max": "max", "true": "max", "false": null}
},
"supported_parameters": ["frequency_penalty", "include_reasoning", "max_tokens", "presence_penalty",
"reasoning", "reasoning_effort", "response_format", "seed", "stop",
"structured_outputs", "temperature", "tool_choice", "tools", "top_p"],
"sampling_defaults": {"temperature": 1.0, "top_p": 0.95, "top_k": null, "min_p": null}
}
}
  • pricing: US dollars per million input, output and cached input tokens.
  • specs.modalities: what the model reads and writes.
  • specs.endpoints: the request endpoints the model serves.
  • specs.reasoning: the reasoning controls and what each standard effort runs at, or null when the model has none.
  • specs.supported_parameters: every request parameter the model accepts. structured_outputs means JSON-schema response_format.
  • specs.sampling_defaults: the model lab’s recommended sampling. A request that omits temperature, top_p, top_k or min_p runs at the value shown here; any value you send wins. null means the serving default applies.

Reasoning effort normalization

Effort written in the standard vocabulary (none, minimal, low, medium, high, xhigh, max) runs at the nearest level the model declares. Ties resolve to the lower level. true selects the model’s default effort, and false turns reasoning off where the model declares none. The specs.reasoning.normalization table shows exactly what each value runs at; a null entry means that value is rejected for the model. Only values outside the vocabulary are always rejected.

Reasoning continuity

On chat, models that declare reasoning history accept prior reasoning_content in assistant messages, with reasoning_history choosing how it is used.

Errors

Errors use the OpenAI envelope:

{"error": {"message": "This model does not support top_k", "type": "invalid_request_error", "param": "top_k", "code": "unsupported_parameter"}}
StatusCodeMeaning
400unknown_parameterThe field is not part of the inference surface
400unsupported_parameterThe model does not declare this field, input type or value
400unsupported_endpointThe model does not serve this endpoint
400unsupported_continuationAPI keys are stateless: store and previous_response_id are not available; send history inline
400invalid_requestA value is invalid; param names it
400model_retiredThe model passed its deprecation date
401invalid_api_keyMissing or invalid key
402credits_exhausted, spend_limit_reachedOut of credits, or the organization’s monthly spend limit or the agent’s spending cap was reached; see Billing
403insufficient_scopeThe key lacks the inference:invoke scope
404model_not_foundUnknown or unavailable model; param is model
413request_too_large, invalid_requestThe request is too large for the model or the API
429rate_limit_exceededCapacity is full; retry after Retry-After
503, 504billing_unavailable, upstream_timeoutTemporarily unavailable; retry after Retry-After
502, 503upstream_unavailable, service_unavailableServing is unavailable; retry later with backoff

A request rejected by Sikaru’s validation never reaches a model and is not charged.