> This page is also available as Markdown: append .md to its URL. The full index of Sikaru docs is at https://docs.sikaru.ai/llms.txt. Code examples use only operations from the public API reference at https://docs.sikaru.ai/api-reference.

# Evaluate traces and review results

Operations on this page (Python SDK `sikaru_api`):

1. Create a job: `client.evaluation_jobs.create_job(project, request_id=..., evaluator=..., revision=..., rubric=..., environment=..., targets=[TargetInput(trace_id=..., target=JudgmentTarget(kind="span", account_id=..., span_id=...))])["job"]`. 1–100 targets. Message targets use `kind="message"` with conversation and message identifiers.
2. Requires `imports:write` and managed evaluation execution enabled. Unavailable: 503. Reused `request_id` with changed inputs: 409.
3. Read a job: `client.evaluation_jobs.get_job(project, job_id)["job"]`. Poll with a bounded deadline until terminal status.
4. List results: `client.evaluation_results.list_results(project, evaluator=..., environment=..., limit=...)`. Job and result listings use cursors; pass the returned continuation cursor and keep filters.
5. Cancel: `cancel_job(project, job_id)`. Cancelling an HTTP wait does not cancel the evaluation.
6. Criteria: `evaluation_criteria.list_criteria`. Record your own judgment: `evaluation_results.record_result` (does not launch a managed evaluator).
7. CLI groups: `evaluation-jobs`, `evaluation-results`, `evaluation-criteria`.

An **evaluation job** applies a rubric to retained trace targets. Trace ingestion and evaluation are separate operations: an ingest receipt does not mean a judgment exists.

```mermaid
flowchart LR
    Trace["Retained trace"] --> Job["Evaluation job"]
    Job --> Results["Result records"]
    Results --> Review["Release review"]
    Review -->|"explicit decision"| Release["Release"]
```

The flow: pick a retained trace, submit a job against its spans or messages, read the result records, and review them before you make an explicit release decision.

## Submit a job

Start with a trace already available in your project. Select an actual span or message from that trace, including its account identity. The evaluator and revision identify the evaluation configuration you intend to use.

```python
from sikaru_api import JudgmentTarget, TargetInput

job = client.evaluation_jobs.create_job(
    project,
    request_id="case-42-quality-v1",
    evaluator="case-quality",
    revision="v1",
    rubric="Pass only if the response answers the question and cites the provided evidence.",
    environment="staging",
    targets=[TargetInput(
        trace_id=trace_id,
        target=JudgmentTarget(
            kind="span", account_id="customer-42", span_id=span_id,
        ),
    )],
)["job"]
```

* Replace `trace_id` and `span_id` with retained identifiers.
* A job accepts 1–100 targets.
* For a message target, use `kind="message"` and the matching conversation and message identifiers.
* The service freezes the target context when it admits the job, so the evaluation uses that submitted boundary.

| Requirement or response              | Meaning                                                                                                  |
| ------------------------------------ | -------------------------------------------------------------------------------------------------------- |
| `imports:write`                      | Permission required to create a job.                                                                     |
| Managed evaluation execution enabled | Required to create a job.                                                                                |
| 503                                  | Evaluation is unavailable.                                                                               |
| 409                                  | You reused a `request_id` with changed inputs. Keep a `request_id` only for the same logical submission. |

## Observe or cancel

```python
current = client.evaluation_jobs.get_job(project, job["id"])["job"]
results = client.evaluation_results.list_results(
    project, evaluator="case-quality", environment="staging", limit=50,
)
```

Poll with a bounded application deadline until the job reaches its terminal status. Inspect the job and result records; do not treat acceptance as success.

> **Warning**
>
> Use `cancel_job(project, job_id)` if the work should stop. Cancelling an HTTP wait alone does not cancel the evaluation.

Result and job listings use cursors. Keep the endpoint's returned continuation cursor and filters when you fetch more pages. `evaluation_criteria.list_criteria` lists available criteria. Result records keep evaluator, revision, verdict, and public reason.

## Use judgments in release review

Review failures and the supporting evidence before you approve a release. Keep evaluator revisions and targets consistent when you compare changes.

Applications that already produce their own judgments can submit them with `evaluation_results.record_result`. This records supplied evidence; it does not launch a managed evaluator.

> **Note**
>
> A passing result does not automatically promote an agent. Use [Create agents and manage releases](/build/agents-and-releases) for explicit release decisions.

CLI operations are available under `evaluation-jobs`, `evaluation-results`, and `evaluation-criteria`.

## Next steps

#### [Agents and releases](/build/agents-and-releases)

Make explicit release decisions.

#### [Trace ingestion](/improve/trace-ingestion)

Get traces into your project to evaluate.

#### [CLI](/sdks-and-cli/cli)

Run evaluation operations from the terminal.