Skip to navigation

Test Agent Grader

View as Markdown

Score an unsaved file with its own judge. Poll a pending test; no Grader or verdict is saved.

Authentication

AuthorizationBearer

Bearer authentication of the form Bearer <token>, where token is your auth token.

Path parameters

project_idstringRequired
agent_idstringRequired

Headers

Idempotency-Keystring or nullOptional

Request

This endpoint expects an object.
filestringRequired<=200000 characters

The Grader file’s text, as a push would read it. It is tested by its chosen judge without saving a Grader or verdict; its ceiling applies while its release check is on.

namestringRequiredformat: "^[a-z0-9]+(?:-[a-z0-9]+)*$"<=64 characters

The Grader file’s name: graders/<name>.toml in the agent folder.

limitintegerOptional1-50Defaults to 20
Recent conversations to score.

Response

Successful Response
ceilingdouble or null
conclusiveinteger
conversationsinteger
flagRatedouble or null
flaggedinteger
resultslist of objects
Flagged first.
statusenum

A pending test is still scoring; poll runId. A failed test is final: test again to start a new run.

Allowed values:
wouldBlockboolean
The flag rate is above the ceiling.
errorstring or nullOptional
Why a failed test could not finish.
runIdstring or nullOptional

Errors

401
Unauthorized Error
402
Payment Required Error
403
Forbidden Error
404
Not Found Error
409
Conflict Error
422
Unprocessable Entity Error
503
Service Unavailable Error