Evals

Evals API

Manage evaluation definitions, trigger runs, and query results.

POST
/api/evals

Create a new eval definition.

jsonRequest Body
{
  "name": "json-check",
  "type": "static",
  "evaluator": "is_json",
  "config": { "field": "completion" },
  "enabled": true
}
GET
/api/evals

List definitions. Filter with ?type= and ?enabled=.

GET
/api/evals/:id

Get a specific eval definition.

PUT
/api/evals/:id

Update an eval definition.

DELETE
/api/evals/:id

Delete an eval definition.

POST
/api/evals/:id/run

Trigger a manual evaluation run against recent traces.

GET
/api/evals/:id/results

List evaluation results for a definition.

jsonResponse
[
  {
    "id": 1,
    "eval_id": 5,
    "log_id": 142,
    "score": 1.0,
    "passed": true,
    "details": "Field 'completion' is valid JSON",
    "created_at": "2025-03-15T10:30:00Z"
  }
]
GET
/api/evals/runs

List all evaluation runs.

GET
/api/evals/runs/:runId

Get details of a specific run.