Pull eval scores
When continuous evals is on, a sample of an Automation’s runs is graded against its eval file. The Evals tab charts each criterion’s average score over time. The evals endpoints return the same numbers, so you can pull them into your own dashboards and alerts. The endpoints are read-only. They need an access token for a service account, and they answernot_found when the organization has traces or continuous evals turned off.
1. Find the Automation
Evals endpoints address an Automation by itsid. Automation names are not unique, so list Automations with a name filter and pick the one you mean:
2. List its criteria
Each criterion is one chart of the Evals tab. Pass the samerange the tab offers: 24h, 7d or 30d.
key stays the same when its text is edited. Use it to ask for that criterion’s scores.
3. Pull the scores
Ask forall criteria or a comma-separated list of keys:
timeis where the bucket starts. Buckets are UTC hours for24h, and UTC days for7dand30d.deployment_versionis n in the tab’s “Deployment #n”. While a new version rolls out, a bucket can have a row for each version.scoreis the average score from 0 to 10 of the runs graded in the bucket.samplesis how many graded runs that average covers.
Score at a point in time
Passat instead of range to get the one bucket that holds a moment:
- A date, such as
at=2026-11-18, returns that UTC day. - A date-time with a UTC offset, such as
at=2026-11-18T14:00:00Z, returns the UTC hour it falls in.
List the graded executions behind a chart
The table under a chart lists the executions graded on that criterion. Get the same rows, newest first, a page at a time:reasonis the judge’s overall reason for the score. It can quote the execution.checksholds the judge’s verdict on each check. Onlypassandfailcount toward the score.execution_idis the id the Automation’s executions endpoint uses, so you can fetch the execution or its trace next.
meta.next_page_token as page_token, with the same criterion and range. next_page_token is null on the last page.
Errors
bad_request: arangethe tab does not offer, anatthat is not a date or an RFC 3339 date-time, bothrangeandat, or an emptycriteria.not_found: the Automation does not exist or you cannot see its traces, the organization has traces or continuous evals off, or acriteriakey has no chart in that range orat. List the criteria for the same window first.
