> ## Documentation Index
> Fetch the complete documentation index at: https://docs-platform.crewai.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Pull eval scores

# Pull eval scores

When continuous evals is on, a sample of an Automation's runs is graded against its eval file. The Evals tab charts each criterion's average score over time. The evals endpoints return the same numbers, so you can pull them into your own dashboards and alerts.

The endpoints are read-only. They need an access token for a [service account](https://docs-platform.crewai.com/api/service-account.md), and they answer `not_found` when the organization has traces or continuous evals turned off.

## 1. Find the Automation

Evals endpoints address an Automation by its `id`. Automation names are not unique, so list Automations with a name filter and pick the one you mean:

```bash theme={null}
curl -G "https://app.crewai.com/api/v1/automations" \
  -H "Authorization: Bearer $ACCESS_TOKEN" \
  --data-urlencode 'filter=name = "Research brief"'
```

## 2. List its criteria

Each criterion is one chart of the Evals tab. Pass the same `range` the tab offers: `24h`, `7d` or `30d`.

```bash theme={null}
curl "https://app.crewai.com/api/v1/automations/$AUTOMATION_ID/evals/criteria?range=7d" \
  -H "Authorization: Bearer $ACCESS_TOKEN"
```

```json theme={null}
{
  "data": [
    {"key": "task:publish:7d59a677c485", "scope": "task", "subject": "publish", "text": "Posts the link", "source": "user_defined", "graded": false},
    {"key": "crew:brief", "scope": "crew", "subject": null, "text": "A usable JSON brief", "source": "user_defined", "graded": true}
  ],
  "meta": {"bucket": "day", "start_time": "2026-11-11T15:30:00Z", "end_time": "2026-11-18T15:30:00Z"}
}
```

The list matches the tab: every criterion the current eval file declares, graded yet or not, and every criterion with a score in the range. A criterion removed from the eval file stays listed while it has scores in the range.

A criterion's `key` stays the same when its text is edited. Use it to ask for that criterion's scores.

## 3. Pull the scores

Ask for `all` criteria or a comma-separated list of keys:

```bash theme={null}
curl -G "https://app.crewai.com/api/v1/automations/$AUTOMATION_ID/evals/scores" \
  -H "Authorization: Bearer $ACCESS_TOKEN" \
  --data-urlencode "range=24h" \
  --data-urlencode "criteria=crew:brief"
```

```json theme={null}
{
  "data": [
    {
      "key": "crew:brief",
      "rows": [
        {"time": "2026-11-18T10:00:00Z", "deployment_version": 1, "score": 2.5, "samples": 1},
        {"time": "2026-11-18T13:00:00Z", "deployment_version": 2, "score": 7.5, "samples": 2}
      ]
    }
  ],
  "meta": {"bucket": "hour", "start_time": "2026-11-17T15:30:00Z", "end_time": "2026-11-18T15:30:00Z"}
}
```

Each row is one point of the chart:

* `time` is where the bucket starts. Buckets are UTC hours for `24h`, and UTC days for `7d` and `30d`.
* `deployment_version` is n in the tab's "Deployment #n". While a new version rolls out, a bucket can have a row for each version.
* `score` is the average score from 0 to 10 of the runs graded in the bucket.
* `samples` is how many graded runs that average covers.

Buckets with nothing graded are left out.

## Score at a point in time

Pass `at` instead of `range` to get the one bucket that holds a moment:

* A date, such as `at=2026-11-18`, returns that UTC day.
* A date-time with a UTC offset, such as `at=2026-11-18T14:00:00Z`, returns the UTC hour it falls in.

```bash theme={null}
curl -G "https://app.crewai.com/api/v1/automations/$AUTOMATION_ID/evals/scores" \
  -H "Authorization: Bearer $ACCESS_TOKEN" \
  --data-urlencode "at=2026-11-18" \
  --data-urlencode "criteria=all"
```

## List the graded executions behind a chart

The table under a chart lists the executions graded on that criterion. Get the same rows, newest first, a page at a time:

```bash theme={null}
curl -G "https://app.crewai.com/api/v1/automations/$AUTOMATION_ID/evals/executions" \
  -H "Authorization: Bearer $ACCESS_TOKEN" \
  --data-urlencode "criterion=crew:brief" \
  --data-urlencode "range=7d" \
  --data-urlencode "page_size=50"
```

```json theme={null}
{
  "data": [
    {
      "execution_id": "0b6f5a51-9a52-4a37-8d43-3c5b2e8b8a10",
      "executed_at": "2026-11-18T13:30:00Z",
      "deployment_version": 2,
      "score": 7.5,
      "reason": "The brief has a title and a summary but cites no source.",
      "checks": [
        {"id": "has_title", "text": "has a title", "verdict": "pass"},
        {"id": "has_summary", "text": "has a summary", "verdict": "pass"},
        {"id": "cites_source", "text": "cites a source", "verdict": "fail"}
      ],
      "executed_by": {"type": "User", "id": "f9630a31-1640-4f02-865b-3f92e4d99335"}
    }
  ],
  "meta": {"total_size": 1, "next_page_token": null}
}
```

* `reason` is the judge's overall reason for the score. It can quote the execution.
* `checks` holds the judge's verdict on each check. Only `pass` and `fail` count toward the score.
* `execution_id` is the id the Automation's executions endpoint uses, so you can fetch the execution or its trace next.

To get the next page, pass `meta.next_page_token` as `page_token`, with the same `criterion` and `range`. `next_page_token` is `null` on the last page.

## Errors

* `bad_request`: a `range` the tab does not offer, an `at` that is not a date or an RFC 3339 date-time, both `range` and `at`, or an empty `criteria`.
* `not_found`: the Automation does not exist or you cannot see its traces, the organization has traces or continuous evals off, or a `criteria` key has no chart in that range or `at`. List the criteria for the same window first.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.