# Eval

> > This page is auto-generated from the formations OpenAPI spec.
> Do not edit manually — run `pnpm generate-formations-resource-docs` to regenerate.

# Eval

> This page is auto-generated from the formations OpenAPI spec.
> Do not edit manually — run `pnpm generate-formations-resource-docs` to regenerate.

Binds an agent under test to a dataset and the scorers its outputs are judged by. `pass_threshold` is the pass rate a run must reach for its `passed` verdict — the gate an agent-version promotion consumes.

## Syntax

```yaml
type: eval
properties:
  name: String
  agent_id: String
  dataset_id: String
  scorers: Object[]
  pass_threshold: Number
```

## Output

The physical resource ID is the **public ID** of the created resource. Reference it from other resources with a `ref` expression:

```yaml
      some_field:
        ref: MyEval
```

## Properties

**`name`**

Eval name, unique within the project

_Required_: Yes
_Type_: String

---

**`agent_id`**

Public ID of the agent under test (or ref expression)

_Required_: Yes
_Type_: String

---

**`dataset_id`**

Public ID of the dataset to run against (or ref expression)

_Required_: Yes
_Type_: String

---

**`scorers`**

Scorer configs — `exact_match`, `contains`, `json_logic`, `output_schema`, `embedding_similarity`, `llm_judge`, or `tool`. Same shape as the evals REST contract.

_Required_: Yes
_Type_: Array of Object

---

**`pass_threshold`**

0–1. A run passes when its pass rate over non-errored items reaches this. Omit for a run that reports scores without a verdict.

_Required_: No
_Type_: Number
_Nullable_: Yes

---