Skip to main content

Dataset Item

This page is auto-generated from the formations OpenAPI spec. Do not edit manually — run pnpm generate-formations-resource-docs to regenerate.

One test case in a dataset: the messages sent to the agent under test and, optionally, the reference answer scorers compare against. Editing or removing an item never rewrites a run that already scored it — each result froze its own copy.

Syntax​

type: dataset_item
properties:
  dataset_id: String
  input: Object[]
  expected_output: String
  metadata: Object

Output​

The physical resource ID is the public ID of the created resource. Reference it from other resources with a ref expression:

      some_field:
        ref: MyDatasetItem

Properties​

dataset_id

Public ID of the parent dataset (or ref expression)

Required: Yes Type: String

input

The messages sent to the agent, as {role, content} objects

Required: Yes Type: Array of Object

expected_output

Reference answer for exact_match / contains / embedding_similarity / llm_judge scorers

Required: No Type: String Nullable: Yes

metadata

Free-form tags on the case, e.g. {"topic": "billing"}

Required: No Type: Object Nullable: Yes