# Evaluations

> SDK methods for the Evaluations module, accessed via `soat.evaluations`. See the [Evaluations module docs](/docs/modules/evaluations) for permissions and data model, and each method's REST page for the full request/response schema.

# Evaluations

SDK methods for the Evaluations module, accessed via `soat.evaluations`. See the [Evaluations module docs](/docs/modules/evaluations) for permissions and data model, and each method's REST page for the full request/response schema.

### `soat.evaluations.listDatasets()`

List datasets

- Method: `GET`
- Path: `/api/v1/datasets`

```ts
await soat.evaluations.listDatasets({
  query: { /* + optional */ },
});
```

→ [Request & response schema](/docs/api/evaluations/list-datasets)

### `soat.evaluations.createDataset()`

Create a dataset

- Method: `POST`
- Path: `/api/v1/datasets`

```ts
await soat.evaluations.createDataset({
  body: { name, /* + optional */ },
});
```

→ [Request & response schema](/docs/api/evaluations/create-dataset)

### `soat.evaluations.getDataset()`

Get a dataset

- Method: `GET`
- Path: `/api/v1/datasets/{dataset_id}`

```ts
await soat.evaluations.getDataset({
  path: { dataset_id },
});
```

→ [Request & response schema](/docs/api/evaluations/get-dataset)

### `soat.evaluations.updateDataset()`

Update a dataset

- Method: `PUT`
- Path: `/api/v1/datasets/{dataset_id}`

```ts
await soat.evaluations.updateDataset({
  path: { dataset_id },
  body: { /* + optional */ },
});
```

→ [Request & response schema](/docs/api/evaluations/update-dataset)

### `soat.evaluations.deleteDataset()`

Delete a dataset

- Method: `DELETE`
- Path: `/api/v1/datasets/{dataset_id}`

```ts
await soat.evaluations.deleteDataset({
  path: { dataset_id },
});
```

→ [Request & response schema](/docs/api/evaluations/delete-dataset)

### `soat.evaluations.listDatasetItems()`

List dataset items

- Method: `GET`
- Path: `/api/v1/datasets/{dataset_id}/items`

```ts
await soat.evaluations.listDatasetItems({
  path: { dataset_id },
  query: { /* + optional */ },
});
```

→ [Request & response schema](/docs/api/evaluations/list-dataset-items)

### `soat.evaluations.createDatasetItem()`

Add a dataset item

- Method: `POST`
- Path: `/api/v1/datasets/{dataset_id}/items`

```ts
await soat.evaluations.createDatasetItem({
  path: { dataset_id },
  body: { input, /* + optional */ },
});
```

→ [Request & response schema](/docs/api/evaluations/create-dataset-item)

### `soat.evaluations.createDatasetItemFromGeneration()`

Curate a dataset item from a generation

- Method: `POST`
- Path: `/api/v1/datasets/{dataset_id}/items/from-generation`

```ts
await soat.evaluations.createDatasetItemFromGeneration({
  path: { dataset_id },
  body: { generation_id, /* + optional */ },
});
```

→ [Request & response schema](/docs/api/evaluations/create-dataset-item-from-generation)

### `soat.evaluations.updateDatasetItem()`

Update a dataset item

- Method: `PUT`
- Path: `/api/v1/datasets/{dataset_id}/items/{item_id}`

```ts
await soat.evaluations.updateDatasetItem({
  path: { dataset_id, item_id },
  body: { /* + optional */ },
});
```

→ [Request & response schema](/docs/api/evaluations/update-dataset-item)

### `soat.evaluations.deleteDatasetItem()`

Delete a dataset item

- Method: `DELETE`
- Path: `/api/v1/datasets/{dataset_id}/items/{item_id}`

```ts
await soat.evaluations.deleteDatasetItem({
  path: { dataset_id, item_id },
});
```

→ [Request & response schema](/docs/api/evaluations/delete-dataset-item)

### `soat.evaluations.listEvals()`

List evals

- Method: `GET`
- Path: `/api/v1/evals`

```ts
await soat.evaluations.listEvals({
  query: { /* + optional */ },
});
```

→ [Request & response schema](/docs/api/evaluations/list-evals)

### `soat.evaluations.createEval()`

Create an eval

- Method: `POST`
- Path: `/api/v1/evals`

```ts
await soat.evaluations.createEval({
  body: { name, agent_id, dataset_id, scorers, /* + optional */ },
});
```

→ [Request & response schema](/docs/api/evaluations/create-eval)

### `soat.evaluations.getEval()`

Get an eval

- Method: `GET`
- Path: `/api/v1/evals/{eval_id}`

```ts
await soat.evaluations.getEval({
  path: { eval_id },
});
```

→ [Request & response schema](/docs/api/evaluations/get-eval)

### `soat.evaluations.updateEval()`

Update an eval

- Method: `PUT`
- Path: `/api/v1/evals/{eval_id}`

```ts
await soat.evaluations.updateEval({
  path: { eval_id },
  body: { /* + optional */ },
});
```

→ [Request & response schema](/docs/api/evaluations/update-eval)

### `soat.evaluations.deleteEval()`

Delete an eval

- Method: `DELETE`
- Path: `/api/v1/evals/{eval_id}`

```ts
await soat.evaluations.deleteEval({
  path: { eval_id },
});
```

→ [Request & response schema](/docs/api/evaluations/delete-eval)

### `soat.evaluations.listEvalRuns()`

List eval runs

- Method: `GET`
- Path: `/api/v1/evals/{eval_id}/runs`

```ts
await soat.evaluations.listEvalRuns({
  path: { eval_id },
  query: { /* + optional */ },
});
```

→ [Request & response schema](/docs/api/evaluations/list-eval-runs)

### `soat.evaluations.startEvalRun()`

Start an eval run

- Method: `POST`
- Path: `/api/v1/evals/{eval_id}/runs`

```ts
await soat.evaluations.startEvalRun({
  path: { eval_id },
  body: { /* + optional */ },
});
```

→ [Request & response schema](/docs/api/evaluations/start-eval-run)

### `soat.evaluations.getEvalRun()`

Get an eval run

- Method: `GET`
- Path: `/api/v1/evals/{eval_id}/runs/{eval_run_id}`

```ts
await soat.evaluations.getEvalRun({
  path: { eval_id, eval_run_id },
});
```

→ [Request & response schema](/docs/api/evaluations/get-eval-run)

### `soat.evaluations.listEvalResults()`

List eval run results

- Method: `GET`
- Path: `/api/v1/evals/{eval_id}/runs/{eval_run_id}/results`

```ts
await soat.evaluations.listEvalResults({
  path: { eval_id, eval_run_id },
  query: { /* + optional */ },
});
```

→ [Request & response schema](/docs/api/evaluations/list-eval-results)

### `soat.evaluations.cancelEvalRun()`

Cancel an eval run

- Method: `POST`
- Path: `/api/v1/evals/{eval_id}/runs/{eval_run_id}/cancel`

```ts
await soat.evaluations.cancelEvalRun({
  path: { eval_id, eval_run_id },
});
```

→ [Request & response schema](/docs/api/evaluations/cancel-eval-run)