# Exceptions

> Triageable failure and anomaly queue with severity, occurrence dedup, and an acknowledge/resolve lifecycle in SOAT.

# Exceptions

A queue of failures and anomalies surfaced as first-class, triageable items rather than log lines.

## Overview

The platform files an **exception** whenever something needs a human's attention — an orchestration run that failed after exhausting retries, a [guardrail](./guardrails.md) tripwire that aborted an action, or an [approval](./approvals.md) that expired without a decision. Each item carries a severity, structured detail, and provenance links, and moves through an `open → acknowledged → resolved` triage lifecycle. Repeated identical failures fold into one item with an occurrence count, so a hot failure loop never floods the queue.

Exceptions are **auto-filed by the platform** (or filed explicitly as `manual`); there is no public create endpoint. They are read, acknowledged, and resolved through the API, and an `exceptions.created` [webhook](./webhooks.md) fires on the first occurrence so alerting is push, not poll.

> See the [Permissions Reference](../permissions.md) for the IAM action strings for this module.

## Related Tutorials

- [Gate a Dangerous Tool with Guardrails - Step 10 (A failing guard files a tripwire exception)](/docs/tutorials/gate-a-tool-with-guardrails#step-10--a-failing-guard-the-tripwire)
- [Cap Spend Per End User - Step 10 (A cost cap with no prices protects nothing)](/docs/tutorials/cap-spend-per-end-user#step-10--a-cost-cap-with-no-prices-protects-nothing)

## Data Model

### ExceptionItem

| Field | Type | Description |
|---|---|---|
| `id` | string | Public ID, `exc_` prefix |
| `project_id` | string | Owning project |
| `status` | string | `open`, `acknowledged`, `resolved` |
| `severity` | string | `info`, `warning`, `critical` |
| `kind` | string | `run_failed`, `guardrail_tripwire`, `approval_expired`, `quota_unpriced`, `event_trigger_loop`, `chain_limit`, `manual` |
| `title` | string | Human-readable one-line summary |
| `detail` | object \| null | Structured context (tool, error, guardrail version) |
| `occurrence_count` | integer | Times this exact failure was observed while open |
| `last_seen_at` | string | Timestamp of the most recent occurrence |
| `orchestration_run_id` | string \| null | Originating orchestration run |
| `node_id` | string \| null | Originating node id within the run's graph |
| `agent_id` | string \| null | Associated agent |
| `guardrail_version` | string \| null | `<guardrailId>@<version>` for a `guardrail_tripwire` item |
| `acknowledged_by` | string \| null | Acknowledging user's public ID |
| `resolved_by` | string \| null | Resolving user's public ID |
| `resolution_note` | string \| null | Optional note recorded at resolution |
| `created_at` / `updated_at` | string | Timestamps |

## Key Concepts

### Severity

Severity is keyed to actionability, not raw "badness". Each `kind` has a default a producer can override:

| Kind | Default severity | Why |
|---|---|---|
| `run_failed` | `critical` | A run died after exhausting retries — needs intervention |
| `guardrail_tripwire` | `warning` | The guard worked as designed; also a feedback-loop signal |
| `approval_expired` | `warning` | Fail-safe missed SLA — the action never ran |
| `quota_unpriced` | `warning` | A cost cap is measuring less than it caps; needs a config fix, not incident response |
| `event_trigger_loop` | `warning` | The causation guard stopped a self-feeding [event trigger](./triggers.md#loops-and-cost); the wiring still needs a human |
| `chain_limit` | `warning` | A [continuation chain](./chains.md) spent its generation budget — the guard stopped it, and an agent that cannot terminate on its own still needs a human |
| `manual` | `warning` | Author-chosen |

### Occurrence dedup

Repeated identical failures fold into one **open** item rather than filing duplicates: a partial unique index keys at most one open exception per dedup key, and each recurrence bumps `occurrence_count` and `last_seen_at` (only the first emits `exceptions.created`). A resolved item frees the key, so a recurrence after resolution opens a fresh exception. `manual` items are never deduped.

### Triage lifecycle

An item is `open` when filed. **Acknowledge** it (`acknowledged`) to signal someone is on it — distinct from **resolve** (`resolved`, "fixed"), which records the resolver and an optional note. A resolved item is terminal: acknowledging or resolving it again returns `409 EXCEPTION_ALREADY_RESOLVED`.

### Producers

Exceptions are filed by subscribing to platform events, so producers stay decoupled: `run_failed` rides the existing `orchestration_runs.failed` event, `approval_expired` rides `approvals.expired`, and `guardrail_tripwire` rides a dedicated `guardrail.tripwire` event emitted from the guardrail dispatch path. Every filing is fire-and-forget — it never disturbs the producer.

`event_trigger_loop` is filed by the [event-trigger](./triggers.md#loops-and-cost) dispatcher when a trigger refuses to extend the causal chain that reached it — because the chain already names that trigger, or because it has run past the depth cap. It is deduped on the trigger and the reason, so a loop that keeps re-arriving is one triage item whose `occurrence_count` reads as how often it was refused; `detail` carries the chain and the event name, which is the only place that wiring is visible (the events themselves are not persisted).

`chain_limit` is filed when a [continuation chain](./chains.md) is refused for spending its generation budget. It rides a dedicated `generations.chain_limit` event and is deduped on the chain's **root generation**, which is the one id every refusal in a chain shares: an over-budget chain is refused once per resumption, so keying on the refused hop would file one item per occurrence of exactly the runaway this reports. `detail` carries the root, the initiator that asked for the refused turn, the chain's size, the budget it hit, and `limit_source` — `agent` when the agent's own [`max_chain_generations`](./agents.md#stop-conditions) refused it, `project` when the project's [`max_chain_generations`](./projects.md) did, `platform` when the deployment's ceiling did, so the number alone does not leave you guessing which knob to turn.

This is the signal that a chain stopped growing. The refusal itself is recorded on a trace and returned to a caller that is usually a background sweep with nothing left to hand it to, so without the exception a runaway would be bounded but still reach nobody until the bill arrived.

`quota_unpriced` is the exception to the event-driven pattern: it is filed inline from the [quota](./quotas.md#token-and-cost-enforcement) pre-generation check, which is the only place that knows a cost cap just evaluated against a window whose usage was not fully priced. A window that priced **nothing** and one that priced only **part** of its usage file the same item — the fix is the same price rows, named in the item's `unpriced_rows` — and it is deduped on the quota rather than the window, so one degraded cap is one triage item and `occurrence_count` reads as the number of generations that ran under it. The check fails open, so a filing error can never block a generation.

## Examples

<Tabs groupId="client">
<TabItem value="cli" label="CLI">

```bash
# List open, critical exceptions in a project
soat list-exceptions --project-id proj_01 --status open --severity critical

# Triage one
soat acknowledge-exception --exception-id exc_01
soat resolve-exception --exception-id exc_01 --note "Root cause fixed; reran the pipeline."
```

</TabItem>
<TabItem value="sdk" label="SDK">

```ts
const { data: exceptions } = await client.GET('/api/v1/exceptions', {
  params: { query: { project_id: 'proj_01', status: 'open' } },
});

await client.POST('/api/v1/exceptions/{exception_id}/resolve', {
  params: { path: { exception_id: 'exc_01' } },
  body: { note: 'Root cause fixed.' },
});
```

</TabItem>
<TabItem value="curl" label="curl">

```bash
curl -H "Authorization: Bearer $SOAT_TOKEN" \
  "$SOAT_BASE_URL/api/v1/exceptions?project_id=proj_01&status=open"

curl -X POST -H "Authorization: Bearer $SOAT_TOKEN" \
  -H "Content-Type: application/json" -d '{"note":"Root cause fixed."}' \
  "$SOAT_BASE_URL/api/v1/exceptions/exc_01/resolve"
```

</TabItem>
</Tabs>