Exceptions

A queue of failures and anomalies surfaced as first-class, triageable items rather than log lines.

Overview

The platform files an exception whenever something needs a human's attention — an orchestration run that failed after exhausting retries, a guardrail tripwire that aborted an action, or an approval that expired without a decision. Each item carries a severity, structured detail, and provenance links, and moves through an open → acknowledged → resolved triage lifecycle. Repeated identical failures fold into one item with an occurrence count, so a hot failure loop never floods the queue.

Exceptions are auto-filed by the platform (or filed explicitly as manual); there is no public create endpoint. They are read, acknowledged, and resolved through the API, and an exceptions.created webhook fires on the first occurrence so alerting is push, not poll.

See the Permissions Reference for the IAM action strings for this module.

Related Tutorials

• Gate a Dangerous Tool with Guardrails - Step 10 (A failing guard files a tripwire exception)
• Cap Spend Per End User - Step 10 (A cost cap with no prices protects nothing)

Data Model

ExceptionItem

Field	Type	Description
id	string	Public ID, exc_ prefix
project_id	string	Owning project
status	string	open, acknowledged, resolved
severity	string	info, warning, critical
kind	string	run_failed, guardrail_tripwire, approval_expired, quota_unpriced, event_trigger_loop, chain_limit, manual
title	string	Human-readable one-line summary
detail	object | null	Structured context (tool, error, guardrail version)
occurrence_count	integer	Times this exact failure was observed while open
last_seen_at	string	Timestamp of the most recent occurrence
orchestration_run_id	string | null	Originating orchestration run
node_id	string | null	Originating node id within the run's graph
agent_id	string | null	Associated agent
guardrail_version	string | null	<guardrailId>@<version> for a guardrail_tripwire item
acknowledged_by	string | null	Acknowledging user's public ID
resolved_by	string | null	Resolving user's public ID
resolution_note	string | null	Optional note recorded at resolution
created_at / updated_at	string	Timestamps

Key Concepts

Severity

Severity is keyed to actionability, not raw "badness". Each kind has a default a producer can override:

Kind	Default severity	Why
run_failed	critical	A run died after exhausting retries — needs intervention
guardrail_tripwire	warning	The guard worked as designed; also a feedback-loop signal
approval_expired	warning	Fail-safe missed SLA — the action never ran
quota_unpriced	warning	A cost cap is measuring less than it caps; needs a config fix, not incident response
event_trigger_loop	warning	The causation guard stopped a self-feeding event trigger; the wiring still needs a human
chain_limit	warning	A continuation chain spent its generation budget — the guard stopped it, and an agent that cannot terminate on its own still needs a human
manual	warning	Author-chosen

Occurrence dedup

Repeated identical failures fold into one open item rather than filing duplicates: a partial unique index keys at most one open exception per dedup key, and each recurrence bumps occurrence_count and last_seen_at (only the first emits exceptions.created). A resolved item frees the key, so a recurrence after resolution opens a fresh exception. manual items are never deduped.

Triage lifecycle

An item is open when filed. Acknowledge it (acknowledged) to signal someone is on it — distinct from resolve (resolved, "fixed"), which records the resolver and an optional note. A resolved item is terminal: acknowledging or resolving it again returns 409 EXCEPTION_ALREADY_RESOLVED.

Producers

Exceptions are filed by subscribing to platform events, so producers stay decoupled: run_failed rides the existing orchestration_runs.failed event, approval_expired rides approvals.expired, and guardrail_tripwire rides a dedicated guardrail.tripwire event emitted from the guardrail dispatch path. Every filing is fire-and-forget — it never disturbs the producer.

event_trigger_loop is filed by the event-trigger dispatcher when a trigger refuses to extend the causal chain that reached it — because the chain already names that trigger, or because it has run past the depth cap. It is deduped on the trigger and the reason, so a loop that keeps re-arriving is one triage item whose occurrence_count reads as how often it was refused; detail carries the chain and the event name, which is the only place that wiring is visible (the events themselves are not persisted).

chain_limit is filed when a continuation chain is refused for spending its generation budget. It rides a dedicated generations.chain_limit event and is deduped on the chain's root generation, which is the one id every refusal in a chain shares: an over-budget chain is refused once per resumption, so keying on the refused hop would file one item per occurrence of exactly the runaway this reports. detail carries the root, the initiator that asked for the refused turn, the chain's size, the budget it hit, and limit_source — agent when the agent's own max_chain_generations refused it, project when the project's max_chain_generations did, platform when the deployment's ceiling did, so the number alone does not leave you guessing which knob to turn.

This is the signal that a chain stopped growing. The refusal itself is recorded on a trace and returned to a caller that is usually a background sweep with nothing left to hand it to, so without the exception a runaway would be bounded but still reach nobody until the bill arrived.

quota_unpriced is the exception to the event-driven pattern: it is filed inline from the quota pre-generation check, which is the only place that knows a cost cap just evaluated against a window whose usage was not fully priced. A window that priced nothing and one that priced only part of its usage file the same item — the fix is the same price rows, named in the item's unpriced_rows — and it is deduped on the quota rather than the window, so one degraded cap is one triage item and occurrence_count reads as the number of generations that ran under it. The check fails open, so a filing error can never block a generation.

Examples

CLI

# List open, critical exceptions in a project
soat list-exceptions --project-id proj_01 --status open --severity critical

# Triage one
soat acknowledge-exception --exception-id exc_01
soat resolve-exception --exception-id exc_01 --note "Root cause fixed; reran the pipeline."

SDK

const { data: exceptions } = await client.GET('/api/v1/exceptions', {
  params: { query: { project_id: 'proj_01', status: 'open' } },
});

await client.POST('/api/v1/exceptions/{exception_id}/resolve', {
  params: { path: { exception_id: 'exc_01' } },
  body: { note: 'Root cause fixed.' },
});

curl

curl -H "Authorization: Bearer $SOAT_TOKEN" \
  "$SOAT_BASE_URL/api/v1/exceptions?project_id=proj_01&status=open"

curl -X POST -H "Authorization: Bearer $SOAT_TOKEN" \
  -H "Content-Type: application/json" -d '{"note":"Root cause fixed."}' \
  "$SOAT_BASE_URL/api/v1/exceptions/exc_01/resolve"