Debugging a Run

SOAT records what happened on several surfaces, and each one is documented on its own module page. That leaves one question unanswered until you have read all of them: you are holding a symptom — which surface do you open?

This page is that map. It routes a symptom to a surface, draws the id graph the surfaces are joined on, and names the cases where a surface is empty by design rather than broken.

For what each surface is, follow the links. For the field-level contrast between the three that are most easily confused — activity, the audit log and traces — see Activity vs. the audit log vs. traces; this page does not repeat it.

Start from the symptom

Symptom	Surface	Open it with
The call came back an error	the error body, then the generation	A failed wait=true run answers GENERATION_FAILED, whose meta carries generation_id and trace_id — the thread to pull
It answered 202 and nothing came back	the generation's status	GET /generations/{generation_id} — in_progress, requires_action, completed or failed
It finished, but the answer was wrong	the transcript	GET /generations/{generation_id}/transcript — the turn read back step by step
It stopped early, or mid-sentence	stop_reason on the generation	The Stop Reason table — max_steps, depth_guard, chain_limit and length are four different bugs
A multi-agent run broke, and you don't know where	the trace tree	GET /traces/{trace_id}/tree from any trace in the run — the node carrying an error is the one that failed
The agent said it queued something, and nothing happened	the chain	GET /chains/{chain_id} — expired means an approval lapsed and nothing resumed the work
A run failed after exhausting its retries	the exceptions queue	GET /exceptions — filed for orchestration runs, tripwires, expired approvals and chain limits
A tool call never ran	the exception's guardrail_version	A guardrail_tripwire item names the exact guardrail document that stopped it
It cost more than you expected	usage events	Attribution runs to the project, agent, generation and action_id, so spend is grouped by the operation that caused it
Something changed and nobody says who	the audit log	The recorded action is the permission string that authorized it, and 403s are recorded too
"What have the agents been doing?"	the activity feed	One entry per autonomous execution — tool calls, schedule firings, approvals resolved

The first four rows are one investigation, in order: the error names the ids, the generation says how it ended, stop_reason says why, and the transcript says what it did. Most debugging never leaves them.

The id graph

Every surface above is reached from a generation, which is why a generation id is the one thing worth capturing at your own boundary.

Three of these are one call rather than a walk:

• The whole tree from any node. GET /traces/{trace_id}/tree resolves the root itself, so the id you happen to be holding is enough — no climb up parent_trace_id. Add include=generations to get each node's generations with it.
• Every turn of one run. GET /generations?trace_id= returns all generations on a trace; chain_id, orchestration_run_id and node_id slice the same list other ways.
• A run's work from its orchestration. A node execution record carries no generation id — the pointer runs the other way, from the generation's attribution columns. See Finding an orchestration run's generations.

The one join SOAT does not keep is session_id. A trace record has no session field, so correlating a customer complaint back to a run means capturing (session_id, generation_id, trace_id) at your own boundary when the generation is created. Do it on day one; it cannot be reconstructed later. Debug Session, Generation, and Trace History walks the full mapping.

When a surface is empty

An empty answer is information, and it has four different meanings. Reading it as "the platform lost my data" sends you looking in the wrong place.

What you see	What it means
steps: [], status: in_progress	The turn has not finished. Poll it.
steps: [], status: failed	The turn threw. A step is recorded when it finishes, and an aborted turn finishes none — so a run that died inside a tool call has no steps to show. The error on the generation is the whole story, and the trace carries the same payload.
steps: [] with content_redacted_at set	The content is gone — or was never written. content_redacted_by_principal_id is zero_retention when it was never stored, and the erasing principal's id when a purge or the retention sweep removed it.
An empty exceptions queue	Quite possibly the wrong surface — see below.

Two things that are not where you would look for them

A failed agent generation files no exception. Every kind names a specific producer — an orchestration run that died after exhausting retries, a guardrail tripwire, an approval that expired undecided, a chain that spent its budget, a self-feeding event trigger, a cost cap with no prices, or a manual filing. A plain POST /agents/{agent_id}/generate that failed is none of them. An empty queue is therefore not evidence that agent runs are healthy — GET /generations?status=failed is the worklist for those.

An approval-gated tool call does not leave the generation pending. The intercepted call returns { "status": "pending_approval" } as the tool result and the turn completes normally — the model closes with something like "queued for your approval". The real work happens later, in a continuation generation linked back through initiator_generation_id. So "it said it queued the refund and nothing happened" is diagnosed on the chain, not on the generation you are looking at, and expired there means nobody decided in time.

What you can debug is a configuration choice

The observability above is not a constant. Zero-retention mode never writes the steps object — and it does not write the generation's error either, because that is content by the same definition. A project on trace_content_mode: none keeps every skeleton field (ids, status, stop_reason, step_count, and the whole usage ledger), so metering, quotas and the audit log are untouched; what it gives up is precisely the material this page tells you to read.

That is a legitimate trade, and it is worth making deliberately rather than discovering mid-incident:

Setting	You keep	You give up
Default (full)	Everything on this page	Content lives until you purge it
Retention window	Full debugging inside the window	Anything older than the window, swept daily
Zero-retention	Status, stop reason, step count, cost	Steps, errors, and restart-recovery of a paused turn

Data Retention and Zero-Retention walks all three end to end.

Next

• Debug Session, Generation, and Trace History — the id mapping as a runnable walkthrough.
• Replay a Bad Turn — what to do once the transcript shows a bad answer: freeze it as a fixture, fork the session, and re-run the same context.