Skip to main content

Generations

Generation records track individual LLM generation runs started by agents, including their lifecycle status and any failure details.

Overview​

Every agent generation (POST /agents/:id/generate, session generation, sub-agent calls) creates a generation record before the model is called. The record tracks the run through its lifecycle and, when the run fails, stores a structured error payload so failed generations are distinguishable from pending ones.

List with GET /generations (filter by agent_id, trace_id, or status); read one with GET /generations/:generation_id.

See the Permissions Reference for the IAM action strings for this module.

Data Model​

FieldTypeDescription
idstringPublic identifier for the generation
project_idstringProject the generation belongs to
agent_idstringAgent that ran the generation
trace_idstringTrace this generation belongs to
initiator_generation_idstring | nullGeneration that started this one (see Initiated generations); null for top-level generations
chain_idstring | nullContinuation chain this generation belongs to — set on every member including the root; null when it is not part of one
session_idstring | nullSession this generation was dispatched through; null when it was started outside one
actor_idstring | nullEnd-user actor the generation is attributed to, derived from the session; null when there is none
started_by_principal_typestring | nullPrincipal kind that started the generation — user or api_key (see Starting principal)
started_by_principal_idstring | nullPublic id of that principal — the key's own key_… when a key was used, else user_…
statusstringLifecycle status: in_progress, requires_action, completed, or failed
started_atstringWhen the generation started
completed_atstring | nullWhen the generation reached a terminal state
last_activity_atstring | nullLast activity timestamp
stop_reasonstring | nullWhy the generation stopped — see Stop Reason
errorobject | nullStructured error payload recorded when the generation failed (see Error Recording)
metadataobject | nullCaller-owned annotations (see Metadata and Tags and metadata)
action_idstring | nullLogical action label supplied on the generate request
trigger_idstring | nullTrigger that initiated the generation
orchestration_run_idstring | nullOrchestration run that dispatched the generation
node_idstring | nullNode within that run
node_attemptnumber | nullThe node's 1-based retry attempt, so a retried node's generations are told apart (see Finding an orchestration run's generations)
agent_versionnumber | nullAgent config version that served the generation
sourcestring | nulleval when an eval run produced this generation; null for ordinary traffic
idempotency_keystring | nullKey the generation was started under, unique within the project (see Running a generation at most once)
routingobject | nullWhat the model route did for this generation
retrievalarray | nullWhat knowledge_config retrieval injected — document, version and chunk, or memory — for this turn (see What retrieval served)
extractionobject | nullWhat each memory rule wrote for this turn, keyed by rule id (see extraction)
content_redacted_atstring | nullWhen the generation's content was purged; null while content is intact
content_redacted_by_principal_typestring | nullPrincipal kind that purged the content (user or api_key)
content_redacted_by_principal_idstring | nullPublic ID of that principal — the key's own id for API-key auth
created_atstringISO 8601 creation timestamp
updated_atstringISO 8601 last-update timestamp

Key Concepts​

Starting principal​

started_by_principal_type / started_by_principal_id record who started the generation. An API-key request records the key itself (key_…), so the generation names which key acted rather than only the owning user; a JWT-authenticated request records the user (user_…).

The pair is durable identity: work that resumes after the original request is gone re-mints a short-lived credential from it. That lets an approval continuation, possibly days later, authenticate its builtin tools as the principal that started the chain, and it is why a generation started by a request-less drive (a workflow dispatch, an orchestration node) records the drive's principal.

Both fields are null when the chain has no re-mintable principal: a generation started by a trigger or an OAuth token. Each carries its authority in the token (the trigger's attached policy, the consented scope), so recording a principal would let a later re-mint act with the whole of the owning user's access.

Lifecycle​

A generation starts as in_progress. It transitions to:

  • requires_action when a client tool call pauses the run and the caller must submit tool outputs.
  • completed when the model finishes (the stop_reason carries the finish reason).
  • failed when the run errors, for example when the upstream AI provider returns an error or is unreachable. stop_reason is set to error and the error field carries the failure details.

Error Recording​

When a generation fails, the failure is persisted on both the generation record and its trace: status becomes failed, stop_reason is error, and error carries { code, message }.

error always contains message. code is set for mapped errors, most notably AI_PROVIDER_ERROR: the upstream AI provider returned an error (e.g. exhausted credits, rate limit) or is unreachable.

Provider Error Surfacing (AI_PROVIDER_ERROR)​

Generation endpoints return HTTP 502 with the AI_PROVIDER_ERROR code when the upstream AI provider fails:

{
  "error": {
    "code": "AI_PROVIDER_ERROR",
    "message": "Provider returned 402: insufficient credits",
    "meta": {
      "provider_status_code": 402,
      "generation_id": "gen_abc123",
      "trace_id": "trace_xyz789"
    }
  }
}

meta carries the generation_id and trace_id of the failed run for inspection via GET /generations/:generation_id and GET /traces/:trace_id.

Metadata​

metadata is a caller-owned bag, returned verbatim, for per-run audit attribution (e.g. the ticket or case an AI action belongs to). What knowledge the turn read is not the caller's to record: the server writes it to retrieval.

PATCH requires generations:UpdateGeneration; the create path requires agents:CreateAgentGeneration.

No key is reserved. Every piece of server-owned state (action_id, trigger_id, orchestration_run_id, node_id, agent_version, routing, extraction, retrieval) is a field of its own, so nothing written into metadata can reach it; a caller key spelled action_id is just an annotation.

Internal recovery state (used to resume a requires_action generation after a server restart) is stored in its own column and is never exposed through the API.

extraction — memory-rule summary​

Every memory rule bound to agents.generation.completed that fired for this turn records what it wrote, keyed by the rule's id:

{ "mrule_V1StGXR8Z5jdHi6B": { "candidates": 3, "created": 2, "superseded": 1, "skipped": 0 } }

A store can have several rules, so one flat pair of counts could not say which produced them. Absent when no rule fired. Rules bound to conversations.message.generated are visible only in GET /api/v1/generations/{generation_id}/memory-assertions, which lists the rows behind every count either way.

Recorded input​

A generation also stores the messages it was asked to answer, resolved (file and document references inlined) but without the agent's own instructions or knowledge injections, which are config recoverable from agent_version.

The record is served by the transcript, not the generation response, and exists so a real turn can be promoted into an evaluation fixture with create-dataset-item-from-generation. It is content, not skeleton: never written under zero-retention, cleared by a purge, swept by retention. A generation whose input is gone can no longer be curated and answers 409 GENERATION_CONTENT_UNAVAILABLE.

What retrieval served​

An agent with a knowledge_config retrieves before the model is called, and retrieval records what that injected, in injection order:

"retrieval": [
  {
    "source_type": "document",
    "document_id": "doc_V1StGXR8Z5jdHi6B",
    "document_version": 3,
    "chunk_id": "dchunk_V1StGXR8Z5jdHi6B",
    "page": null,
    "similarity_score": 0.82
  },
  {
    "source_type": "memory",
    "memory_store_id": "mstore_V1StGXR8Z5jdHi6B",
    "memory_id": "mem_V1StGXR8Z5jdHi6B",
    "similarity_score": 0.74
  }
]

Together with agent_version, it answers "which playbook version produced this action". document_version is the durable pointer: a document edited between two turns leaves each citing the version it read, and the document's versions keep that text. chunk_id stops resolving once the document is re-chunked. The array order is the rank. similarity_score is raw cosine, the number that keeps its meaning once stored; null when the turn had no query or the search answered from the lexical channel alone.

null means no retrieval ran — no knowledge_config, or neither a query nor a filter to search with. [] means it ran and matched nothing.

It is written by the server when the turn starts and cannot be set by a caller. Being pointers and no text, it is not content: it survives a purge and is written even on a zero-retention agent.

Transcript​

GET /generations/{generation_id}/transcript reads one turn back step by step: what it was asked, each model step with its tool calls and results, and how it ended.

soat get-generation-transcript --generation-id gen_abc

The transcript is assembled at read time from the generation record and the trace's steps object; there is no transcript table and no extra write on the generation path, so it always reflects the current records.

Requires traces:GetTrace in addition to generations:GetGeneration, since the response merges content from both resources.

Each entry in steps carries index, text, finish_reason, tool_calls, tool_results and usage. args on a call and result on a result are tool-owned payloads, passed through exactly as recorded.

The stored steps are projected, never forwarded; their on-disk shape belongs to the ai package.

Two states return 200 with a skeleton rather than an error, so a caller never has to distinguish "no content" from "no such generation":

Statestatusinput / outputstepscontent_redacted_at
Still runningin_progressnull[]null
Never stored (zero-retention)terminalnull[]set, principal zero_retention
Erased by a purge or sweepterminalnull[]set, purging principal

step_count survives all three, being a counter rather than content. It counts this turn's steps: when a trace_id groups several generations, the trace's own step_count covers every one of them, while each transcript reports only its own slice — see Traces → Grouping Generations Under One Trace.

A purged generation returns the skeleton even though the trace's steps object may still exist (see the warning under Content Purge); the redaction marker governs the whole transcript.

What the turn cost, and what it cost to ask​

Two different figures, deliberately kept apart.

usage is what the provider reported and was billed for — token counts and cost_usd, in the shape every usage surface uses. Present on the single-generation read, omitted from the listing (it is a query per row), and null until the turn is metered.

tool_surface is what SOAT sent: how many tool definitions the turn carried and how large they were.

FieldMeaning
toolsTools resolved for the turn, before any per-step narrowing by step_rules
bytesSerialized length of the definitions — name, description, input schema — in canonical JSON, not the provider's wire format, so two agents are comparable
estimated_tokensbytes over a measured bytes-per-token ratio

It answers the one question the meter cannot. A provider reports totals, and the tool block is a constant inside them — byte-identical on every step of a turn — so it cannot be recovered by differencing steps either. An agent bound to a large MCP catalogue can spend most of its prompt describing tools it never calls, and without this nothing says so.

tool_surface is not a meter

estimated_tokens is an estimate and is never priced. usage and the usage events are the billing record.

null means the surface was never measured — a generation from before the field, or a path that resolves none. That is not {"tools": 0}, which is a measured agent with nothing bound. Reading the two as one would make every older row look tool-less.

Being three integers and no text, tool_surface is not content: it survives a purge and is written even on a zero-retention agent.

Content Purge​

DELETE /generations/{generation_id}/content clears the generation's content — metadata, error, extraction, the recorded input messages, and the internal recovery state of a paused run — and stamps content_redacted_at. It requires the generations:PurgeGenerationContent action.

The usage and audit skeleton is preserved (the billing ledger outlives the erasure): ids, timestamps, status, stop reason, and every attribution field (action_id, trigger_id, orchestration_run_id, node_id, node_attempt, agent_version, routing, retrieval). A purged generation reads back as that skeleton, not a 404.

The operation is idempotent: a second purge succeeds and leaves the original content_redacted_at untouched.

warning

A generation purge does not delete the parent trace's steps object, which holds this generation's content alongside its siblings'. To erase a run's content completely, purge the trace — DELETE /traces/{trace_id}/content deletes the steps bytes from storage and cascades the content purge to every generation in the tree. See Traces.

Automatic content lifecycle​

Two project settings turn the manual purge into a policy:

  • Retention — trace_content_retention_days on the project runs a daily sweep that purges content past the window, through this same purge path.
  • Zero-retention — trace_content_mode: "none" on the project or the agent means the content columns are never written. The generation is still created and metered; it reads back as a skeleton stamped content_redacted_by_principal_id: "zero_retention" from the moment it exists.

Initiated generations​

initiator_generation_id names the generation that started this one; top-level generations leave it null. It is set on:

  • a sub-agent invocation — an agent calling another agent via a builtin tool;
  • an approval continuation — the turn that resumes after an approval settles, in a session or standalone;
  • a client-tool re-handoff — the generation an approved client tool call is handed back to the caller on;
  • a memory-rule handler turn — the agent a memory rule runs on a finished turn.

Intermediate steps of multi-step reasoning composed by the calling application are ordinary generations of their own, not metadata on or children of the calling generation.

Finding an orchestration run's generations​

An orchestration run's node executions record what each node received and produced but carry no generation id. The pointer runs the other way: a generation dispatched by an agent node stores orchestration_run_id, node_id and node_attempt as attribution columns, next to action_id and trigger_id. Filter the list endpoint:

# every generation the run produced
soat list-generations --orchestration-run-id run_abc123

# just one node's — one row per attempt if the node was retried
soat list-generations --orchestration-run-id run_abc123 --node-id summarize

node_attempt distinguishes the generations of a retried node: one node execution record and one generation per attempt, matched exactly on node_attempt.

From a generation reached this way, the rest of the graph is reachable: trace_id opens the trace for that turn, initiator_generation_id walks down into any generations it started, chain_id opens the continuation chain it belongs to (filtering generations by that id returns every member), and session_id / actor_id name the session and end user it ran for, the same pair its usage event is attributed to.

session_id and actor_id also filter the listing, so the turns behind a conversation's or an end user's cost are one call away:

soat list-generations --session-id sess_abc123
soat list-generations --actor-id actor_abc123

An id naming nothing in scope yields an empty page, never an unfiltered one.

Tool context​

The generation-creation endpoints (POST /agents/{agent_id}/generate, and the session and conversation generate endpoints) accept an optional tool_context object. Its entries are forwarded as X-Soat-Context-* request headers on every http, mcp and builtin tool call the generation makes; an invalid key is rejected with 400 INVALID_TOOL_CONTEXT_KEY before the provider is called. It is not persisted on the Generation record. See the Tool Context reference.

Who may act on a generation​

Every route that acts on one generation — read it, attach metadata, purge its content, or read its transcript — is authorized against that generation's SRN, srn:<project_id>:generation:<generation_id>, not against the project. A policy may therefore name the generations it covers:

{
  "statement": [
    {
      "effect": "Allow",
      "action": ["generations:GetGeneration", "generations:UpdateGeneration"],
      "resource": ["srn:proj_V1StGXR8Z5jdHi6B:generation:gen_V1StGXR8Z5jdHi6B"]
    }
  ]
}

The transcript projects the generation and its trace, so it checks both: generations:GetGeneration against the generation and traces:GetTrace against the trace it was recorded on. A grant naming only one of them does not reach it.

Refusals keep the shapes IAM defines: a read the caller may not perform is 404 (a generation it may not see does not announce itself), a write is 403, and a credential scoped to another project is 403 API_KEY_PROJECT_SCOPE. A write on a resource in a project none of the caller's policies name is 404 too — the same answer their read would get, so a refusal never confirms existence across a tenant boundary. The trace half of a transcript is 403 rather than 404: the generation read has already succeeded by then, so there is nothing left to conceal.

Listing generations stays project-scoped: GET /api/v1/generations asks whether the caller may list generations in a project at all, so a policy that names individual generations grants no listing.

Examples​

List generations​

Filter by agent_id, trace_id, initiator_generation_id, chain_id, orchestration_run_id, node_id, or status.

soat list-generations --trace-id trace_abc123 --status failed

Get a generation​

soat get-generation --generation-id gen_abc123

List a generation's memory writes​

soat list-generation-memory-assertions --generation-id gen_abc123

Attach audit metadata​

Merge caller-supplied metadata onto a generation for per-run audit attribution.

soat update-generation --generation-id gen_abc123 \
  --metadata '{"team":"payments","ticket_id":"OPS-4821"}'