The Layers of an Agent System

An agent system decomposes into four layers:

• the harness — what the agent can see and do, what survives between runs, what it must not touch;
• the loop — how work is checked, what evidence proves success, when it stops;
• the graph — what happens next, what runs in parallel, where a human signs off;
• the ratchet — how a change is proven an improvement before it reaches production.

Investment order, per You Probably Don't Need a Graph: harness first, loop second, graph last, and last frequently means never. The ratchet acts on the other three and belongs as soon as the loop produces evidence you trust.

Layer	The question it answers	SOAT modules
Harness	What can this agent reach, and what is it forbidden?	Tools, Knowledge, Documents, Memories, Sessions, IAM, Secrets, Formations
Loop	What proves it did the job, and when does it stop?	Agents (output_schema, max_steps, stop_conditions), Guardrails, Approvals, Quotas, Usage, Traces, Exceptions
Graph	What is allowed to happen next?	Orchestrations, Workflows, Triggers
Ratchet	How does the system change, and what proves the change was an improvement?	Evaluations, agent versions, the approvals recurrence view, Guardrails, Formations

Layer 1 — The harness

Most failures live here: wrong tool, stale state, lost context, unscoped permissions.

What the agent can do

Tools are project-scoped resources shared across agents. Four types:

Type	Reaches
http	Any HTTP endpoint
mcp	An external MCP server
client	Client-side execution — the run pauses and resumes with the result
builtin	SOAT platform actions, including invoking another agent

Agents narrow the surface with three dials: active_tool_ids restricts which bound tools are live, tool_choice forces or forbids a call, and step_rules changes both per step. Shrink these before adding structure.

What the agent can see

Knowledge is one semantic search across a project's Documents and memory entries, ranked by vector similarity and tagged by source. knowledge_config on the agent runs retrieval inside the run.

Ingestion path: Files → Documents → Embeddings; Ingestion Rules turn uploads into searchable documents automatically.

What survives between runs

Sessions: one user ↔ one agent with managed history. Conversations: the multi-party layer underneath. Actors: the participants. Durable facts belong in Memories, read through knowledge_config and written through the built-in write_memory tool. See Agent with Persistent Memory. When many agents write the same documents and memories, Single Source of Truth maps what the shared record needs to the module that provides it.

What it must never touch

Three independent scopes:

• IAM and Policies — the caller's resource:Action permissions, enforced identically across REST, MCP, CLI, and SDK.
• boundary_policy on the agent — a ceiling on which builtin actions the agent may perform; combined with the caller's permissions, an agent can never exceed them.
• Secrets — encrypted provider credentials referenced by AI Providers; no key travels in a request body or instructions.

Making the harness reproducible

Formations declare agents, tools, providers, knowledge, and their wiring as one deployable stack. See the Formations tutorial.

Layer 2 — The loop

Every loop needs a success predicate, a budget, and an escalation path.

A success predicate — evidence, not assertion

Evidence	SOAT primitive
The output validates	output_schema on the agent — the result is checked against a schema
The action is permitted at call time	Guardrails classify each tool call from its actual arguments, deterministically, with no LLM in the evaluation path
The claim is grounded	Knowledge results carry a source_type and a link to the document or memory entry they came from
A branch condition holds	A condition node in an orchestration, evaluated over JSON Logic

Guardrails run after the model produces the call and before anything touches the outside world, composing stricter-wins across project, agent, and tool scope. A class-C action becomes an approval item instead of executing. See Gate a Dangerous Tool with Guardrails.

A budget — a loop with no ceiling is a cost leak

• max_steps and stop_conditions bound a single agent run.
• Quotas fail closed on an aggregate cap (requests, tokens, or cost_usd) scoped to a project, API key, agent, or end user, blocking with 429 QUOTA_EXCEEDED before the generation starts.
• Usage meters every LLM call, orchestration node execution, API request, and stored byte, with thresholds that alert before a quota bites.
• Per-node retry in an orchestration takes fixed or exponential backoff with a delay ceiling.

See Metering and Budgets and Cap Spend per End User.

An escalation path — where the loop hands off

Approvals is the queue a risky action lands in: the action is frozen, evidence travels with it, a hard expiry bounds it. A human approves, edits-then-approves, or rejects. Exhausted retries, a guardrail tripwire, or expiry without a decision file an Exception: a triageable item with severity and occurrence dedup.

Closing the loop needs to be observable

Traces record every reasoning step and tool call, linking parent to child so a multi-agent run reconstructs as one tree. Generations carry per-run status and result, Activity what agents did autonomously, the Audit Log what principals did to the platform. See Debug a Session Generation.

Layer 3 — The graph, last

Reach for a graph when you observe one of these:

Pain you actually observed	What SOAT gives you
A mandatory human gate that must be enforced, not requested	An approval node in an orchestration, or requires_approval on a workflow transition, which parks the state change itself
An audit requirement — the path taken must be reconstructable	A node execution record per attempt, a linked trace, and append-only transition history with principal_type on the workflow side
Expensive parallel stages needing deterministic joins	Parallel execution rounds plus activation_group fan-in
Durable resumption — interrupted at hour six, restart at step nine	A durable queue with leases and a reaper, the sleeping / awaiting_input statuses, and resume/cancel on a run

These are properties of the process, not the model, so they do not improve as models do.

Two shapes: an orchestration is a directed acyclic pipeline that runs and ends; a workflow is a state graph a task lives in and can move backward through. They compose in both directions: Choosing an Automation Model.

Triggers start a graph on a cron schedule, an inbound webhook, or on demand.

Layer 4 — The ratchet

The loop asks did this run succeed?; the ratchet asks did this change make the agent better? A reworded instruction, a swapped model, an added tool, a correction applied in chat for the fourth time: each is a production change with no verdict attached.

The ratchet's shape: produce a verdict from evidence, gate the change on the verdict, keep the history append-only. Current state of each piece: Status.

From a capability to a decision

A verdict supports a claim, and the claim is written before the evidence. Each verb of the ratchet's shape owns a span of the nine steps that turn a capability into an engineering decision.

Produce a verdict from evidence (steps 1–8):

Step	Artifact	Where it lives
1. Capability	The pain observed: a correction that recurs, an exception, a turn that went wrong	Approvals recurrence view, Exceptions, Replay a Bad Turn
2. Construct	The observable property that stands for the capability	One scorer per property; a decider when production already asks the question
3. Claim	The sentence a passing run authorizes about the agent, over which inputs	Written before the dataset: Eval Design
4. Evidence required	The kinds of input that must be present and the pass rate the claim needs	group_by over the items' metadata, pass_threshold: Eval Design
5. Cases	The dataset, weighted toward failures, curated from production	Datasets, a dev and a held-out set
6. Measurements	One real generation per item, against one pinned agent version	Eval runs
7. Interpretation	The pass rate with its interval, per kind, against a baseline	Uncertainty, Grouped aggregates, Baseline deltas
8. Supported claim	The run's verdict	Pass semantics

Gate the change on the verdict (step 9): promoting the canary, encoding a guardrail deny or rewording an instruction waits for the verdict. Eval-gated promotion is the gate.

Keep the history append-only: the decision creates an agent version, every generation records the version that served it, and production yields the next step 1. A run against the new version, with the old version's run as its baseline_run_id, reports what moved.

It is not simply "after the graph"

The ratchet acts on the other three layers and needs no graph. Order in practice: build the harness, close the loop, ratchet the loop as soon as it produces evidence you trust, add a graph only when a process pain demands one.

Pain you actually observed	What SOAT gives you
You changed an instruction, model, or tool and cannot say whether it got better	An evaluation — a dataset of cases, scorers that produce a number, and a run compared against a named baseline
A regression is live and nobody can name what changed	Append-only agent versions, with the served version stamped on every generation as agent_version
Promotion is a judgment call made under deadline	A staged release (stable/canary split) whose promotion is gated on an eval verdict
A human keeps making the same correction	The approvals recurrence view rolls up repeated rejections with their reasons — the prompt to encode a guardrail deny that stops the pattern upstream
Guidance was added to an agent and nobody knows if it works	Instructions are soft context, so efficacy is an eval question: run the regression set with and without the wording and compare against the baseline

A verdict, not an opinion

Evaluations: a project-scoped dataset holds the cases; scorers grade each output (deterministic: exact match, substring, JSON Logic, schema validation; an LLM judge for fuzzy criteria; a decider, so the eval asks the questions production asks; or your own algorithm as a tool); a run executes the real agent (its true instructions, tools, model, and knowledge) against every item. Compared against a named baseline it yields per-scorer deltas and a pass/fail verdict; every result links to its generation and trace.

Eval runs are real generations; eval spend is attributed separately from production spend in usage rollups. Agents are stochastic: judge aggregates over a dataset, not single items, and set a pass threshold below 1.0 (Eval Design).

Change that cannot slide backward

Agent versions: every write that changes an agent's config archives it and increments version; a no-op write creates no version. Restore copies an archived config forward as a new version. Every generation records the agent_version that served it.

A staged release splits traffic deterministically between a stable and a canary version; the canary accumulates real evidence before promotion.

Corrections that outlive the conversation

Every approval rejection carries its reason, every edit-then-approve carries the argument diff, and re-proposals thread onto the item they recur from. The recurrence view rolls that up by exact key, with no LLM in the path; it reports that a correction recurs and leaves the judgment to a human.

Memories are facts the agents learn about the world; a correction is doctrine about how the agent should act. Doctrine has two homes:

• Hard — a guardrail deny, refusing the action upstream. For constraints that must never be violated; recurring rejections are the signal.
• Soft — the agent's instructions, which the model is expected but not forced to follow, archived by agent versions on every write.

Whether a wording change improved anything is an eval, not a judgment.

What the ratchet must never do

Promote by itself. The platform owns the queue, the recurrence signal, and the verdict; a human owns the judgment: rewording an instruction, encoding a guardrail deny, and promoting a canary stay human-gated.

The pieces for a self-modifying loop exist: a builtin tool can expose update-agent, update-guardrail, set-agent-release, start-eval-run, promote-agent-release, so an agent with those actions and a permitting credential can propose changes to its own definitions, with eval-gated promotion as the deterministic brake. Two boundaries mark where platform enforcement ends:

• The promotion gate verifies evidence; it does not produce it. promote fails closed (409 PROMOTION_GATE_UNMET) until a passing run pinned to the canary exists; running the eval is an explicit call outside the gate, only as trustworthy as its scorers and dataset.
• boundary_policy ceilings only builtin actions. An agent reaching the platform through an http or mcp tool is outside that boundary; only a guardrail at tool, agent, or project scope reaches those calls.

Deciding what the system should optimize is not a layer SOAT automates.

Status

Pieces marked coming soon are designed but not built.

Piece	State
Agent versions — history, restore, canary split, served-version stamping	Shipped — see Agent Versioning and Canary Rollout
Approvals recurrence view	Shipped
Evaluations — datasets, scorers, runs, baselines	Shipped
Eval-gated promotion of a canary release	Shipped
Memories forgetting — importance scoring, recency blending, compaction	Coming soon
An eval run's verdict read on the pass-rate interval and on the kinds the claim requires, with its reasons	Coming soon
The claim written on the eval and frozen on each run	Coming soon
Claims held per agent version; promotion refused while the canary loses one the stable version supports	Coming soon

Layers are concerns, not autonomy levels

Every agent system has all four layers. Delegation (maturity grades 0–4) is a separate axis: how much runs without a human in the step. Moving up the ladder never adds a layer; it changes which primitives carry each one.

Delegation grade	What runs without a human	SOAT primitives
0 — Deterministic tooling	Everything — no model in the path	pipeline and http tools, tool-only orchestrations, workflow guards, triggers — all JSON Logic, no LLM
1 — Assistant	Nothing — the model reads and proposes, the application executes	Agents with client tools (the run pauses on requires_action), knowledge_config, sessions
2 — Supervised autonomy	Each action a policy clears, per call	Guardrail action classes A/B/C/D, approvals, quotas
3 — End-to-end automation	Whole flows, agents invoking agents	Orchestrations, workflows, nested agent calls, triggers
4 — Self-modification	The system changes its own definitions	A composition, not a shipped mode — see What the ratchet must never do

Grades 0–2 are one mechanism: a guardrail's action classes apply this dial per tool call (class A is grade 0/1, class B grade 2 with a deterministic guard, class C hands the call to a human), and default_class falling back to C means an unclassified call never gains autonomy by accident.

Diagnose before you build

Attribute a failure to a layer before changing anything, in this order: harness defects impersonate the other three.

Symptom	Layer at fault	Where to look in SOAT
The agent cannot operate — wrong tool, stale answers, forgets the user	Harness	Tools and active_tool_ids; knowledge_config and Knowledge; Sessions and Memories
It almost works but is unreliable — right sometimes, unverifiable, or runs away	Loop	output_schema, max_steps, Guardrails, Quotas — read the Trace first
It did something it should not have been able to do	Harness	Policies, the agent's boundary_policy, Audit Log
Each step is fine but the process is unmanageable — no gate, no resume, no record of the path	Graph	Orchestrations or Workflows
It was fine last month — a change made it worse, nobody can say which one, and the same correction keeps being applied by hand	Ratchet	Agent versions and the agent_version on the generation; Evaluations; the approvals recurrence view

Where to start

1. Build the harness. One agent, an AI provider, the few tools it needs, knowledge_config on real project content. Quick Start.
2. Close the loop. output_schema, max_steps, a quota, a guardrail on anything touching the outside world; read the traces.
3. Fit the ratchet. Version the agent, roll significant changes out as a canary, and build a small evaluation dataset from real traffic (a dozen cases is enough to start).
4. Add a graph only when a specific pain unlocks it: a human gate, an audit requirement, a parallel join, or durable resume. Then Choosing an Automation Model.