Skip to main content

SOAT vs OpenAI Agents platform

OpenAI Agents platform is a managed agent platform (MIT (Agents SDK); platform services are proprietary, managed). SOAT is agent infrastructure (Apache-2.0, self-hosted). Of the 15 capabilities the benchmark tracks, OpenAI Agents platform covers 8 natively and SOAT 12.

At a glance​

OpenAI Agents platformSOAT
Typea managed agent platformagent infrastructure
LicenseMIT (Agents SDK); platform services are proprietaryApache-2.0
Deploymentmanagedself-hosted
Native capabilities8 of 1512 of 15
Last verified2026-08-142026-08-14

Where each one leads​

  • OpenAI Agents platform is native where SOAT is not: code execution.
  • SOAT is native where OpenAI Agents platform is not: identity & governance, data & secrets, evaluation, declarative deployment, versioning & rollout.

What OpenAI Agents platform is​

OpenAI's agent stack: the MIT-licensed Agents SDK (agent loop, sessions, handoffs, guardrails, tracing) plus the Responses API, OpenAI-hosted tools (web search, file search, code interpreter, hosted MCP), and a managed Traces dashboard. Execution and hosted tools depend on OpenAI's API; there is no self-hosted control plane. OpenAI Agents platform documentation.

Capability by capability​

Agent runtime​

Agent loop, sessions, and generation lifecycle.

  • OpenAI Agents platform: Native. The SDK ships a built-in agent loop (Runner) plus Sessions as a persistent working-context memory layer across turns. (source)
  • SOAT: Native. Agents, sessions, chats, and generations with full lifecycle tracking. (source)

Orchestration​

Multi-step pipelines, workflows, and triggers.

  • OpenAI Agents platform: Native. Handoffs and agents-as-tools provide multi-agent delegation; Agent Builder adds a visual multi-step workflow canvas. (source)
  • SOAT: Native. DAG orchestrations, workflow state machines, durable tasks, and triggers (manual, webhook, schedule). (source)

Knowledge & memory​

Documents, embeddings, semantic search, and recall.

  • OpenAI Agents platform: Native. FileSearchTool retrieves from OpenAI-hosted vector stores with managed chunking, embeddings, and hybrid semantic plus keyword search. (source)
  • SOAT: Native. Documents with per-chunk embeddings, named memories, unified semantic search, and ingestion rules for PDFs, images, and audio. (source)

Tools & integration​

Tool calling, MCP, providers, and webhooks.

  • OpenAI Agents platform: Native. Function tools with automatic schema generation, hosted tools, and HostedMCPTool for remote MCP servers are built into the SDK. (source)
  • SOAT: Native. HTTP, MCP, client, and agent-to-agent tools; model routes with failover; webhooks; any LLM provider. (source)

Identity & governance​

IAM, guardrails, approvals, quotas, and audit.

  • OpenAI Agents platform: Partial. Input, output, and tool guardrails with tripwires plus human-in-the-loop exist, but there is no tenant IAM, per-project authorization, quota policy, or audit log. (source)
  • SOAT: Native. AWS-inspired IAM, actors, scoped API keys, guardrails, human approvals, quotas, and an append-only audit log. (source)

Observability​

Traces, activity, usage, and failure queues.

  • OpenAI Agents platform: Native. Tracing is on by default and records model generations, tool calls, handoffs, and guardrail evaluations to the OpenAI Traces dashboard. (source)
  • SOAT: Native. Traces of every tool call and model response, activity feed, usage/cost tracking, and an exceptions queue. (source)

Data & secrets​

File storage and encrypted secrets.

  • OpenAI Agents platform: Partial. The Files API and hosted vector stores provide managed file storage, but there is no encrypted secrets manager for agent credentials. (source)
  • SOAT: Native. File storage over local/S3/GCS backends and encrypted secrets storage. (source)

Evaluation​

Datasets, scorers, and eval-gated improvement.

  • OpenAI Agents platform: Partial. The Evals API supports JSONL datasets, graders, and eval runs, but it is scheduled to become read-only and then shut down. (source)
  • SOAT: Native. Datasets, scorers, and scored runs; eval-gated promotion of agent versions. (source)

Channels​

Built-in gateways to messaging platforms and UIs.

  • OpenAI Agents platform: Partial. ChatKit provides an embeddable web chat UI, but no gateways to Slack, WhatsApp, Telegram, or Discord are documented. (source)
  • SOAT: Absent. Surfaces are REST, MCP, CLI, and SDK — no built-in gateways to messaging platforms like Slack, WhatsApp, or Telegram.

Code execution​

Sandboxed code or terminal runtimes for agents.

  • OpenAI Agents platform: Native. CodeInterpreterTool is a hosted tool that executes model-written code in an OpenAI-managed sandbox. (source)
  • SOAT: Absent. Tool types are HTTP, MCP, client, and agent-to-agent — no built-in sandboxed code or terminal runtime.

Human-in-the-loop​

Pausing a run for human approval, then resuming it.

  • OpenAI Agents platform: Native. Tools marked needs_approval pause the run and surface approval items in RunResult.interruptions, resolved with approve/reject and resumed from serialized RunState. (source)
  • SOAT: Native. Approvals are a first-class human-decision queue with frozen evidence, and guardrails classify each tool call so risky actions pause for a decision. (source)

Multi-tenancy​

Projects or workspaces scoping resources and permissions.

  • OpenAI Agents platform: Native. Organizations contain projects that scope keys, files, and resources, with roles assignable at organization or project level. (source)
  • SOAT: Native. Projects are multi-tenant namespaces scoping every resource, with IAM policies and quotas enforced per project. (source)

Declarative deployment​

Templates that provision agent resources as one stack.

  • OpenAI Agents platform: Absent. No template or infrastructure-as-code format provisions agent resources; agents are created imperatively through the API or SDK.
  • SOAT: Native. Formations provision providers, memories, tools, agents, orchestrations, and webhooks from one template, resolving refs into a dependency graph with tracked operations. (source)

Versioning & rollout​

Agent versions, canary rollout, gated promotion.

  • OpenAI Agents platform: Partial. Agent Builder publishes a workflow as a versioned snapshot that can be pinned by ID, but there is no canary rollout or eval-gated promotion. (source)
  • SOAT: Native. Every config change is archived as an append-only agent version, with canary rollout to a slice of traffic and eval-gated promotion. (source)

Skill learning​

Agents authoring reusable skills from their own experience.

  • OpenAI Agents platform: Absent. Sessions persist conversation state, but agents do not author or refine reusable skills from their own experience.
  • SOAT: Absent. Memories and knowledge store retrievable context, but agents do not author or refine reusable skills from their own experience.

How this is rated​

Each rating comes from the public documentation linked beside it: Native is built in, Via plugin needs an add-on, Partial covers part of the capability, Absent is not offered. Every solution side by side is on the benchmark. A rating that is out of date is a bug: open an issue.