Files

File upload, download, metadata management, and deletion over a pluggable storage backend (local filesystem, S3, or GCS).

Overview

Files are associated with a project and persisted through the configured storage backend — local filesystem, S3, or GCS. Every file record exposes a public id; the internal database primary key is never returned. File metadata is tracked in PostgreSQL, while the physical location and backend selection are system-managed and not exposed through the API (see Configuration).

See the Permissions Reference for the IAM action strings for this module.

Related Tutorials

• Debug Session, Generation, and Trace History - Step 6 (Download raw trace steps)
• Orchestrate a Sonnet - Step 8 (Read the persisted poem document)
• Permissions in Practice - Step 7 (Verify permissions with file operations)

Data Model

Field	Type	Description
id	string	Public identifier
prefix	string	Directory within the project (e.g. /assets). Optional on write; defaults to / (root). Read-only on the record (derived from path).
filename	string	Original / download name and the key's leaf segment (e.g. logo.png). Optional on write; defaults to the uploaded file's name.
path	string | null	Read-only. Full key = prefix + / + filename (e.g. /assets/logo.png). Unique per project; the file's identity and the resource ID segment in path-based SRNs.
content_type	string	MIME type
size	number	File size in bytes
metadata	string	Arbitrary JSON string for custom metadata
project_id	string	ID of the owning project
created_at	string	ISO 8601 creation timestamp
updated_at	string	ISO 8601 last-updated timestamp

path is normalized at write time and unique per project; it is the file's identity and the target of path-based policy SRNs. To move a file, change its prefix; to rename it, change its filename — either rebuilds path. Writing to a prefix + filename that resolves to an existing path in the project returns 409 NAME_CONFLICT.

Key Concepts

Storage Backends

The physical location of a file's bytes is handled by a storage provider, selected at runtime with FILES_STORAGE_PROVIDER (default local). The backend is transparent to the API: the same endpoints, records, and download flow work identically regardless of where the bytes live. Each file records which backend stored it, so reads and deletes always route back to the correct provider even if the active backend is later changed.

Provider	FILES_STORAGE_PROVIDER	Where bytes live
Local filesystem	local (default)	A project-scoped directory tree under FILES_STORAGE_DIR
S3 / S3-compatible	s3	Objects in the bucket named by FILES_S3_BUCKET

Both backends use the same logical object layout, {projectPublicId}/{category}/{fileId}{ext}:

Segment	Description
projectPublicId	Public project ID (e.g. proj_ABC) — isolates files by project
category	Derived from the first segment of the file's logical path (e.g., /traces/foo.json → traces/)
fileId	The file's public ID
ext	File extension from the original filename

Every writer builds this key the same way, so a new storage backend inherits the layout rather than defining its own. ext describes the stored bytes: a document's text object is always .txt, whatever the document is named.

If a file has no path, the category defaults to files/. For the local backend this becomes a path under FILES_STORAGE_DIR; for S3 it becomes the object key (optionally namespaced by FILES_S3_KEY_PREFIX):

# local:   {FILES_STORAGE_DIR}/{projectPublicId}/{category}/{fileId}{ext}
/data/files/proj_1a123a/traces/trace_abc123.json
/data/files/proj_1a123a/documents/doc_xyz.md

# s3:      s3://{FILES_S3_BUCKET}/{FILES_S3_KEY_PREFIX}/{projectPublicId}/{category}/{fileId}{ext}
s3://my-bucket/proj_1a123a/traces/trace_abc123.json

Traces persist their raw step payloads as files in the traces/ category; see it end to end in Debug Session, Generation, and Trace History - Step 6 (Download raw trace steps).

Path-Based SRNs

Policies can target files by their logical path rather than their id. When a file has a path set, the server evaluates both the id-based SRN and the path-based SRN:

SRN form	Matches
srn:proj_ABC:file:file_XYZ	Specific file by ID
srn:proj_ABC:file:/assets/logo.png	File at the exact path /assets/logo.png
srn:proj_ABC:file:/exports/*	All files under /exports/
srn:proj_ABC:file:*	All files in the project (id wildcard)

The list endpoint applies policy filters at the SQL level — the database returns only rows the caller is permitted to see. See IAM for full SRN syntax and policy authoring guidance, or walk through scoping a read-only policy to files in Permissions in Practice - Step 7 (Verify permissions with file operations).

Upload Tokens (decoupled uploads)

Upload tokens provide a two-step upload flow — the local-storage equivalent of an S3 presigned URL — usable from any client (SDK, CLI, curl, or an MCP agent):

1. Request a token — POST /api/v1/files/presigned-url returns a single-use upload_token, an upload_url, and an expires_at (15-minute lifetime). This step is authenticated and requires files:UploadFile. By default upload_url is relative (e.g. /api/v1/files/upload/upt_xxx); when the server is configured with SOAT_BASE_URL, it is returned as a fully-qualified absolute URL so clients and MCP agents can POST to it without knowing the server base URL in advance — see Configuration.
2. Upload the content — POST /api/v1/files/upload/{token} writes the file and returns the standard file record. This endpoint requires no bearer credential — the token is the credential — and accepts either multipart/form-data (field file) or JSON with a base64 content field.

Because the two steps are decoupled, the party that authorizes the upload (step 1) need not be the party that transfers the bytes (step 2) — the token can be handed to a browser, a worker, or a CLI to complete the upload directly over HTTP.

The token is invalidated after a single successful upload. Subsequent uploads return 409; expired tokens return 410; unknown tokens return 404.

Downloading from a tool

GET /api/v1/files/{file_id}/download streams the raw bytes and is a REST/SDK/CLI operation only — raw bytes have no JSON form, so it is not offered as an MCP or builtin tool action. Use download-file-base64, which returns the same content as a base64 string in a normal JSON response. Large files are subject to the client's tool-call payload limit, so an agent should fetch the download URL out-of-band with whatever HTTP capability its runtime provides.

Large files via MCP

MCP tool-call payloads larger than ~100 KB are truncated, so upload-file-base64 cannot carry a large file. Use the token flow instead: step 1 (create-presigned-url, exposed as an MCP tool) is always small; perform step 2 out-of-band — via a shell (curl), a fetch/HTTP tool, or a direct SDK call — using multipart/form-data streamed from disk, not the base64 content field:

# Step 1 returned upload_url = /api/v1/files/upload/upt_xxx
curl -F "file=@/path/to/large-report.pdf" "$BASE_URL/api/v1/files/upload/upt_xxx"

Configuration

Environment Variable	Required	Description
FILES_STORAGE_PROVIDER	No	Storage backend: local (default) or s3. Selects where new files are written.
FILES_STORAGE_DIR	For local	Absolute path to the directory where uploaded files are stored. Must be writable by the server process. Required when the provider is local.
FILES_S3_BUCKET	For s3	Name of the S3 bucket that stores file objects. Required when the provider is s3.
FILES_S3_REGION	No	AWS region of the bucket. Falls back to AWS_REGION if unset.
FILES_S3_KEY_PREFIX	No	Key prefix prepended to every object, to namespace files within a shared bucket (e.g. soat/).
FILES_S3_ENDPOINT	No	Custom endpoint URL for S3-compatible stores (e.g. MinIO, Cloudflare R2). Omit for AWS S3.
FILES_S3_FORCE_PATH_STYLE	No	Set to true to use path-style bucket addressing (required by some S3-compatible stores).
FILE_UPLOAD_MAX_BYTES	No	Ceiling on a multipart upload, in bytes. Defaults to 26214400 (25 MB). A larger body is refused with UPLOAD_TOO_LARGE (413) while it is still streaming, so nothing is buffered or stored.
SOAT_BASE_URL	No	Public base URL of the server (e.g. https://api.example.com). When set, the presigned-URL flow returns an absolute upload_url; otherwise the URL is relative. A trailing slash is trimmed.

AWS credentials for the s3 backend are resolved through the standard AWS SDK credential chain (AWS_ACCESS_KEY_ID / AWS_SECRET_ACCESS_KEY, a shared profile, or an instance/task role).

When running the local backend via Docker, mount a volume at FILES_STORAGE_DIR to persist files across container restarts:

services:
  server:
    image: soat-server
    environment:
      FILES_STORAGE_DIR: /data/files
    volumes:
      - files-data:/data/files

volumes:
  files-data:

To use S3 instead, set the provider and bucket (no volume needed):

services:
  server:
    image: soat-server
    environment:
      FILES_STORAGE_PROVIDER: s3
      FILES_S3_BUCKET: my-soat-files
      FILES_S3_REGION: us-east-1

Examples

Upload a file (base64)

CLI

soat upload-file-base64 \
  --project-id proj_ABC \
  --content "iVBORw0KGgo..." \
  --prefix /assets \
  --filename logo.png

SDK

import { SoatClient } from '@soat/sdk';
const soat = new SoatClient({ baseUrl: 'https://api.example.com', token: 'sk_...' });

const { data, error } = await soat.files.uploadFileBase64({
  body: {
    project_id: 'proj_ABC',
    content: 'iVBORw0KGgo...',
    prefix: '/assets',
    filename: 'logo.png',
  },
});
if (error) throw new Error(JSON.stringify(error));

curl

curl -X POST https://api.example.com/api/v1/files/upload-base64 \
  -H "Authorization: Bearer <token>" \
  -H "Content-Type: application/json" \
  -d '{
    "project_id": "proj_ABC",
    "content": "iVBORw0KGgo...",
    "prefix": "/assets",
    "filename": "logo.png"
  }'

Upload a file via an upload token

CLI

# Step 1 — request a single-use token
TOKEN=$(soat create-presigned-url \
  --project-id proj_ABC \
  --content-type application/pdf \
  --prefix /documents \
  --filename report.pdf | jq -r .upload_token)

# Step 2 — upload the content directly (no payload limit)
soat upload-file-with-token \
  --token "$TOKEN" \
  --content "$(base64 -w0 report.pdf)"

SDK

const { data: token } = await soat.files.createPresignedUrl({
  body: {
    project_id: 'proj_ABC',
    content_type: 'application/pdf',
    prefix: '/documents',
    filename: 'report.pdf',
  },
});

const { data, error } = await soat.files.uploadFileWithToken({
  path: { token: token!.upload_token! },
  body: { content: base64Content },
});
if (error) throw new Error(JSON.stringify(error));

curl

# Step 1 — request a token
TOKEN=$(curl -s -X POST https://api.example.com/api/v1/files/presigned-url \
  -H "Authorization: Bearer <token>" \
  -H "Content-Type: application/json" \
  -d '{"project_id":"proj_ABC","prefix":"/documents","filename":"report.pdf"}' | jq -r .upload_token)

# Step 2 — upload the file (token is the credential, no Authorization header)
curl -X POST "https://api.example.com/api/v1/files/upload/$TOKEN" \
  -F "file=@report.pdf"

List files in a project

CLI

soat list-files --project-id proj_ABC

SDK

const { data, error } = await soat.files.listFiles({
  query: { project_id: 'proj_ABC' },
});
if (error) throw new Error(JSON.stringify(error));

curl

curl https://api.example.com/api/v1/files?project_id=proj_ABC \
  -H "Authorization: Bearer <token>"