Chat with an LLM

This tutorial walks through the full flow of having a back-and-forth conversation with an LLM. You will:

1. Log in as admin.
2. Create a project.
3. Create a local AI provider backed by Ollama.
4. Create an agent.
5. Open a session.
6. Send messages and receive replies from the model.
7. View the conversation history.
8. Run async generation.
9. Capture generation lifecycle events via webhook.

By the end you will understand how AI Providers, Agents, Sessions, and Webhooks compose together to drive both sync and async LLM conversations.

Prerequisites

• SOAT running locally. Follow the Quick Start guide to bring the stack up with Docker Compose.
• New to SOAT? Read Key Concepts to understand projects, agents, and sessions before diving in.
• CLI installed and configured, or SDK set up. See CLI or SDK.
• For production hardening (secrets, env vars), see Advanced Configuration.
• Server is at http://localhost:5047.
• Ollama running locally with a chat model available.
• This repo's tutorial test stack already provisions Ollama with qwen2.5:0.5b, so this tutorial runs in automated tests without external credentials.

CLI

export SOAT_BASE_URL=http://localhost:5047

CLI path flags in this tutorial are resource-specific and kebab-cased, for example --agent-id, --session-id, and --webhook-id.

SDK

All code snippets below use a SoatClient instance. The authenticated instance is created in Step 1 after login.

import { SoatClient } from '@soat/sdk';

curl

export SOAT_BASE_URL=http://localhost:5047

---

Step 1 — Log in as admin

Admin is the built-in superuser role. It bypasses policy evaluation entirely. See Users for full authentication and user management details.

CLI

ADMIN_TOKEN=$(soat login-user --username admin --password Admin1234! | jq -r '.token')
export SOAT_TOKEN=$ADMIN_TOKEN

SDK

const soat = new SoatClient({ baseUrl: 'http://localhost:5047' });

const { data: session, error } = await soat.users.loginUser({
  body: { username: 'admin', password: 'Admin1234!' },
});

if (error) throw new Error(JSON.stringify(error));

// Rebuild with the admin token
const adminSoat = new SoatClient({
  baseUrl: 'http://localhost:5047',
  token: session.token,
});

curl

ADMIN_TOKEN=$(curl -s -X POST "$SOAT_BASE_URL/api/v1/users/login" \
  -H "Content-Type: application/json" \
  -d '{"username":"admin","password":"Admin1234!"}' | jq -r '.token')

---

Step 2 — Create a project

Every resource in SOAT lives inside a project. Create one to hold the agent and its supporting configuration.

CLI

PROJECT_ID=$(soat create-project --name "LLM Chat Demo" | jq -r '.id')
echo "PROJECT_ID: $PROJECT_ID"
# PROJECT_ID: proj_vh9qHLINTdsrAqwK

SDK

const { data: project, error } = await adminSoat.projects.createProject({
  body: { name: 'LLM Chat Demo' },
});

if (error) throw new Error(JSON.stringify(error));

const PROJECT_ID = project.id; // proj_…

curl

PROJECT_ID=$(curl -s -X POST "$SOAT_BASE_URL/api/v1/projects" \
  -H "Authorization: Bearer $ADMIN_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"name":"LLM Chat Demo"}' | jq -r '.id')
echo "PROJECT_ID: $PROJECT_ID"

---

Step 3 — Create a local AI provider

For local development and tutorial tests, the simplest setup is an AI provider backed by Ollama. It uses the server's OLLAMA_BASE_URL, so no secret is required.

Step 3 — Create a local AI provider

For local development and tutorial tests, the simplest setup is an AI provider backed by Ollama. It uses the server's OLLAMA_BASE_URL, so no secret is required. This tutorial uses a local Ollama provider so it can run without external credentials. To connect xAI, OpenAI, Anthropic, or Amazon Bedrock instead, see Connect Third-Party LLMs.

CLI

AI_PROVIDER_ID=$(soat create-ai-provider \
  --project-id "$PROJECT_ID" \
  --name "Local Ollama" \
  --provider "ollama" \
  --default-model "qwen2.5:0.5b" | jq -r '.id')
echo "AI_PROVIDER_ID: $AI_PROVIDER_ID"
# AI_PROVIDER_ID: aip_8BTcGUvXnehCCQKs

SDK

const { data: aiProvider, error } =
  await adminSoat.aiProviders.createAiProvider({
    body: {
      project_id: PROJECT_ID,
      name: 'Local Ollama',
      provider: 'ollama',
      default_model: 'qwen2.5:0.5b',
    },
  });

if (error) throw new Error(JSON.stringify(error));

const AI_PROVIDER_ID = aiProvider.id; // aip_…

curl

AI_PROVIDER_ID=$(curl -s -X POST "$SOAT_BASE_URL/api/v1/ai-providers" \
  -H "Authorization: Bearer $ADMIN_TOKEN" \
  -H "Content-Type: application/json" \
  -d "{\"project_id\":\"$PROJECT_ID\",\"name\":\"Local Ollama\",\"provider\":\"ollama\",\"default_model\":\"qwen2.5:0.5b\"}" \
  | jq -r '.id')
echo "AI_PROVIDER_ID: $AI_PROVIDER_ID"

---

Step 4 — Create an agent

An agent is bound to an AI provider and carries a system prompt (instructions). It is the entity that generates responses.

CLI

AGENT_ID=$(soat create-agent \
  --project-id "$PROJECT_ID" \
  --ai-provider-id "$AI_PROVIDER_ID" \
  --name "Local Assistant" \
  --instructions "You are a concise assistant running on a local Ollama model. Keep answers short (max 20 words), clear, and practical." \
  | jq -r '.id')
echo "AGENT_ID: $AGENT_ID"
# AGENT_ID: agt_KO5nAMmsSOVBWLlN

SDK

const { data: agent, error } = await adminSoat.agents.createAgent({
  body: {
    project_id: PROJECT_ID,
    ai_provider_id: AI_PROVIDER_ID,
    name: 'Local Assistant',
    instructions:
      'You are a concise assistant running on a local Ollama model. Keep answers short (max 20 words), clear, and practical.',
  },
});

if (error) throw new Error(JSON.stringify(error));

const AGENT_ID = agent.id; // agt_…

curl

AGENT_ID=$(curl -s -X POST "$SOAT_BASE_URL/api/v1/agents" \
  -H "Authorization: Bearer $ADMIN_TOKEN" \
  -H "Content-Type: application/json" \
  -d "{\"project_id\":\"$PROJECT_ID\",\"ai_provider_id\":\"$AI_PROVIDER_ID\",\"name\":\"Local Assistant\",\"instructions\":\"You are a concise assistant running on a local Ollama model. Keep answers short (max 20 words), clear, and practical.\"}" \
  | jq -r '.id')
echo "AGENT_ID: $AGENT_ID"

---

Step 5 — Create a session

A session is a single conversation thread tied to an agent. Setting auto_generate to true means the agent generates a reply automatically every time you send a user message.

CLI

SESSION_ID=$(soat create-agent-session \
  --agent-id "$AGENT_ID" \
  --name "My first chat" \
  --auto-generate true | jq -r '.id')
echo "SESSION_ID: $SESSION_ID"
# SESSION_ID: sess_N0oEzsx3ayvgKwy3

SDK

const { data: session2, error } = await adminSoat.sessions.createAgentSession({
  path: { agent_id: AGENT_ID },
  body: { name: 'My first chat', auto_generate: true },
});

if (error) throw new Error(JSON.stringify(error));

const SESSION_ID = session2.id; // sess_…

curl

SESSION_ID=$(curl -s -X POST "$SOAT_BASE_URL/api/v1/agents/$AGENT_ID/sessions" \
  -H "Authorization: Bearer $ADMIN_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"name":"My first chat","auto_generate":true}' | jq -r '.id')
echo "SESSION_ID: $SESSION_ID"

---

Step 6 — Send messages and receive replies

Because auto_generate is enabled, every call to add-session-message triggers generation immediately and returns the assistant reply inline. The conversation context is maintained across calls — the model sees all previous messages. See Sessions for the full message and generation API.

7a — First message

CLI

soat add-session-message \
  --agent-id "$AGENT_ID" \
  --session-id "$SESSION_ID" \
  --message "What is the capital of France?"

Example output:

{
  "status": "completed",
  "message": {
    "role": "assistant",
    "content": "The capital of France is Paris.",
    "model": "qwen2.5:0.5b"
  },
  "generation_id": "agt_gen_mznGfHSV4YAGiBXy",
  "trace_id": "agt_trace_8rcvif0n29WE37NL"
}

SDK

const { data: reply1, error: err1 } =
  await adminSoat.sessions.addSessionMessage({
    path: { agent_id: AGENT_ID, session_id: SESSION_ID },
    body: { message: 'What is the capital of France?' },
  });

if (err1) throw new Error(JSON.stringify(err1));

console.log(reply1.message?.content);
// "The capital of France is Paris."

curl

curl -s -X POST "$SOAT_BASE_URL/api/v1/agents/$AGENT_ID/sessions/$SESSION_ID/messages" \
  -H "Authorization: Bearer $ADMIN_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"message":"What is the capital of France?"}'

7b — Queue a follow-up message for async generation

Now disable auto_generate and add a follow-up user message. We will generate the assistant reply in Step 10 using async mode.

CLI

soat update-session \
  --agent-id "$AGENT_ID" \
  --session-id "$SESSION_ID" \
  --auto-generate false

soat add-session-message \
  --agent-id "$AGENT_ID" \
  --session-id "$SESSION_ID" \
  --message "In one short sentence, what is the population of Paris?"

SDK

const { data: reply2, error: err2 } =
  await adminSoat.sessions.addSessionMessage({
    path: { agent_id: AGENT_ID, session_id: SESSION_ID },
    body: {
      message: 'In one short sentence, what is the population of Paris?',
    },
  });

if (err2) throw new Error(JSON.stringify(err2));
console.log(reply2.status);
// "pending"

curl

curl -s -X PATCH "$SOAT_BASE_URL/api/v1/agents/$AGENT_ID/sessions/$SESSION_ID" \
  -H "Authorization: Bearer $ADMIN_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"auto_generate":false}'

curl -s -X POST "$SOAT_BASE_URL/api/v1/agents/$AGENT_ID/sessions/$SESSION_ID/messages" \
  -H "Authorization: Bearer $ADMIN_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"message":"In one short sentence, what is the population of Paris?"}'

---

Step 7 — View the conversation history

Fetch all messages in the session to review the full exchange. Messages are persisted on the underlying Conversation model; the session provides a scoped view into it.

CLI

soat list-agent-session-messages \
  --agent-id "$AGENT_ID" \
  --session-id "$SESSION_ID" | jq '.data[] | {role, content}'

Example output:

{ "role": "user",      "content": "What is the capital of France?" }
{ "role": "assistant", "content": "The capital of France is Paris." }
{ "role": "user",      "content": "What is the population of that city?" }
{ "role": "assistant", "content": "The population of Paris … approximately 2.1 million …" }

SDK

const { data: messages, error } =
  await adminSoat.sessions.listAgentSessionMessages({
    path: { agent_id: AGENT_ID, session_id: SESSION_ID },
  });

if (error) throw new Error(JSON.stringify(error));

for (const msg of messages.data ?? []) {
  console.log(`[${msg.role}] ${msg.content}`);
}

curl

curl -s "$SOAT_BASE_URL/api/v1/agents/$AGENT_ID/sessions/$SESSION_ID/messages" \
  -H "Authorization: Bearer $ADMIN_TOKEN" | jq '.data[] | {role, content}'

---

Step 8 - Start a local webhook listener

Start the CLI listener before creating the webhook. It opens a local HTTP endpoint and prints each matching delivery. In the automated tutorial tests, SOAT_WEBHOOK_BASE_URL is injected so the server container can reach this listener. See CLI Commands for all soat listen flags and Webhooks for the full delivery and signing model.

CLI

WEBHOOK_BASE_URL=${SOAT_WEBHOOK_BASE_URL:-http://localhost:8787}
soat listen --port 8787 --path /webhook --filter sessions.generation.* --json > session-webhooks.log 2>&1 &
LISTENER_PID=$!
sleep 2

Optional: pass --secret <webhook-secret> to validate X-Soat-Signature.

SDK

Start a local HTTP server to receive webhook deliveries. In the automated tutorial tests, SOAT_WEBHOOK_BASE_URL is injected so the server container can reach this listener.

curl

WEBHOOK_BASE_URL=${SOAT_WEBHOOK_BASE_URL:-http://localhost:8787}
soat listen --port 8787 --path /webhook --filter sessions.generation.* --json > session-webhooks.log 2>&1 &
LISTENER_PID=$!
sleep 2

---

Step 9 - Create a session webhook subscription

Subscribe to session generation events so you can observe the async lifecycle. See Webhooks for the full list of event types, retry rules, and HMAC signing.

CLI

WEBHOOK_ID=$(soat create-webhook \
  --project-id "$PROJECT_ID" \
  --name "session-events" \
  --url "$WEBHOOK_BASE_URL/webhook" \
  --events '["sessions.generation.*"]' | jq -r '.id')
echo "WEBHOOK_ID: $WEBHOOK_ID"

SDK

const WEBHOOK_BASE_URL =
  process.env.SOAT_WEBHOOK_BASE_URL ?? 'http://localhost:8787';

const { data: webhook, error: webhookErr } =
  await adminSoat.webhooks.createWebhook({
    path: { project_id: PROJECT_ID },
    body: {
      name: 'session-events',
      url: `${WEBHOOK_BASE_URL}/webhook`,
      events: ['sessions.generation.*'],
    },
  });

if (webhookErr) throw new Error(JSON.stringify(webhookErr));

const WEBHOOK_ID = webhook.id; // whk_...

curl

WEBHOOK_ID=$(curl -s -X POST "$SOAT_BASE_URL/api/v1/projects/$PROJECT_ID/webhooks" \
  -H "Authorization: Bearer $ADMIN_TOKEN" \
  -H "Content-Type: application/json" \
  -d "{\"name\":\"session-events\",\"url\":\"$WEBHOOK_BASE_URL/webhook\",\"events\":[\"sessions.generation.*\"]}" \
  | jq -r '.id')
echo "WEBHOOK_ID: $WEBHOOK_ID"

---

Step 10 - Trigger async generation

Disable auto_generate, add a user message, then trigger generation with async=true. See Sessions — Async Generation for status codes and how to poll for completion.

CLI

soat update-session \
  --agent-id "$AGENT_ID" \
  --session-id "$SESSION_ID" \
  --auto-generate false

soat add-session-message \
  --agent-id "$AGENT_ID" \
  --session-id "$SESSION_ID" \
  --message "Give me 1 concise fact about Sao Paulo."

soat generate-session-response \
  --agent-id "$AGENT_ID" \
  --session-id "$SESSION_ID" \
  --async true

Expected immediate response (accepted):

{
  "status": "accepted",
  "session_id": "sess_..."
}

SDK

const { error: updateErr } = await adminSoat.sessions.updateSession({
  path: { agent_id: AGENT_ID, session_id: SESSION_ID },
  body: { auto_generate: false },
});

if (updateErr) throw new Error(JSON.stringify(updateErr));

const { error: addErr } = await adminSoat.sessions.addSessionMessage({
  path: { agent_id: AGENT_ID, session_id: SESSION_ID },
  body: { message: 'Give me 1 concise fact about Sao Paulo.' },
});

if (addErr) throw new Error(JSON.stringify(addErr));

const { data: accepted, error: generateErr } =
  await adminSoat.sessions.generateSessionResponse({
    path: { agent_id: AGENT_ID, session_id: SESSION_ID },
    query: { async: true },
  });

if (generateErr) throw new Error(JSON.stringify(generateErr));

console.log(accepted.status); // "accepted"

curl

curl -s -X PATCH "$SOAT_BASE_URL/api/v1/agents/$AGENT_ID/sessions/$SESSION_ID" \
  -H "Authorization: Bearer $ADMIN_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"auto_generate":false}'

curl -s -X POST "$SOAT_BASE_URL/api/v1/agents/$AGENT_ID/sessions/$SESSION_ID/messages" \
  -H "Authorization: Bearer $ADMIN_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"message":"Give me 1 concise fact about Sao Paulo."}'

curl -s -X POST "$SOAT_BASE_URL/api/v1/agents/$AGENT_ID/sessions/$SESSION_ID/generate?async=true" \
  -H "Authorization: Bearer $ADMIN_TOKEN" \
  -H "Content-Type: application/json"

When generation runs, your soat listen terminal should log events such as:

• sessions.generation.started
• sessions.generation.completed

---

Step 11 - Verify delivery and final assistant message

Wait for the async delivery, inspect the webhook listener output, then fetch session messages again. Delivery records are queryable via the Webhooks module.

CLI

for _ in $(seq 1 20); do soat list-webhook-deliveries --project-id "$PROJECT_ID" --webhook-id "$WEBHOOK_ID" | jq -e '[.data[] | select(.status == "completed")] | length > 0' && break || sleep 1; done # → ignore

soat list-webhook-deliveries \
  --project-id "$PROJECT_ID" \
  --webhook-id "$WEBHOOK_ID" | jq '.data[] | {event_type, status, status_code}'

cat session-webhooks.log

soat list-agent-session-messages \
  --agent-id "$AGENT_ID" \
  --session-id "$SESSION_ID" | jq '.data[] | {role, content}'

kill "$LISTENER_PID"
wait "$LISTENER_PID" 2>/dev/null || true

SDK

const { data: deliveries, error: deliveriesErr } =
  await adminSoat.webhooks.listWebhookDeliveries({
    path: { project_id: PROJECT_ID, webhook_id: WEBHOOK_ID },
  });

if (deliveriesErr) throw new Error(JSON.stringify(deliveriesErr));

for (const d of deliveries.data ?? []) {
  console.log(d.event_type, d.status, d.status_code);
}

const { data: messages2, error: messagesErr } =
  await adminSoat.sessions.listAgentSessionMessages({
    path: { agent_id: AGENT_ID, session_id: SESSION_ID },
  });

if (messagesErr) throw new Error(JSON.stringify(messagesErr));

for (const msg of messages2.data ?? []) {
  console.log(`[${msg.role}] ${msg.content}`);
}

curl

curl -s "$SOAT_BASE_URL/api/v1/projects/$PROJECT_ID/webhooks/$WEBHOOK_ID/deliveries" \
  -H "Authorization: Bearer $ADMIN_TOKEN" \
  | jq '.data[] | {event_type, status, status_code}'

curl -s "$SOAT_BASE_URL/api/v1/agents/$AGENT_ID/sessions/$SESSION_ID/messages" \
  -H "Authorization: Bearer $ADMIN_TOKEN" \
  | jq '.data[] | {role, content}'

---

What's next

• Manual generation: Create a session without auto_generate and call generate-session-response (soat generate-session-response --agent-id … --session-id …) explicitly for full control over when the model responds.
• Session tags: Use replace-session-tags / merge-session-tags to attach metadata (e.g. user ID, conversation topic) to a session for filtering.
• Agents with tools: Attach SOAT tools or HTTP tools to the agent so the model can take actions. See the Agents module.