--- title: System Architecture canonical: "https://agentbarn.dev/guides/develop/system-architecture" pubDate: "2026-08-29T00:00:00.000Z" updatedDate: "2026-09-13T13:41:42.000Z" author: Agent Barn description: "Map Product API, Ingest, Communications, Web Chat, runtime execution, persisted costs, and Organization and Agent authorization boundaries." tags: [Develop and extend, Concept, New contributors and solution architects, system architecture, Web Chat, Agent Template pins, cost records, Ingest, Communications, authorization] categories: [Guides, Develop and extend] --- Architecture outcome ## What you will understand By the end of this guide, you will understand: - The four major operational areas of the monorepo - How the UI, Product API, Ingest API, Communications, Agent runtimes, and infrastructure communicate - Why the Agent domain orchestrates execution while Communications owns delivery - How Organization and Agent authorization differ - Where persistence and transaction boundaries live - Why communication, telemetry, Domain Events, Event Deliveries, audit records, and costs are separate - How versioned configuration becomes a running Kubernetes workload - Where a new feature or system change belongs ## Overview Understand where Agent Barn responsibilities live, how data crosses system boundaries, and which contracts must move together when you make a change. Agent Barn manages Organization-owned Agents that run through Hermes or OpenClaw. An Agent may be headless, or it may own zero or many Communication Connections to Slack, Microsoft Teams, Telegram, or Discord, which are delivered through the Communications Gateway and its Platform Plugins. At a high level: Agent Barn system topology PeopleBrowserNext.js web app **Product API `:8000`** - PostgreSQL - Kubernetes API → Agent Deployments - LiteLLM → OpenRouter - Firecrawl - Email and OAuth providers - Redis → Event Delivery worker **Slack / Teams / Telegram / Discord****Platform Plugin****Communications `:8002`****Hermes or OpenClaw Agent runtime** **Hermes or OpenClaw Agent runtime**Tool Call telemetry + Ingest key → Ingest API `:8001` → PostgreSQL The system separates: - Human product requests - Communication delivery - Runtime Tool Call telemetry - Internal business events - Background delivery - Provider cost reporting - Dynamic Agent execution - Platform deployment These paths interact, but they do not share one event model, authentication mechanism, or source of truth. For the non-implementation summary of the same boundaries, see [How Agent Barn fits together](/guides/system-overview). **Important** Agent Barn is not event-sourced. Current product state lives in ordinary domain tables. Domain Events record selected business facts and drive internal handlers; they do not reconstruct every entity. ## Four operational areas Agent Barn is a monorepo with four major operational areas. ### API Three composed applications: product contracts, Runtime Tool Call telemetry, and Communications ingress, delivery, and Runtime protocol Primary source: `api/` ### Web app Authenticated Organization and Platform experiences Primary source: `ui/` ### Agent runtimes Execute rendered Agent configuration, claim Communication Deliveries, and call tools Primary source: `hermes-base/, openclaw-base/, Agent builders` ### Deployment Build and deploy databases, services, runtime images, monitoring, and Agent infrastructure Primary source: `helm/, helmfile.yaml.gotmpl, .github/workflows/` These are operational boundaries, not four independently owned domain services. API domains are modules inside one codebase, they share the Agent Barn application database, and they are composed into three separate HTTP applications. ### API The API provides: - Product HTTP routes - Runtime Tool Call telemetry ingestion - Provider ingress and communication delivery - Authentication and token management - Organization and Agent authorization - Database persistence - Template and Skill management - Agent lifecycle orchestration - Kubernetes resource creation - LiteLLM key management - Tool Integration credentials - Domain Event delivery workers - Monitoring metrics ### Web app The web app provides: - Public authentication routes - Organization View - Platform View - Agent configuration and lifecycle controls - Communication Connection configuration - Template and Skill management - Activity, Tool Call, cost, and log views - Platform administration - Permission-aware actions ### Agent runtimes Hermes and OpenClaw: - Load generated Agent configuration - Claim durable Communication Deliveries through the runtime-neutral protocol - Call models through LiteLLM - Use mounted Skills and tool Integrations - Maintain Agent workspace state - Push Tool Call telemetry to Ingest - Expose health and metrics ### Deployment The deployment system owns: - PostgreSQL, Redis, LiteLLM, Firecrawl, Product API, Ingest, Communications, UI, and monitoring releases - Runtime image builds - Alembic migration hooks - LiteLLM virtual-key hooks - Kubernetes Secrets and ingress - Production and staging namespaces - Image and chart version inputs ## System topology ### Product request path A normal browser request follows: ``` Browser ↓ Next.js route or feature component ↓ Shared UI API client ↓ Product API /api/v1 ↓ Route ↓ Service ↓ Repository ↓ Agent Barn PostgreSQL ``` Services can branch to infrastructure adapters when the workflow needs Kubernetes, LiteLLM, email, OAuth, encryption, or another provider. ### Runtime execution path A running Agent works from a claimed delivery: ``` Claimed Communication Delivery ↓ Hermes or OpenClaw ├── generated Template and policy ├── mounted Skills ├── decrypted Agent Secrets for tool Integrations ├── Agent workspace PVC └── model request → LiteLLM → OpenRouter ``` The Agent runtime is a dynamically created Kubernetes workload, not code executing inside the Product API process. ### Inbound communication flow ``` Platform provider ↓ Platform Plugin admission and normalization ↓ Communications Gateway ↓ Durable inbound Communication Delivery ├── canonical Conversation Message └── Runtime protocol claim ↓ Hermes or OpenClaw ``` Provider message identity is idempotent within a Connection, and conversation location identity is `(connection_id, channel_id)`. ### Outbound communication flow ``` Hermes or OpenClaw ↓ Reply submitted against source Delivery ↓ Durable outbound Communication Delivery ↓ Source Communication Connection ↓ Platform Plugin ↓ Platform provider ``` The reply returns through the Connection and Platform Plugin that produced the source delivery. ### Delivery guarantees - Delivery processing is durable and at least once. - Outbound ordering is preserved per conversation. - Provider retries reuse stable delivery identity. - Replies remain bound to the source Connection. - Connection health is separate from Agent lifecycle. See [Communication Connections](/guides/agents/communication-connections) for the Connection lifecycle. ### Internal event path ``` Business mutation ↓ one PostgreSQL transaction Business state + Outbox Message + Event Deliveries ↓ post-commit enqueue Redis / Dramatiq ↓ Worker ↓ Event Handler ├── Security Audit projection └── Agent lifecycle email ``` If immediate enqueue fails, committed delivery state remains in PostgreSQL and reconciliation can republish it later. ## Domain model Organization is the ownership and tenancy root for user-facing resources. **Organization** - Memberships → User and authentication - Organization Agent Settings - Templates and Template Versions - Organization Skills and Skill Versions - Shared Credentials - Domain Events - Outbox Messages - Event Deliveries - **Agent** - Runtime and lifecycle - pinned Template or Override Version - assigned Skill Version pins - Agent-private Skills - Agent Secrets and Integrations - Communication Connections - Platform Plugin - Communication Deliveries - Connection Journal - Runtime Kubernetes resources - Conversation Messages - Tool Calls - LiteLLM cost identity Communication Connections are Agent subordinate resources for authorization, but they are not fields within the Agent aggregate. An Agent may have zero or many Connections, including several Connections for the same Platform. ### Platform-owned resources Some resources belong to Agent Barn itself rather than an Organization: - Platform Privileges - Predefined Platform Templates - Platform Skills - The Platform Plugin registry - Platform-level Domain Events - Platform oversight views - Platform configuration and catalogue data A new installation has no default Organization. Application startup ensures: - The bootstrap Platform Administrator - RBAC seed data - Bundled Platform Skills - The predefined Platform Template catalogue Platform Templates live in their own global table. Platform Skills use global ownership rather than being copied into every Organization. ## Tenancy and authorization ### Organization is the tenancy axis Organization-owned routes normally include: ``` /api/v1/organizations/{organization_id}/... ``` Authentication produces a current user context. Organization-scoped services resolve a real persisted Membership for the requested Organization. Platform Administrators do not receive implicit Organization Membership, an implicit Organization Role, or Agent access through their platform authority. ### Authentication seams Several distinct trust boundaries write to or read from the system: | Seam | Required identity and scope | | --- | --- | | Product API user request | Authenticated user context; persisted Organization Membership for Organization routes; Organization Permissions; Agent Access for Agent and subordinate resources | | Platform API request | Platform Administrator authority; no implicit active Organization | | Ingest write | Agent identity; per-start Ingest key; Tool Call telemetry only | | Runtime Communications protocol | Separate per-start protocol identity; claims and completes durable Communication Deliveries | | Provider webhook | Platform Plugin verification; Connection-scoped provider identity | | Driver callback | Separate non-user trust boundary | ### Organization View and Platform View | View | Route shape | Authority | | --- | --- | --- | | Organization View | `/dashboard/[orgId]` | Membership and Agent Access | | Platform View | `/dashboard/platform` | Platform Privilege | Platform View has no Active Organization. Platform oversight is explicitly allowlisted. It does not mean unrestricted access to Organization-owned credentials, configuration payloads, conversations, or raw telemetry. ### Two role families Organization Roles govern Organization-level capabilities: - Organization Owner - Organization Admin - Organization Member Agent Access Roles govern one Agent aggregate: - Agent Viewer - Agent Editor - Agent Owner - Organization-defined custom Agent Access Roles Organization Owner and Admin receive implicit Agent Owner authority over Agents in their Organization. Members can receive Agent authority through: - Explicit Agent Access - Agent General Access - Both, with additive Permissions Creator fields are provenance. They are not permanent authorization exceptions. ### Authorization placement The architecture divides authorization responsibility: - Repositories constrain visibility before count, pagination, and return. - Services enforce action Permissions and lifecycle rules. - Routes authenticate, parse, delegate, and return. - The UI uses server-reported permitted actions to render controls. - Every backend mutation independently reauthorizes the request. Subordinate resources (Communication Connections, conversations, Tool Calls, activity, costs, logs, Secrets, Skills, and configuration) must be accessed through the same accessible-Agent boundary. **Security** Fetching every Agent and filtering unauthorized rows in a service is not a safe implementation. Visibility must be applied in the repository query before totals, ordering, or pagination. ## API architecture The API has three separately composed FastAPI applications. | Application | Composition root | Mounted path | Responsibility | | --- | --- | --- | --- | | Product API | `api/api_app.py` | `/api/v1` | User-facing product and administration operations | | Ingest API | `api/ingest_app.py` | `/ingest/v1` | Authenticated Runtime Tool Call telemetry | | Communications | `api/communications_app.py` | `/communications/v1` | Provider ingress, durable communication delivery, supervised provider sessions, and Runtime protocol | Product runs on port `8000`, Ingest on `8001`, and Communications on `8002`. They use separate Prometheus registries and expose separate `/metrics` endpoints. Provider webhooks belong to the Communications application and are Connection-scoped. There is no provider-specific webhook route on the Product API. For route registration and request contracts, see [API architecture and request boundaries](/guides/api-architecture) and [Develop against the API](/guides/develop/api). ### Layering The normal dependency direction is: RoutesServicesRepositoriesPostgreSQL ``` routes.py ↓ service.py ↓ repository.py ↓ PostgresRepositoryDelegate ``` Services may also call: ``` Kubernetes LiteLLM OpenRouter Platform Plugin provider clients Google OAuth Cloudflare email Cryptography Other provider adapters ``` ### Routes Routes should: - Authenticate - Parse path, query, and body data - Resolve dependencies - Delegate to a service - Return the response Routes should not own business workflows, SQL, or database transactions. ### Services Services own: - Business rules - Permission-sensitive behavior - Lifecycle validation - Error translation - Cross-domain orchestration - Calls to infrastructure adapters - Post-commit Event Delivery enqueue The Agent Service is intentionally broader than a CRUD service because starting an Agent crosses Templates, Skills, credentials, LiteLLM, Kubernetes, Ingest, and the Communications protocol. ### Repositories Repositories own: - SQLModel and SQLAlchemy queries - Tenant and visibility predicates - Persistence - Ordering and pagination - Explicit transaction boundaries where required - Atomic business mutation plus event staging ### Infrastructure adapters Infrastructure code isolates external concerns such as: - PostgreSQL - Kubernetes - Redis transport - LiteLLM - OpenRouter - Email - Platform Plugin provider clients - OAuth - Encryption - Time Dependency injection is assembled through the API’s Injector modules. ## Persistence and transactions ### Application database Agent Barn’s PostgreSQL database stores: - Users and authentication state - Organizations and Memberships - Organization Agent Settings - Roles and Agent Access - Agents and lifecycle state - Templates and versions - Skills and Skill Versions - Encrypted credentials - Communication Connections and Communication Deliveries - Conversation Messages - Tool Calls - Connection Journal entries - Domain Events, Outbox Messages, and Event Deliveries - Security Audit Records - Persisted model-call cost records Schema changes use Alembic migrations under: ``` api/migrations/versions/ ``` Integration tests migrate a real PostgreSQL test database. ### Separate databases The deployment includes distinct databases: | Database | Owner | | --- | --- | | Agent Barn application PostgreSQL | Agent Barn API and Alembic | | LiteLLM PostgreSQL | LiteLLM | | Firecrawl PostgreSQL | Firecrawl | Agent Barn Alembic migrations do not manage the LiteLLM or Firecrawl schemas. ### Ordinary repository operations Most repositories use a shared delegate that opens and commits a session per operation. Therefore: ``` service call ├── repository operation A → commit └── repository operation B → commit ``` is not automatically one transaction. If operation B fails, operation A may already be committed. ### Explicit transactions A workflow requiring all-or-nothing persistence needs a domain-specific repository transaction: ``` one SQLModel session ├── business mutation ├── Outbox Message ├── intended Event Deliveries └── one commit ``` The outbox stages rows inside the repository-owned session. It does not open or commit its own session. **Important** Do not add optional transaction or Domain Event behavior to every generic repository method. Create an explicit domain transaction for the workflow that requires atomicity. ## Event and data flows Several similarly named records have deliberately different roles. | Concept | Origin | Authentication | Persistence | Purpose | | --- | --- | --- | --- | --- | | Product request | Browser or API consumer | Human token and scoped authority | Domain tables | Read or mutate product state | | Conversation Message | Platform provider through the Communications Gateway | Connection-scoped provider identity | Conversation Message tables | Record canonical inbound and outbound communication | | Tool Call telemetry | Hermes or OpenClaw | Agent ID and per-start Ingest key | Tool Call tables | Report Runtime tool execution | | Communication Delivery | Communications Gateway | Internal Communications boundary | Durable PostgreSQL delivery rows | Track one inbound or outbound delivery attempt chain | | Domain Event | Business mutation | Internal application boundary | Immutable event envelope in PostgreSQL | Record a typed business fact | | Outbox Message | Domain Event transaction | Internal | Immutable PostgreSQL row | Record durable publication intent | | Event Delivery | One intended handler | Internal worker boundary | Mutable PostgreSQL lifecycle row | Track handler-specific delivery | | Security Audit Record | Selected Domain Event handler | Internal | Immutable PostgreSQL projection | Preserve compliance evidence | | Cost report | LiteLLM spend logs and OpenRouter recovery | Authorized report access | cost\_record | Synchronize and attribute stored model-call costs to Agents and Organizations | ### Activity has two writers The Communications Gateway writes canonical inbound and outbound Conversation Messages. Ingest writes Tool Call telemetry. Ingest does not write Conversation Messages. ``` Platform Provider ↓ Communications Gateway ↓ Conversation Message repository ↓ Product read API ↓ Activity UI Agent Runtime ↓ Ingest API ↓ Tool Call repository ↓ Product read API ↓ Activity UI ``` - Product API provides authorized read routes for both. - Conversation read paths preserve Connection identity. - Tool Call correlation uses Runtime invocation identity. - Neither Activity path writes Domain Events. - Costs are not derived from Conversation or Tool Call rows. Tool Call telemetry is authenticated with a per-start Ingest key rather than a human Membership, and it is not copied into the Domain Event outbox. See [Activity, conversations, and runtime telemetry](/guides/activity-conversations-and-telemetry). ### Domain Events Domain Events are immutable typed business facts. They include: - Event ID - Event name and schema version - Event Scope - Optional Organization ID - Actor and Subject identities - Correlation and optional causation identity - Bounded, secret-safe payload Domain Event payloads must reject credentials, secrets, unsupported values, sensitive key names, and unbounded content. Current events include Organization Agent Settings changes and Communications operational facts; this guide does not reproduce the complete event catalogue. ### Event delivery The business mutation, its Outbox Message, and its intended Event Deliveries commit in one domain-owned transaction. PostgreSQL is authoritative for: - The event - Publication intent - Intended handlers - Delivery status - Attempts - Current bounded error - Dead-letter reason Redis and Dramatiq provide low-latency transport, and reconciliation republishes eligible Event Deliveries. The delivery guarantee is at least once. A worker can fail after a handler commits its side effect but before the delivery becomes `SUCCEEDED`, so handlers must be idempotent. The system does not promise: - Exactly-once side effects - Strict global ordering - A distributed transaction with external providers - Automatic replay of dead-lettered deliveries See [Domain Events, outbox, and delivery](/guides/domain-events-and-delivery). **Keep these records separate** Runtime telemetry is not a Domain Event. A Communication Delivery is not an Event Delivery. A Connection Journal entry is not a Domain Event. Conversation Messages are written by the Communications Gateway and Tool Calls by Ingest, while PostgreSQL remains authoritative for Domain Event intent and delivery state. ### Costs Cost reports query persisted model-call records populated by synchronization and recovery. Organization cost reporting requires Organization membership authority; Platform Administrator privilege does not bypass that requirement. Separate Platform cost routes provide explicitly authorized cross-Organization oversight without granting Organization content access. Costs are not calculated from: - Conversation Messages - Tool Calls - Domain Events - Kubernetes resource usage ## Agent runtime architecture The Agent domain orchestrates execution. It does not own communication delivery. ### The Agent domain owns - Agent lifecycle - Runtime selection - Pinned Template or Agent Template Override Version - Assigned Skill Version pins - Agent Secrets and Shared Credential references for tool Integrations - Runtime configuration assembly - Kubernetes resources - LiteLLM key identity ### The Communications domain owns - Platform Plugin registry - Communication Connections - Connection settings - Encrypted provider credentials - Provider sessions - Provider admission and normalization - Durable Communication Deliveries - Canonical Conversation Message writes - Connection health - Connection diagnostics and recovery operations ### Runtime and Platform are separate Hermes and OpenClaw consume one versioned, runtime-neutral Communications protocol, and Runtimes receive protocol credentials rather than provider credentials. Platform support is supplied by trusted, release-shipped Platform Plugins. Generic Connection persistence, CRUD, schema-driven UI, durable delivery, and Runtime adapters do not branch by Platform. A new shipped Platform normally adds one Platform Plugin, its provider client, and focused tests rather than changes to every Runtime. For current per-Platform behavior, see [Compare platform compatibility](/guides/platforms/compatibility); for Runtime selection, see [Choose an Agent runtime](/guides/agents/choose-runtime). ### The Platform Plugin seam Each plugin under `api/domains/communications/plugins/` owns: - Typed settings schema - Typed credential schema - Credential validation - Credential identity and uniqueness rules - Provider admission - Payload normalization - Optional inbound enrichment - Supervised ingress or webhook verification - Outbound sending - Optional processing feedback - Platform capabilities and setup guidance The registry is code-owned and currently ships Slack, Microsoft Teams, Telegram, and Discord. Platform Plugins are not dynamically installed packages. Ingress differs by Platform: Slack uses supervised Socket Mode, Telegram supervised polling, Discord a supervised Gateway session, and Microsoft Teams an authenticated provider webhook. ### Agent start flow Starting an Agent performs: 1. 1 Load the Organization-owned Agent. 2. 2 Authorize the lifecycle operation. 3. 3 Resolve the pinned Template or Agent Template Override Version. 4. 4 Render Template Markdown with Agent identity. 5. 5 Resolve assigned Skill Version pins and eligible Platform Skills. 6. 6 Decrypt Agent Secrets used by tool Integrations. 7. 7 Select the Hermes or OpenClaw Runtime builder. 8. 8 Materialize supported tool Integration configuration. 9. 9 Append tool pointers and unconditional runtime behavior policies. 10. 10 Generate fresh Ingest and Communications protocol credentials. 11. 11 Build and apply the Kubernetes resources, including the runtime-neutral Communications adapter. 12. 12 Mark the Agent `RUNNING`. The generated resources include: - ConfigMap - Secret - PVC - Service - Deployment **Security** Communication Connection credentials are not decrypted into the Runtime. Provider tokens for Slack, Microsoft Teams, Telegram, or Discord are never materialized into Hermes or OpenClaw. A failed credential check or Kubernetes start can place the Agent in `ERROR`, and a successful start clears the previous error. A provider-session or Connection failure changes Connection health instead of Agent lifecycle. ### Desired and realized runtime state | State | Source of truth | | --- | --- | | Agent identity, Runtime, model, and lifecycle status | Agent Barn PostgreSQL | | Selected Template and Skill Versions | Agent Barn PostgreSQL | | Agent Secrets for tool Integrations | Agent Barn PostgreSQL | | Communication Connections, settings, and provider credentials | Agent Barn PostgreSQL, owned by Communications | | Generated runtime configuration | Kubernetes ConfigMap and Secret | | Running process | Kubernetes Deployment and pod | | Workspace files | Agent PVC | | Conversation Message history | Agent Barn PostgreSQL through the Communications Gateway | | Tool Call history | Agent Barn PostgreSQL through Ingest | | Provider spend | Stored cost\_record rows synchronized from LiteLLM, with OpenRouter missing-cost recovery | Runtime configuration is generated at start. A running Agent does not automatically receive a new runtime image, Template selection, Skill assignment, tool Integration policy, or builder change. Applying a runtime-relevant change requires a deliberate stop and start or Apply & Restart workflow. Connection settings and credentials are different: updating a Connection increments its revision and the Communications Gateway reconciles the provider session without restarting the Agent. See [Runtime assembly and deployment](/guides/runtime-and-deployment). ## Configuration and versioning ### Templates Templates are versioned Markdown configuration lineages. An active Agent selects exactly one immutable Template source: a Platform Template Version, an Organization Template Version, or an Agent Template Override Version. The persistence constraint includes all three pin fields. Soft-deleted Agents can retain historical pins or be detached when an old shared lineage is purged; do not describe the active constraint as an unconditional two-foreign-key model. - Predefined Templates are Platform Resources. - Custom Templates belong to one Organization. - Organization forks preserve source lineage. - Published versions are immutable snapshots. - Agents pin a specific version. - Existing Agent pins do not automatically move to the latest version. ### Agent Template Overrides An Agent can have its own private Override lineage: - One mutable draft - Immutable published versions - Explicit version selection - Source-version provenance - Apply & Restart for a running Agent Publishing an Override does not automatically activate it. ### Skills Skills are packaged instructions or references that can be assigned to Agents and required by Templates. They exist at three ownership levels with additive visibility: - Platform Skills - Organization Skills - Agent-private Skills Published Skill Versions are immutable, each custom lineage keeps at most one mutable draft, and Agents and Template versions pin exact versions. Bundled Platform Skills are mounted under isolated `aai-/SKILL.md` roots. Agent startup combines explicitly assigned Skill Version pins, Template-required Skill Versions, and eligible Platform Skills. Provider requirements are validated when configuring the Agent; changing a Skill’s provider metadata later does not retroactively revalidate every existing Agent. See [Templates, versions, overrides, and Skills](/guides/templates-versions-and-skills). ### Organization Agent Settings Model configuration resolves across two layers: - Organization `allowed_models` bounds what the Organization may use. - The Organization Agent Settings default model is nullable: when it is unset, Agents follow the platform default; when it is set, it becomes the Organization-owned default. - An Agent inherits that default unless it carries an explicit override. The effective model is resolved at start, so it can differ from the model a currently running Agent was started with. ### Runtime snapshots Agent startup materializes a snapshot of current configuration into Kubernetes. This creates an intentional boundary: ``` Versioned source configuration ↓ explicit selection Agent persisted configuration ↓ start or restart Generated runtime resources ``` It prevents a new Template, Skill, tool Integration policy, or runtime-image release from silently changing a running Agent. ## Web app architecture The web app uses Next.js App Router with feature-oriented organization. ### Provider hierarchy The root application composes shared providers including: - URL query-state adapter - TanStack Query provider - Tooltip provider - Application provider - User context - Organization context Public authentication routes bypass the protected user and Organization context. ### Route ownership App Router pages under `ui/src/app/` are composition points. Feature behavior belongs under: ``` ui/src/features/ ``` Authentication behavior belongs under: ``` ui/src/auth/ ``` Shared transport and query infrastructure belongs under: ``` ui/src/shared/ ``` ### API client Feature hooks use the shared API transport and centralized query-key infrastructure. It owns: - Authentication token attachment and refresh - Cookie-bearing requests - Request keys converted to `snake_case` - Response keys converted to `camelCase` - Structured `ApiError` - Optional feature-local Zod response validation UI components should not create unrelated transport clients for ordinary product requests. ### Organization switching The Active Organization comes from: ``` /dashboard/[orgId] ``` Platform View uses: ``` /dashboard/platform ``` The Organization provider removes known Organization-scoped query families during a genuine Organization switch so data from the previous Organization does not remain visible under the new URL. Adding a new Organization-scoped query requires either: - Including Organization identity in its query key, or - Adding it to the Organization-switch eviction boundary ### Agent log streaming Agent logs are an exception to the ordinary API flow. A dedicated Next.js route proxies backend server-sent events through a streaming response. The client log hook owns browser reconnection. This prevents ordinary proxy buffering and keeps the internal API hostname on the server. ## Integrations and credentials Agent Barn separates several credential classes. | Credential class | Owner | Purpose | | --- | --- | --- | | Deployment Secret | Platform operator | Configure databases, providers, signing, and infrastructure | | Connection credential | One Communication Connection | Authenticate one provider endpoint inside Communications | | Agent Secret | One Agent | Give one Runtime access to a tool Integration | | Shared Credential | Organization | Reuse one tool Integration credential across Agents | | LiteLLM virtual key | Agent or API service | Attribute and authorize model access | | Ingest key | One Agent start | Authenticate Runtime Tool Call telemetry | | Communications protocol credential | One Agent start | Claim and complete durable Communication Deliveries | ### Encryption boundary Provider payloads are: 1. Validated against typed schemas. 2. Encrypted before persistence. 3. Validated again after decryption. 4. Returned through read APIs only as safe metadata. 5. Decrypted by the domain that owns them. Credential plaintext is never returned by normal read APIs. ### Runtime materialization At start, Agent Barn can produce: - aai-cli profiles and secret-store setup - Google Workspace `gog` configuration - Tool Integration environment variables - Eligible Platform Skills - Tool Integration policy appended to Agent configuration - Firecrawl platform defaults or per-Agent overrides - Ingest and Communications protocol credentials Storage validation is only half of a tool Integration: Runtime support must also exist in the Runtime builders and their generated artifacts. **Important** A Communication Platform is not a tool Integration. Provider credentials for Slack, Microsoft Teams, Telegram, and Discord belong to a Communication Connection inside the Communications domain, are validated by that Platform Plugin, and never reach a Runtime. ## Deployment architecture Helmfile deploys the platform into one Kubernetes namespace. The general dependency order is: PostgreSQL services and RedisLiteLLM and FirecrawlAgent Barn API hooksAPI, Communications, and workersAgent Barn UIMonitoring ``` PostgreSQL services and Redis ↓ LiteLLM and Firecrawl ↓ Agent Barn API hooks ↓ API, Communications, and worker workloads ↓ Agent Barn UI ↓ Monitoring ``` ### Service workloads | Workload | Responsibility | | --- | --- | | Product API | Product HTTP contracts, orchestration, and metrics on port 8000 | | Ingest API | Runtime Tool Call telemetry and metrics on port 8001 | | Communications | Provider ingress, durable delivery, provider sessions, Runtime protocol, and metrics on port 8002 | | API worker | Dramatiq Event Delivery processing | | Reconciliation CronJob | Republish eligible pending or stale Event Deliveries | | UI | Next.js application and API proxy | | LiteLLM | Model proxy, Agent virtual keys, spend records | | Firecrawl | Web search and scraping | | Redis | Event Delivery and Firecrawl transport | | Monitoring | Prometheus, Grafana, Alertmanager, kube-state-metrics | The API image is reused for: - Product, Ingest, and Communications processes - Worker deployment - Reconciliation CronJob - Alembic migration Job These workloads run different commands but share application code and configuration contracts. Provider webhook ingress reaches the Communications service, not the Product API. Runtime protocol traffic stays internal. Operational procedures belong to the deployment guides; see [Runtime assembly and deployment](/guides/runtime-and-deployment). Data and infrastructure responsibilities divide as follows. ### PostgreSQL - Product state - Communication Connections and Deliveries - Conversation Messages - Tool Calls - Domain Event outbox and deliveries - Connection Journal ### Redis - Domain Event delivery transport - Not the source of durable Domain Event truth ### LiteLLM - Model routing - Agent cost identity and reporting ### Kubernetes - Runtime ConfigMaps - Runtime Secrets - PVCs - Services - Deployments ### Communications service - Provider-session leases - Provider ingress - Outbound processing - Connection health and metrics ### Dynamic Agent workloads Agent Deployments are not static entries in Helmfile. The API dynamically creates and removes them through the Kubernetes client. Each running Agent owns its runtime resources while remaining part of the same `agent-farm` or `agent-farm-staging` namespace. ### Environment isolation The k3s testing deployments use: ``` agent-farm ``` and: ``` agent-farm-staging ``` Every release and Helmfile dependency uses the selected namespace. The API also receives that namespace so it creates Agent resources in the correct environment. ## Observability The Product API, Ingest, Communications, LiteLLM, and Agent runtimes expose Prometheus metrics. The namespace-scoped monitoring stack contains: - Prometheus - Grafana - Alertmanager - kube-state-metrics It observes: - Product, Ingest, and Communications availability - HTTP requests and errors - Application database connectivity - Connection status and delivery outcomes - Agent health and ERROR state - Agent restarts - Tool Call outcomes - LiteLLM availability and usage - OpenRouter credit state The Product API health endpoint proves application PostgreSQL connectivity. It does not prove Redis, Communications, workers, Event Deliveries, provider Connections, LiteLLM, Firecrawl, email, or external providers are healthy. Application logs, Agent logs, the Connection Journal, Event Delivery state, Activity, Tool Calls, costs, and Prometheus metrics are complementary operational sources. ## Where changes belong ### Code map | Concern | Source path | | --- | --- | | Product application composition | `api/api_app.py` | | Ingest application composition | `api/ingest_app.py` | | Communications application composition | `api/communications_app.py` | | Agent lifecycle and runtime assembly | `api/domains/agents/` | | Organization Agent Settings | `api/domains/agent_settings/` | | Connections, plugins, and delivery | `api/domains/communications/` | | Conversation Message persistence | `api/domains/conversations/` | | Tool Call telemetry | `api/domains/ingest/` | | Domain Events, outbox, and delivery | `api/domains/events/` | | Skills and Skill Versions | `api/domains/skills/` | | Templates and versions | `api/domains/templates/` | | Dashboard Web Chat and authenticated SSE | `api/domains/web_chat/` | | External system adapters | `api/infrastructure/` | | App Router pages | `ui/src/app/` | | UI feature modules | `ui/src/features/` | | Shared UI transport and query keys | `ui/src/shared/` | | Hermes runtime image | `hermes-base/` | | OpenClaw runtime image | `openclaw-base/` | | Charts | `helm/` | | Release composition | `helmfile.yaml.gotmpl` | | Build and deployment workflows | `.github/workflows/` | ### Source map | Concern | Start with | | --- | --- | | Domain terminology | `CONTEXT.md` | | Context routing | `docs/INDEX.md` | | Cross-system relationships | `docs/architecture/system-map.md` | | API layering and tenancy | `docs/architecture/api.md` | | UI providers and data flow | `docs/architecture/ui.md` | | Runtime and deployment | `docs/architecture/runtime-and-deployment.md` | | Identity and Organizations | `docs/features/identity-and-organizations.md` | | Roles and Agent Access | `docs/features/rbac/IMPLEMENTATION-BRIEF.md` | | Agent lifecycle and configuration | `docs/features/agents.md` | | Templates and Skills | `docs/features/templates-and-skills.md` | | Runtime activity and Ingest | `docs/features/activity-and-ingest.md` | | Internal events and delivery | `docs/features/domain-events.md` | | Cost attribution | `docs/features/costs.md` | | Tool Integration credentials | `docs/features/integrations.md` | | Hard-to-reverse rationale | `docs/adr/` | | Repeatable implementation rules | `docs/guidelines/` | ### Choose the authoritative document | Information | Authoritative location | | --- | --- | | Current behavior and invariants | Feature or architecture document | | Canonical product language | `CONTEXT.md` | | Repeatable engineering convention | Matching guideline | | Consequential architectural rationale | ADR | | Active multi-ticket delivery state | Feature changelog | | Proposed work | Issue tracker or plan | Do not treat an implementation plan as proof that a feature is delivered. ### Change-impact questions Before changing a boundary, ask: - Does it affect Organization tenancy? - Does it expose an Agent or subordinate resource? - Does it require a new Permission? - Does it change the Agent start snapshot? - Does it affect both Hermes and OpenClaw? - Does it belong in a Platform Plugin rather than a Runtime? - Does it change the Communications protocol version? - Does it change encrypted credential compatibility? - Does it change a persisted schema? - Does it require an Alembic migration? - Does it produce a Domain Event? - Does the business mutation need one explicit transaction? - Does it change Tool Call telemetry? - Does it affect cost attribution? - Does the UI need a Zod schema or query-cache update? - Does the deployment or monitoring contract change? - Do authoritative docs need to move with the code? ## Architecture invariants Keep these invariants intact unless the change deliberately revises the documented contract: - Organization is the user-visible tenant boundary. - Platform authority and Organization authority remain separate. - Agent authorization covers the complete Agent aggregate, including subordinate Communication Connections. - Routes remain thin. - Services own orchestration. - Repositories own persistence and tenant visibility. - Infrastructure adapters own external systems. - Runtime and Platform remain separate. - Runtimes receive protocol credentials, not provider credentials. - Communication Connections are Agent subordinate resources, not fields inside the Agent aggregate. - The Communications Gateway owns canonical Conversation Message writes. - Generated Runtime configuration changes only through deliberate lifecycle action, while Connection settings and credentials reconcile without restarting the Agent. - Runtime telemetry remains separate from Domain Events. - PostgreSQL remains authoritative for Domain Event delivery state. - Event Handlers remain idempotent under at-least-once delivery. - Cost reports read persisted model calls populated by synchronization and recovery. - Secret plaintext is never returned through read APIs. - Schema changes include Alembic migrations. - API, UI, runtime, tests, deployment, and documentation move together when a contract crosses those boundaries. ## Next steps Continue with the API guide to understand route registration, request contracts, service orchestration, repository visibility, dependency injection, and integration patterns. [Develop against the API ](/guides/develop/api) ## Web Chat ownership The `api/domains/web_chat/` domain owns dashboard chat requests, user-scoped thread history, and the authenticated SSE surface. It uses the Communications delivery pipeline and built-in Web Chat Connection. Reads require Agent activity permission; mutations require Agent update permission. This is distinct from external provider webhook authentication and runtime protocol credentials. Dashboard Chat → Product API Web Chat domain → Communications delivery pipeline → built-in Web Chat Connection. See [Dashboard Web Chat](/guides/agents/web-chat) for the user workflow.