Develop and extend

On this page

Overview
Distinguish the concepts
Understand the event envelope
Choose an event scope
Design a safe payload
Register an event
Produce an event atomically
Dispatch committed deliveries
Add an Event Handler
Understand delivery lifecycle
Run workers and reconciliation
Monitor Event Deliveries
Test the event slice
Troubleshooting
Next steps

Agent Barn uses Domain Events to preserve and deliver immutable business facts — an Agent starting, a Membership role changing, Platform Privilege being granted.

A Domain Event is committed with the business mutation that produced it. PostgreSQL records both the fact and its intended handler deliveries. Dramatiq and Redis then provide low-latency, at-least-once delivery.

Overview

Use a Domain Event when…

A committed business fact must be consumed asynchronously
An internal projection must survive the originating request
A Security Audit Record should be derived from a security-sensitive change
A notification or other handler must be recoverable after queue failure
Business state and event intent must either both commit or both roll back

Do not use one…

To move ordinary route logic to a queue
To represent Hermes or OpenClaw telemetry
To implement a public webhook
To replace API response data
To store a mutable entity’s complete history
To guarantee strict ordering
To guarantee exactly-once side effects

Domain Events are internal, immutable, typed business facts. They are not runtime Telemetry Events, public webhooks, Security Audit Records, queue messages, or event-sourced entity history.

The system makes a deliberate set of promises — and just as deliberately withholds others:

Agent Barn provides

Durable event intent in PostgreSQL
One intended Delivery per registered handler
At-least-once handler execution
Atomic Delivery claims
Bounded Dramatiq retries
Stale-delivery reconciliation
Terminal success and dead-letter states

Agent Barn does not provide

Exactly-once handler side effects
Strict global ordering
Distributed transactions with external handlers
Automatic replay of terminal deliveries
Historical delivery creation when a handler is added later
Event-sourced reconstruction of product entities

Distinguish the concepts

Several related records participate in the flow, but they are not interchangeable.

Domain Event

Immutable business fact

A typed, internal business fact represented by a validated envelope carrying scope, identity, correlation, and a bounded payload.

Outbox Message

Immutable business fact

The immutable PostgreSQL record for one committed Domain Event, in event_outbox_message. It stores transport-neutral event data — no retry state, Redis routing, or worker configuration.

Event Delivery

Mutable operational state

Delivery state for one event and one named handler, in event_delivery. Its identity is (event_id, handler_name), so a retry updates the same row rather than adding another.

Event Handler

Mutable operational state

A statically registered internal consumer with a stable name and declared supported event versions. It receives the envelope and a small delivery context, and performs an idempotent side effect.

Security Audit Record

A separate projection

An immutable compliance projection derived from selected Domain Events. It stores stable actor and subject snapshots and survives deletion of the referenced product record.

Telemetry Event

Not a Domain Event

Runtime activity posted by Hermes or OpenClaw through the Ingest API. It can become a Conversation Message or Tool Call, and is never written to the Domain Event outbox.

Domain Event names look like this:

agent.started
organization.role.changed
agent.access.granted
platform.user_privilege.revoked

An Outbox Message lives in event_outbox_message, and an Event Delivery in event_delivery. One event can produce several Event Deliveries, whose identity is (event_id, handler_name) — so a retry updates the same Event Delivery rather than creating another row.

Runtime Ingest stays entirely separate from the Domain Event outbox:

Runtime telemetry                    Internal Domain Events
       │                                      │
       ▼                                      ▼
Ingest API                            Business repository transaction
       │                                      │
       ▼                                      ▼
Conversation Messages                Outbox Message
and Tool Calls                        and Event Deliveries

Understand the event envelope

Every Domain Event uses a frozen DomainEventEnvelope.

DomainEventEnvelope
├── event_id
├── event_name
├── schema_version
├── occurred_at
├── event_scope
├── organization_id
├── actor
├── subject
├── correlation_id
├── causation_id
└── payload

Field	Purpose
`event_id`	Unique immutable identity for the event
`event_name`	Stable dot-separated business fact name
`schema_version`	Version of the registered payload contract
`occurred_at`	Time the business fact occurred
`event_scope`	ORGANIZATION or PLATFORM
`organization_id`	Required for Organization scope, prohibited for Platform scope
`actor`	Typed identity responsible for the change
`subject`	Typed identity of the changed or affected resource
`correlation_id`	Required ID grouping related work
`causation_id`	Optional ID of the event or operation that caused this fact
`payload`	Validated, bounded, secret-safe JSON object

Actor identity types

The current envelope supports MEMBERSHIP, USER, SYSTEM, and RUNTIME. For an authenticated Organization operation, use the shared resolver:

actor = resolve_actor_identity(context, organization_id)

The resolver uses the actor’s Membership when one exists for the Organization. Otherwise it returns a User identity, without inventing Membership authority.

For maintenance or infrastructure work, use an explicit stable System identity:

ActorIdentity(
    type=ActorIdentityType.SYSTEM,
    id="agent-maintenance",
)

Subject identity types

The current subject catalogue includes AGENT, MEMBERSHIP, ORGANIZATION, TEMPLATE, SKILL, SYSTEM, and USER.

Choose the resource the event is fundamentally about. Related resource IDs can appear in the payload when the payload schema requires them.

Choose an event scope

Every event definition has an explicit scope.

Organization event

Uses EventScope.ORGANIZATION
Requires exactly one organization_id
Can use Organization resource subjects
Can use a Membership actor
Rejects actor, subject, and payload Organization references that disagree with the envelope

event = EVENT_REGISTRY.build_event(
    event_name=AGENT_UPDATED,
    schema_version=1,
    occurred_at=datetime.now(UTC),
    organization_id=agent.organization_id,
    actor=resolve_actor_identity(context, agent.organization_id),
    subject=SubjectIdentity(
        type=SubjectIdentityType.AGENT,
        id=agent.id,
        organization_id=agent.organization_id,
    ),
    correlation_id=uuid4(),
    payload={
        "organization_id": agent.organization_id,
        "agent_id": agent.id,
        "field_changes": field_changes,
        "actor_display": context.user.full_name or context.user.email,
        "subject_display": agent.name,
    },
)

The registered event definition supplies the Organization scope. A caller may supply the same scope explicitly, but cannot override the registered one.

Platform event

Uses EventScope.PLATFORM
Requires organization_id=None
Cannot reference an Organization in actor, subject, or payload data
Cannot use a Membership actor
Can use a User or System subject

event = EVENT_REGISTRY.build_event(
    event_name=PLATFORM_USER_PRIVILEGE_GRANTED,
    schema_version=1,
    occurred_at=datetime.now(UTC),
    event_scope=EventScope.PLATFORM,
    organization_id=None,
    actor=ActorIdentity(
        type=ActorIdentityType.USER,
        id=actor_user_id,
    ),
    subject=SubjectIdentity(
        type=SubjectIdentityType.USER,
        id=subject_user_id,
    ),
    correlation_id=uuid4(),
    payload={
        "actor_user_id": actor_user_id,
        "actor_display": actor_email,
        "subject_user_id": subject_user_id,
        "subject_display": subject_email,
        "reason": reason,
    },
)

Do not create a synthetic Organization to represent Platform authority.

Design a safe payload

Each event version has a Pydantic payload model. extra="forbid" prevents a producer from silently adding unreviewed fields.

class AgentUpdatedPayload(BaseModel):
    model_config = ConfigDict(extra="forbid")

    organization_id: UUID
    agent_id: UUID
    field_changes: dict[str, dict[str, Any]]
    actor_display: str
    subject_display: str

After Pydantic validation, the registry recursively validates the serialized JSON.

Allowed

UUIDs serialized through the payload model
Short identifiers
Bounded display snapshots
Enum or status values
Small before-and-after field values
Added and removed lists
Safe metadata a handler requires
null, booleans, integers, finite floats, strings, objects, and arrays

Rejected

A top-level value that is not an object
Unsupported event names or versions
Missing required schema fields
Extra fields forbidden by the payload model
Unsupported Python values
Non-string object keys
Non-finite floating-point values
Payloads larger than 16 KiB after JSON serialization
Cross-Organization references
Organization references in Platform events
Sensitive-looking keys at any nesting depth

Prefer a concise diff over two potentially unbounded full catalogues:

{
  "added": ["model-a"],
  "removed": ["model-b"]
}

Sensitive key fragments

Sensitive key matching is case-insensitive and normalizes hyphens and underscores. These fragments are rejected at any nesting depth:

api_key
apikey
authorization
client_secret
credential
password
private_key
refresh_token
secret
token

record_id
provider
label
shared_reference_id

Build payloads from an allowlist

Do not serialize a database model, request body, provider response, or configuration object directly into a Domain Event. Explicitly select the safe fields the event’s consumers require.

Never include:

Passwords
Tokens
API keys
Authorization headers
Encrypted credential values
Private keys
Full provider callback payloads
Raw prompt bodies
Unbounded logs
Arbitrary exception objects
Mutable model dumps

Register an event

Event contracts live in api/domains/events/catalog.py. Registration has two parts: define the event name and payload model, then register the name, schema version, scope, and intended handler names.

AGENT_UPDATED = "agent.updated"


class AgentUpdatedPayload(BaseModel):
    model_config = ConfigDict(extra="forbid")

    organization_id: UUID
    agent_id: UUID
    field_changes: dict[str, dict[str, Any]]
    actor_display: str
    subject_display: str


registry.register(
    DomainEventDefinition(
        event_name=AGENT_UPDATED,
        schema_version=1,
        payload_model=AgentUpdatedPayload,
        handler_names=(SECURITY_AUDIT_HANDLER,),
        event_scope=EventScope.ORGANIZATION,
    )
)

The registry rejects duplicate (event_name, schema_version) definitions.

Two registries

Registry	Responsibility
`DomainEventRegistry`	Event names, schema versions, scope, payload model, and intended handler names
`EventHandlerRegistry`	Actual handler implementations, and the event versions each implementation supports

The first registry decides which Event Deliveries are created. The second determines whether the worker can execute a named delivery. These contracts must remain aligned.

Version an event safely

Persisted Outbox Messages are immutable. Do not change a version-1 payload model incompatibly after version-1 events have been committed. For a breaking change:

Add a version-2 payload model.
Register (event_name, 2).
Keep version 1 registered while persisted version-1 deliveries remain relevant.
Update intended handlers to declare support for each version they can process.
Test both versions.
Update projections, monitoring assumptions, and documentation.

Handlerless events

An event can intentionally have no handlers:

DomainEventDefinition(
    event_name=AGENT_CREATED,
    schema_version=1,
    payload_model=AgentCreatedPayload,
    event_scope=EventScope.ORGANIZATION,
)

Such an event still creates an Outbox Message, but creates zero Event Deliveries and does not appear in the Event Delivery Monitor.

Produce an event atomically

An event-producing business mutation requires one explicit repository transaction. Business rows, the Outbox Message, and the Event Deliveries all commit together — or none of them do.

BEGIN
  ├── lock or load business rows
  ├── validate and mutate business state
  ├── flush business state
  ├── build Domain Event
  ├── stage Outbox Message
  ├── stage intended Event Deliveries
  ├── capture Delivery IDs
  └── COMMIT

If event validation, Outbox insertion, Delivery insertion, or the business write fails, the whole transaction rolls back.

A useful repository result carries both the changed resource and the committed Delivery IDs:

@dataclass(frozen=True)
class ResourceChangeResult:
    resource: Resource
    delivery_ids: list[UUID]

A representative repository operation is:

def update_with_event(
    self,
    resource_id: UUID,
    *,
    organization_id: UUID,
    actor: ActorIdentity,
    new_name: str,
    correlation_id: UUID,
) -> ResourceChangeResult | None:
    with Session(
        self.delegate.engine,
        expire_on_commit=False,
    ) as session:
        resource = session.exec(
            select(Resource)
            .where(
                col(Resource.id) == resource_id,
                col(Resource.organization_id) == organization_id,
            )
            .with_for_update()
        ).first()

        if resource is None:
            return None

        previous_name = resource.name
        resource.name = new_name
        session.add(resource)
        session.flush()

        event = EVENT_REGISTRY.build_event(
            event_name=RESOURCE_UPDATED,
            schema_version=1,
            occurred_at=datetime.now(UTC),
            organization_id=organization_id,
            actor=actor,
            subject=SubjectIdentity(
                type=SubjectIdentityType.SYSTEM,
                id=resource.id,
                organization_id=organization_id,
            ),
            correlation_id=correlation_id,
            payload={
                "organization_id": organization_id,
                "resource_id": resource.id,
                "previous_name": previous_name,
                "new_name": new_name,
            },
        )

        self.outbox_repository.stage(
            session=session,
            event=event,
            registry=EVENT_REGISTRY,
        )

        delivery_ids = list(
            session.exec(
                select(EventDelivery.id).where(
                    EventDelivery.event_id == event.event_id
                )
            )
        )

        session.commit()

        return ResourceChangeResult(
            resource=resource,
            delivery_ids=delivery_ids,
        )

The example uses generic resource names. A real implementation must use an existing typed Subject Identity, or deliberately extend the identity catalogue when a new domain subject is required. The important contract is the transaction shape.

Repository rules

Do

The domain-specific repository owns the session
Scope the business query to its Organization
Lock rows when the invariant is concurrency-sensitive
Flush the business mutation before staging when generated state is required
Build the event through EVENT_REGISTRY
Call outbox_repository.stage() with the existing session
Query the generated Event Delivery IDs before commit
Commit once
Return committed Delivery IDs to the service
Use explicit result types

Do not

Build events in a route
Pass a database session to a route
Publish to Dramatiq inside the database transaction
Let the outbox open a second transaction
Save business state through a session-per-operation delegate and stage the event afterward
Add optional event arguments to the generic repository delegate
Create a Domain Event for a no-op, unless the event represents the attempted action

Dispatch committed deliveries

After the repository commits, the service performs best-effort immediate enqueue:

result = repository.update_with_event(...)

if result is None:
    raise HTTPException(status_code=404)

event_delivery_dispatcher.enqueue_immediate(
    result.delivery_ids,
)

return result.resource

The order is mandatory — commit first, enqueue second:

Database commit
      │
      ▼
Best-effort immediate enqueue
      ├── success → mark Delivery ENQUEUED
      └── failure → leave committed work recoverable

EventDeliveryDispatcher sends the Delivery ID through the transport, marks the Delivery ENQUEUED only after the transport call returns successfully, bounds and redacts enqueue errors, then logs failure and continues.

Queue message contents

A Dramatiq message contains

Event Delivery ID
correlation_id
published_at
source

It must not contain

Domain Event payload
Actor or subject identity
Handler-routing authority
Retry truth
Credentials
Secrets
Authorization data

The worker reloads the Event Delivery and Outbox Message from PostgreSQL. Redis is transport, not the source of business truth.

Add an Event Handler

An Event Handler has a stable name, declared supported event versions, and one handle operation.

The handler contract

A unique, stable name
Declared supported event names and schema versions
Receives a reconstructed DomainEventEnvelope
Receives a small EventDeliveryContext
Performs an idempotent side effect
Returns normally, or raises a classified error

from collections.abc import Sequence
from dataclasses import dataclass
from typing import ClassVar

from api.domains.events import (
    DomainEventEnvelope,
    EventDeliveryContext,
    RetryableEventHandlerError,
    SupportedEvent,
    TerminalEventHandlerError,
)


@dataclass
class ResourceProjectionHandler:
    name: ClassVar[str] = "resource.projection"

    supported_events: ClassVar[Sequence[SupportedEvent]] = (
        SupportedEvent("resource.updated", 1),
    )

    def handle(
        self,
        event: DomainEventEnvelope,
        context: EventDeliveryContext,
    ) -> None:
        # Use event.event_id or
        # (event.event_id, context.handler_name)
        # as the idempotency key.
        ...

@provider
@singleton
def provide_event_handler_registry(
    self,
    lifecycle_email_handler: AgentLifecycleEmailHandler,
    security_audit_projection: SecurityAuditProjection,
    resource_projection: ResourceProjectionHandler,
) -> EventHandlerRegistry:
    return EventHandlerRegistry(
        [
            lifecycle_email_handler,
            security_audit_projection,
            resource_projection,
        ]
    )

The handler’s supported_events must agree with the event definition that names it.

Outcomes

Handler outcome	Processor behavior
Returns normally	Mark the Delivery SUCCEEDED and clear the current error
Raises RetryableEventHandlerError	Record a bounded error and use Dramatiq’s retry path
Raises TerminalEventHandlerError	Mark the Delivery DEAD_LETTERED with TERMINAL_HANDLER_ERROR
Raises an unexpected exception	Treat it as retryable, so unclassified transient failures are not silently dropped
Handler name is unknown	Dead-letter with UNKNOWN_HANDLER
Handler does not support the event version	Dead-letter with UNSUPPORTED_EVENT
Outbox Message is missing or invalid	Dead-letter with INVALID_DELIVERY
Dramatiq exhausts retries	Dead-letter with RETRY_EXHAUSTED

Handlers do not update Event Delivery state directly. The processor owns claims, attempts, success, retry metadata, and dead-letter transitions.

Design for idempotency

Use a durable idempotency key:

event_id
(event_id, handler_name)
(delivery_id, recipient)

The Security Audit projection uses a unique Event ID and saves only when absent. The Agent lifecycle email handler records each notified recipient for a Delivery, and on retry skips recipients already notified.

A handler that writes its own database projection should normally use its own explicit transaction, enforce a unique idempotency constraint, treat an existing projection as success, avoid updating Event Delivery lifecycle state, and keep external side-effect idempotency separate from database delivery state.

Understand delivery lifecycle

PENDING
ENQUEUED
PROCESSING
SUCCEEDED
DEAD_LETTERED

PENDING
   │ publish
   ▼
ENQUEUED
   │ atomic worker claim
   ▼
PROCESSING
   ├── success ─────────────────► SUCCEEDED
   ├── retryable failure ───────► retry path
   └── terminal/exhausted error ► DEAD_LETTERED

The diagram reads: a Delivery starts PENDING, becomes ENQUEUED once published, and becomes PROCESSING once a worker claims it atomically. From PROCESSING it either succeeds, returns to the retry path, or dead-letters.

Reconciliation can republish eligible old PENDING, stale ENQUEUED, and stale PROCESSING deliveries back into delivery processing. It never automatically republishes SUCCEEDED or DEAD_LETTERED deliveries.

Lifecycle fields

Field	Meaning
`status`	Current delivery lifecycle state
`attempt_count`	Number of handler execution claims
`created_at`	Delivery creation, and the PENDING age clock
`enqueued_at`	Last known enqueue time
`claimed_at`	Last worker claim time
`completed_at`	Success or dead-letter completion time
`last_error`	Current unresolved bounded and redacted error
`dead_letter_reason`	Required reason for a dead-lettered delivery

attempt_count is not a queue-publication count. It increments when a worker successfully claims the Delivery for handler execution. A later success clears last_error, but preserves attempt_count as historical metadata.

Dead-letter reasons

RETRY_EXHAUSTED
TERMINAL_HANDLER_ERROR
UNKNOWN_HANDLER
UNSUPPORTED_EVENT
INVALID_DELIVERY

The system does not currently provide a manual replay or handler-remapping workflow for dead-lettered deliveries.

Run workers and reconciliation

Complete local stack

Starts Redis and the general-purpose Dramatiq worker along with everything else.

make run

Native development

Running the Product API outside Docker needs three terminals:

make redis-up

make dev-worker

make dev-api

make dev-worker runs:

uv run dramatiq api.worker_app --path .. --processes 1 --threads 4 --watch .

The worker reads REDIS_URL from the repository environment.

Run one reconciliation pass with:

make reconcile

Current operational constants

Setting	Value
Pending grace period	60 seconds
Enqueued stale threshold	300 seconds
Processing stale threshold	900 seconds
Dramatiq maximum retries	20
Minimum retry backoff	15 seconds
Maximum retry backoff	900 seconds
Reconciliation batch size	100
Reconciliation maximum runtime	60 seconds
Reconciliation publish concurrency	5

These timing and retry values are code constants. The Redis URL and the infrastructure reconciliation schedule are externally configured.

Reconciliation selects eligible Event Deliveries, uses row locking and SKIP LOCKED, claims a bounded batch, republishes with low concurrency, logs individual failures, leaves failed publications eligible once they become stale again, and reports a run summary. It scans Event Deliveries, not Outbox Messages.

Monitor Event Deliveries

`GET` endpoint

/api/v1/platform/event-deliveries/summary

Lifecycle counts, stale and unknown-age counts, and the oldest active-state age.

`GET` endpoint

/api/v1/platform/event-deliveries/event-types

Event types capable of producing Deliveries. Handlerless definitions are excluded, because they cannot.

`GET` endpoint

/api/v1/platform/event-deliveries

The filtered, deterministically paginated explorer over individual Event Deliveries.

Platform View route

/dashboard/platform/event-deliveries

Manual refresh rather than polling, with infinite loading and list virtualization for the explorer.

The monitor provides

Lifecycle counts, stale counts, and oldest active-state age
Delivery identity, event name, and schema version
Handler name and Organization display where applicable
Attempt count and lifecycle timestamps
Dead-letter reason
Bounded and redacted current error
Curated actor and subject display snapshots when the safe payload provides them

Intentionally excluded

The full Event Payload
Raw Actor Identity
Raw Subject Identity
Correlation ID
Causation ID
Secrets or credentials
Replay actions
Delivery state mutation

The explorer supports filters for status, Organization, event name, created-at range, sort direction, and search. Search matches an exact Delivery ID or Event ID, or a case-insensitive Organization-name, event-name, or handler-name prefix. It never searches last_error.

State age

Status	Age clock
`PENDING`	`created_at`
`ENQUEUED`	`enqueued_at`
`PROCESSING`	`claimed_at`
`SUCCEEDED`	`completed_at`
`DEAD_LETTERED`	`completed_at`

A missing required timestamp is reported as unknown age. It is never silently replaced with created_at.

Test the event slice

A new event or handler needs coverage at several layers.

Concern	Test
Event name and version registration	Registry unit test
Required payload fields	Payload validation unit test
Extra or unsupported fields	Payload validation unit test
Sensitive nested keys	Secret-safety unit test
Payload-size limit	Boundary unit test
Organization and Platform scope	Scope-validation unit test
Cross-Organization references	Tenant-safety unit test
Business state plus outbox atomicity	PostgreSQL integration test
Validation or insert failure rollback	PostgreSQL integration test
Intended Delivery creation	PostgreSQL integration test
Immediate enqueue success	Dispatcher or service test
Immediate enqueue failure	Confirm the committed Delivery remains recoverable
Handler normal completion	Processor test
Retryable handler failure	Processor and worker test
Terminal handler failure	Processor test
Unknown handler or unsupported version	Dead-letter test
Duplicate delivery message	No-op or idempotency test
Handler side-effect idempotency	Handler-specific test
Retry exhaustion	Worker callback test
Reconciliation selection	Repository and reconciler test
Platform monitor contract	API integration test
Monitor UI change	Playwright test

Representative test locations include:

api/tests/unit/test_domain_events.py
api/tests/unit/test_event_handlers.py
api/tests/unit/test_event_delivery_transport.py
api/tests/unit/test_event_delivery_reconciliation.py
api/tests/integration/test_outbox_messages.py
api/tests/integration/test_event_delivery_monitor.py
ui/tests/e2e/event-deliveries.spec.ts

A new event-producing mutation should prove that:

The business state commits with its Outbox Message and intended Deliveries
An event validation failure rolls back the business mutation
An Outbox or Delivery insert failure rolls back the business mutation
Enqueue happens after commit
Queue failure does not lose the committed Delivery
The intended handler can process the reconstructed envelope
Repeated handler execution does not duplicate its side effect

Run the standard API verification:

make check-api
make test-api
make check-migrations

If the monitor UI changes, also run:

make lint-ui
make check-ui
make test-ui

Run Kubernetes checks only when the change affects deployment or scheduled reconciliation behavior:

make test-api-k8s

Troubleshooting

Issue	Likely cause	What to check
Unsupported Domain Event	The event name or schema version is not registered	Check catalog.py and the exact (event_name, schema_version) pair
A payload is rejected for a harmless field	Its name contains a sensitive substring	Rename the field to a safe, allowlisted domain term
A payload is rejected as too large	Serialized JSON exceeds 16 KiB	Store a concise diff or identifiers instead of full documents and catalogues
The event rejects an actor or subject	Identity Organization metadata does not match the envelope scope	Check the actor, the subject, and the organization_id values
A Platform event rejects a resource	Platform events cannot reference Organization-owned subjects	Use a User or System subject, or reconsider the scope
Business state commits but no Event Delivery exists	The definition has no intended handlers, or the mutation bypassed transactional staging	Check handler_names, the Outbox Message, and the repository transaction
An Outbox Message exists with zero Deliveries	The event is intentionally handlerless	This is expected. It will not appear in the Event Delivery Monitor
A Delivery stays PENDING	Immediate enqueue failed, or the worker path is unavailable	Check Redis, worker logs, the pending grace period, and reconciliation
A Delivery stays ENQUEUED	The message was lost, delayed, or never claimed	Check the worker, then wait for or run stale-delivery reconciliation
A Delivery stays PROCESSING	A worker crashed or stopped after claiming it	Check claimed_at, worker logs, and the processing stale threshold
A Delivery is DEAD_LETTERED with UNKNOWN_HANDLER	A handler was renamed, removed, or never registered	Restore compatibility, or apply an explicitly designed migration
A Delivery is DEAD_LETTERED with UNSUPPORTED_EVENT	The handler does not declare support for the persisted version	Add compatible version support and verify registration
A Delivery is DEAD_LETTERED after retries	A transient failure never recovered	Inspect the bounded error and the handler’s external dependency
A handler side effect happens twice	The handler is not idempotent across crash recovery	Add a durable idempotency key, or a recipient or projection record
attempt_count increases unexpectedly	The Delivery was claimed more than once	This can occur with at-least-once recovery. Verify handler idempotency
last_error displays a redaction message	The error text contained a sensitive-key fragment	Fix the underlying exception so it does not include sensitive information
Reconciliation scans no rows	Deliveries are not old or stale enough, or are already terminal	Check lifecycle timestamps and the configured thresholds
An event is absent from the monitor event filter	The definition has no handlers	The monitor lists only event types capable of producing Deliveries
A non-administrator receives 403 from monitor endpoints	The monitor is Platform Administrator-only	Use an authenticated Platform Administrator session
A newly added handler receives no historical events	Handler registration is not retroactive	The current system has no automatic backfill or replay path

Next steps

Next Test a change Select the correct unit, integration, browser, and operational coverage. Domain Events, Outbox, and Delivery The compact engineering reference. Develop against the API Repository transactions, dependency injection, migrations, and API tests. Develop the web app For changes to the Platform Event Delivery Monitor. Monitor the platform The broader production observability surface. Review Agent activity Why runtime telemetry is separate from Domain Events.

Overview

Use a Domain Event when…

Do not use one…

Agent Barn provides

Agent Barn does not provide

Distinguish the concepts

Domain Event

Outbox Message

Event Delivery

Event Handler

Security Audit Record

Telemetry Event

Understand the event envelope

Actor identity types

Subject identity types

Choose an event scope

Organization event

Platform event

Design a safe payload

Allowed

Rejected

Sensitive key fragments

Build payloads from an allowlist

Two registries

Version an event safely

Handlerless events

Repository rules

Do

Do not

Queue message contents

A Dramatiq message contains

It must not contain

The handler contract

Outcomes

Design for idempotency

Understand delivery lifecycle

Lifecycle fields

Dead-letter reasons

Run workers and reconciliation

Complete local stack

Native development

Current operational constants

Monitor Event Deliveries

GET endpoint

GET endpoint

GET endpoint

Platform View route

The monitor provides

Intentionally excluded

State age

Test the event slice

Troubleshooting

Next steps

`GET` endpoint

`GET` endpoint

`GET` endpoint