Publish Events with a Transactional Outbox

Write the event to an outbox table in the same transaction as the state change, then relay it asynchronously with retries and idempotent consumers.

~45 minmessagingreliabilitytransactions

Task

Guarantee at-least-once event publication without a two-phase commit between the database and a broker.

When to use this

A service commits a state change then tries to publish, but the publish fails and the event is lost, so downstream state never catches up.

Prerequisites

A database that can write both tables in one transaction
A relay worker and a broker

Steps

1. Insert the event into an outbox table inside the same transaction as the state change.

Expected output: The event and the state change commit together.

2. Run a relay that polls unpublished outbox rows, publishes them, and marks them sent.

Expected output: Events reach the broker without a distributed transaction.

3. Delete or archive published rows on a schedule to bound table growth.

Expected output: The outbox stays small.

4. Make consumers idempotent so retries after a crash do not double-apply.

psql -c "SELECT count(*) FROM outbox WHERE published_at IS NULL;"

Expected output: At-least-once delivery is safe.

Troubleshooting

Problem

Relay publishes but crashes before marking

Solution

Accept duplicates and rely on consumer idempotency; deduplicate by event id.

Problem

Outbox table grows unbounded

Solution

Add a retention job that deletes rows published more than a defined window ago.

Limitations

The outbox provides at-least-once delivery, not exactly-once; consumers must tolerate duplicates.
Polling adds latency and load; change-data-capture can replace polling but adds operational complexity.

Context

Guarantee at-least-once event publication without a two-phase commit between the database and a broker. A service commits a state change then tries to publish, but the publish fails and the event is lost, so downstream state never catches up.

Steps

Insert the event into an outbox table inside the same transaction as the state change. Expected: The event and the state change commit together.
Run a relay that polls unpublished outbox rows, publishes them, and marks them sent. Expected: Events reach the broker without a distributed transaction.
Delete or archive published rows on a schedule to bound table growth. Expected: The outbox stays small.
Make consumers idempotent so retries after a crash do not double-apply. Expected: At-least-once delivery is safe.

BEGIN;
  UPDATE orders SET status = 'shipped' WHERE id = $1;
  INSERT INTO outbox (aggregate_id, type, payload)
  VALUES ($1, 'order.shipped', $2);
COMMIT;

Verify

The event and the state change commit together.
Events reach the broker without a distributed transaction.
The outbox stays small.
At-least-once delivery is safe.

Limits

The outbox provides at-least-once delivery, not exactly-once; consumers must tolerate duplicates.
Polling adds latency and load; change-data-capture can replace polling but adds operational complexity.