Rebuild Read Models by Replaying Events

Keep a projection version, consume events into a read model, and be able to replay from zero or from a checkpoint to rebuild after a bug.

~50 mincqrsevent-sourcingdata

Task

Make a CQRS read model rebuildable by replay so a projection bug can be fixed without touching the source of truth.

When to use this

A projection bug writes wrong values into the read model, and there is no way to correct it short of restoring a database backup.

Prerequisites

An append-only event log with retained history
A read store the projection writes

Steps

1. Track the projection version and the last processed event position.

Expected output: The projection knows how far it has consumed.

2. Process events idempotently so replays overwrite rather than accumulate.

Expected output: Replaying an event twice yields the same state.

3. Support rebuilding into a new read store without affecting the live one.

Expected output: A bugfix can be validated before cutover.

4. Cut over atomically once the rebuilt projection matches expected totals.

psql -c "SELECT id, type FROM events WHERE id > $1 ORDER BY id LIMIT 1000;"

Expected output: The new read model serves only after verification.

Troubleshooting

Problem

Replay is slow

Solution

Batch events and parallelize across partitions where ordering allows, then verify totals before cutover.

Problem

Events too old have been compacted

Solution

Keep enough history or snapshot periodically so a replay never needs events that no longer exist.

Limitations

Replay is only possible if events are retained; aggressive compaction removes the ability to rebuild.
Rebuilding is offline work that needs its own capacity and verification before it can serve traffic.

Context

Make a CQRS read model rebuildable by replay so a projection bug can be fixed without touching the source of truth. A projection bug writes wrong values into the read model, and there is no way to correct it short of restoring a database backup.

Steps

Track the projection version and the last processed event position. Expected: The projection knows how far it has consumed.
Process events idempotently so replays overwrite rather than accumulate. Expected: Replaying an event twice yields the same state.
Support rebuilding into a new read store without affecting the live one. Expected: A bugfix can be validated before cutover.
Cut over atomically once the rebuilt projection matches expected totals. Expected: The new read model serves only after verification.

SELECT id, type, payload
FROM events
WHERE id > $1
ORDER BY id
LIMIT 1000;

Verify

The projection knows how far it has consumed.
Replaying an event twice yields the same state.
A bugfix can be validated before cutover.
The new read model serves only after verification.

Limits

Replay is only possible if events are retained; aggressive compaction removes the ability to rebuild.
Rebuilding is offline work that needs its own capacity and verification before it can serve traffic.