← Recipes

Coordinate a Saga with Compensating Actions

Break a multi-service operation into local steps, define a compensation for each step, and run compensations in reverse order when a later step fails.

~50 mindistributed-systemstransactionsarchitecture

Task

Implement a saga so a distributed operation either completes or is rolled back through explicit compensations.

When to use this

A multi-step purchase reserves inventory and charges the card, then shipping fails, leaving the customer charged with no order and no automatic recovery.

Prerequisites

  • A workflow or state machine to persist saga progress
  • A compensating action for each forward step

Steps

1. Model the operation as ordered local steps, each with a forward action and a compensation.

Expected output: Every successful step has a defined undo.

2. Persist progress after each step so a crash resumes from the last completed step.

Expected output: The saga is durable across restarts.

3. On failure, run compensations in reverse order for the steps that succeeded.

Expected output: Earlier effects are undone.

4. Make compensations idempotent and retryable.

curl -X POST https://api.example.com/orders/$ID/cancel

Expected output: A repeated compensation does not double-refund.

Troubleshooting

Problem

Compensation itself fails

Solution

Retry with backoff and alert; a saga that cannot compensate needs manual intervention and a clear runbook.

Problem

Saga state lost on crash

Solution

Persist step state before acknowledging each step so resumption is deterministic.

Limitations

  • Compensations are semantic undo, not true rollback; a refund is not the same as never charging.
  • Sagas add latency and state complexity and are unnecessary when a single transaction suffices.

Context

Implement a saga so a distributed operation either completes or is rolled back through explicit compensations. A multi-step purchase reserves inventory and charges the card, then shipping fails, leaving the customer charged with no order and no automatic recovery.

Steps

  1. Model the operation as ordered local steps, each with a forward action and a compensation. Expected: Every successful step has a defined undo.
  2. Persist progress after each step so a crash resumes from the last completed step. Expected: The saga is durable across restarts.
  3. On failure, run compensations in reverse order for the steps that succeeded. Expected: Earlier effects are undone.
  4. Make compensations idempotent and retryable. Expected: A repeated compensation does not double-refund.
for (const step of done.reverse()) {
  await retry(() => step.undo());
}

Verify

  • Every successful step has a defined undo.
  • The saga is durable across restarts.
  • Earlier effects are undone.
  • A repeated compensation does not double-refund.

Limits

  • Compensations are semantic undo, not true rollback; a refund is not the same as never charging.
  • Sagas add latency and state complexity and are unnecessary when a single transaction suffices.