Retry with Exponential Backoff and Jitter

Retry transient failures a bounded number of times with exponential delay plus jitter, and only retry operations that are safe to repeat.

~30 minreliabilityresiliencehttp

Task

Add a retry policy that reduces load spikes and never retries non-idempotent failures blindly.

When to use this

Failures caused by a brief dependency blip trigger retries that all fire at once, worsening the outage instead of riding it out.

Prerequisites

A way to categorize errors as retryable or permanent
Idempotent operations or idempotency-key protection

Steps

1. Classify errors: retry timeouts, connection resets, and 429/503; never 400 or validation errors.

Expected output: Permanent failures fail fast.

2. Wait base \* 2^attempt with full jitter before the next try.

Expected output: Retries spread out instead of synchronizing.

3. Cap total attempts and total elapsed time.

curl --retry 3 --retry-all-errors https://api.example.com/flaky

Expected output: A stuck dependency cannot hang the caller indefinitely.

4. Respect a Retry-After header when the server provides one.

Expected output: Retries follow the server guidance on rate limits.

Troubleshooting

Problem

Retries amplify an outage

Solution

Add jitter and a circuit breaker; unjittered retries create synchronized thundering herds.

Problem

Duplicate side effects

Solution

Only retry idempotent calls, or attach an idempotency key so repeats are deduplicated.

Limitations

Retries increase load exactly when a dependency is unhealthy, so pair them with a circuit breaker.
They cannot fix a permanent error and merely delay failure if the classification is wrong.

Context

Add a retry policy that reduces load spikes and never retries non-idempotent failures blindly. Failures caused by a brief dependency blip trigger retries that all fire at once, worsening the outage instead of riding it out.

Steps

Classify errors: retry timeouts, connection resets, and 429/503; never 400 or validation errors. Expected: Permanent failures fail fast.
Wait base * 2^attempt with full jitter before the next try. Expected: Retries spread out instead of synchronizing.
Cap total attempts and total elapsed time. Expected: A stuck dependency cannot hang the caller indefinitely.
Respect a Retry-After header when the server provides one. Expected: Retries follow the server guidance on rate limits.

for (let attempt = 0; ; attempt++) {
  try { return await call(); }
  catch (e) {
    if (!isRetryable(e) || attempt >= 4) throw e;
    await sleep(Math.random() * 100 * 2 ** attempt);
  }
}

Verify

Permanent failures fail fast.
Retries spread out instead of synchronizing.
A stuck dependency cannot hang the caller indefinitely.
Retries follow the server guidance on rate limits.

Limits

Retries increase load exactly when a dependency is unhealthy, so pair them with a circuit breaker.
They cannot fix a permanent error and merely delay failure if the classification is wrong.