← Recipes

阅读中文版本

Retry with Exponential Backoff and Full Jitter

Add full jitter to the backoff so retries spread across time and do not synchronize into a retry storm on recovery.

~15 minapiretrybackoff

Task

Implement const delay = Math.random() * Math.min(cap, base * 2 ** attempt); so the service degrades gracefully under failure instead of cascading timeouts to callers.

When to use this

A downstream service slows down and without const delay = Math.random() * Math.min(cap, base * 2 ** attempt); the upstream exhausts its thread pool in 12 seconds.

Prerequisites

  • A service with inter-service calls or external dependencies
  • A retry or circuit breaker library available

Steps

1. Configure const delay = Math.random() * Math.min(cap, base * 2 ** attempt); in the project configuration file with the recommended defaults.

const delay = Math.random() * Math.min(cap, base * 2 ** attempt);

Expected output: The configuration is accepted and const delay = Math.random() * Math.min(cap, base * 2 ** attempt); runs without errors on the first invocation.

2. Run the check locally to verify const delay = Math.random() * Math.min(cap, base * 2 ** attempt); produces the expected result before pushing.

pnpm test

Expected output: Local run matches CI behavior; no surprises after push.

3. Add the gate to the CI pipeline so const delay = Math.random() * Math.min(cap, base * 2 ** attempt); blocks merge on failure.

- name: Run add retry jitter exponential
  run: pnpm test

Expected output: Pull requests with failing const delay = Math.random() * Math.min(cap, base * 2 ** attempt); are blocked from merging.

Troubleshooting

Problem

The check passes locally but fails in CI

Solution

Pin dependency versions and the runtime to match CI exactly; reproduce with the same Docker image or act.

Problem

Configuration is rejected with a schema error

Solution

Check the tool version matches the config schema; older versions may not support newer fields.

Limitations

  • The gate only runs on events that trigger the workflow; locally pushed commits bypass it.
  • Configuration drift is possible if someone edits settings outside version control.

Isolate failure

Start by measuring the problem. Add full jitter to the backoff so retries spread across time and do not synchronize into a retry storm on recovery. The most common mistake is adding the gate without first confirming it catches the issue you care about. Run const delay = Math.random() * Math.min(cap, base * 2 ** attempt); against the current codebase and inspect the output before wiring it into CI.

const delay = Math.random() * Math.min(cap, base * 2 ** attempt);

Verify graceful degradation

Once the gate is in place, make failures actionable. A failing const delay = Math.random() * Math.min(cap, base * 2 ** attempt); should point to the exact file and line, not a generic error. Pin the tool version in the lockfile so a upstream release does not silently change behavior. Review the gate output in every pull request just like a test result.