阅读中文版本

Thundering Herd — Cache Stampede on Expiry

Fix cache stampedes by using single-flight (request coalescing), jittered TTLs, and stale-while-revalidate so expiry does not overload the origin.

redis-cli INFO stats | grep expired_keys; # correlate with origin latency

grep -c "SELECT.*popular" app.log

Symptoms

The error “Thundering Herd — Cache Stampede on Expiry” points at a specific failure mode. This guide documents the symptoms you can observe, the commands that isolate the cause, the root cause, the fix, and the checks that prove the fix works.

Observed symptoms include:

• The origin spikes every time a popular cache entry expires
• Latency surges in bursts at regular intervals
• All instances recompute the same value simultaneously

Reproduce and Diagnose

Reproduce the failure in isolation before changing any code. Work through the checks below in order; each one narrows the search space.

Step 1: Observe synchronized expiry spikes

redis-cli INFO stats | grep expired_keys; # correlate with origin latency

Bursts of expiries aligned with origin latency spikes indicate a stampede.

Step 2: Count concurrent origin calls for one key

grep -c "SELECT.*popular" app.log

Many identical concurrent queries confirm a lack of coalescing.

Likely contributors, in order of frequency:

• A popular key expires and every request recomputes it at once
• All instances use the same TTL so expiries align
• There is no request coalescing, so N concurrent misses trigger N origin calls

Root Cause

When a hot entry expires, all concurrent requests see a miss and independently recompute it. Without coalescing or staggered expiry, the origin receives a synchronized burst far beyond steady load.

The Fix

Coalesce misses with single-flight so only one request recomputes per key, add jitter to TTLs to spread expiries, and serve stale content while refreshing in the background.

Watch out: Serving stale data must be acceptable for the use case; document the tolerance window per key.

import { setTimeout as sleep } from 'node:timers/promises';

const inflight = new Map(); // single-flight per key

async function getWithCoalescing(key, compute, ttlMs = 60000) {
  const cached = await cache.get(key);
  if (cached) return cached;
  if (inflight.has(key)) return inflight.get(key);

  const p = (async () => {
    const value = await compute();
    const jitter = Math.floor(Math.random() * 0.1 * ttlMs); // spread expiries
    await cache.set(key, value, ttlMs + jitter);
    return value;
  })().finally(() => inflight.delete(key));

  inflight.set(key, p);
  return p;
}

Verify and Prevent

Trigger expiry under load and confirm only one origin recompute occurs while others wait briefly; verify origin QPS stays within budget.

Once verified, add a regression test that reproduces the original symptom so the failure cannot silently return.

This class of failure is usually environmental rather than pure logic, so capture the exact command, the environment variables, and the working directory in your bug report. Reproduce it in the smallest possible script, then widen the scope only once the minimal case is understood. A short written checklist for this failure mode saves hours the next time it appears.