Cache Stampede

A cache stampede happens when a hot key expires and many concurrent requests miss it at once, sending a burst of identical load to the origin.

performance

Definition

A cache stampede is a cache miss amplification pattern. A popular key expires or is invalidated, and concurrent requests that would normally be served from cache all miss at the same time. Each request recomputes or fetches the same value, so the origin receives a burst of duplicate work. The result can be a latency spike, an origin overload, or a second failure that keeps the key uncached.

Context and scope

Stampedes are common for expensive queries, rendered fragments, configuration snapshots, and API responses with a single hot key. They differ from a thundering herd in scope: a stampede is usually one key, while a herd can be many clients or keys synchronizing at once.

Common misconceptions

A longer TTL prevents stampedes — it reduces frequency but the burst still happens when the key finally expires
A cache lock is always required — request coalescing or stale-while-revalidate can solve it without a distributed lock
Negative caching makes the problem worse — caching a not-found result briefly can prevent repeated origin hits

Anti-patterns

Recomputing a hot key independently in every concurrent request
Invalidating a key and immediately forcing every reader to refresh synchronously
Serving stale data forever without a refresh path or freshness signal

Related concepts

Related examples

Authoritative source

https://en.wikipedia.org/wiki/Cache_stampede

One miss, many recomputations

Consider a homepage fragment cached for 60 seconds and served 5,000 times per second. At expiry, every request that arrives before the first recomputation finishes sees a miss. If recomputation takes 300 milliseconds, hundreds of requests can start the same query. The origin is now handling a load spike caused entirely by the cache doing its job.

The fix is to ensure that only one caller recomputes a missing key while the others wait for that result, serve a stale value, or receive a short retry signal. Those choices have different freshness and latency trade-offs.

Coalesce concurrent misses

Single flight groups concurrent calls for the same key into one in-flight operation:

type Entry<T> = { promise: Promise<T> };

export function createCoalescer<T>() {
  const inFlight = new Map<string, Entry<T>>();

  return function load(key: string, compute: () => Promise<T>): Promise<T> {
    const existing = inFlight.get(key);
    if (existing) return existing.promise;

    const promise = compute().finally(() => inFlight.delete(key));
    inFlight.set(key, { promise });
    return promise;
  };
}

Every caller receives the same promise and the same result or error. Remove the entry in finally so a later miss can retry. In a single process this is enough; across processes, use a distributed lock or a short-lived lease with a timeout, and make sure the lock holder cannot be stuck forever.

Serve stale while refreshing

When the product can tolerate slightly stale data, return the cached value and refresh it in the background. The first request after expiry triggers a refresh but does not wait for it. Subsequent requests continue to receive the old value until the refresh completes. Add a freshness header or a short client TTL so downstream caches do not treat stale data as fresh indefinitely.

TTL jitter reduces the chance that many keys expire together. Instead of a fixed 60 seconds, use 60 seconds plus a random offset. This does not solve a single hot key, but it prevents a cache-wide synchronization.

Protect the origin

Even with coalescing, a cold cache after a deploy or restart can send many distinct keys to the origin. Add a concurrency limit at the origin boundary, warm critical keys before taking traffic, and use negative caching for repeated not-found lookups. If the origin fails, do not delete the key and retry immediately; back off and serve stale data if policy allows.

Measure cache hit ratio, miss latency, origin QPS during misses, and the number of coalesced requests. A stampede is visible as a sharp miss-rate spike combined with duplicate origin queries. Test it by expiring a hot key under load and verifying that the origin receives one request rather than hundreds.

When to use it

Use single-flight coalescing when every caller needs a fresh value and duplicate work is expensive. Use stale-while-revalidate when the product tolerates bounded staleness and latency matters more than immediate freshness. The advantage of stale-while-revalidate is that no caller waits for the origin; the drawback is that a bug may serve old data for longer than expected. Compared to a distributed lock, request coalescing is simpler but only covers one process; cross-process protection needs a lease or a coordinated cache library.