Cache Stampede Prevention Workflow
Prevent cache stampede on key expiry by adding jittered TTLs, early refresh, and a single-flight loader so one miss does not flood the backend.
Objective
Ensure that a cache key expiry does not cause a stampede of concurrent backend loads, by spreading expiries and serializing the refresh.
Environment
- A cache with TTL-based expiry
- A backend that is expensive to call
- A traffic pattern with concurrent reads of the same key
Steps
1. Add TTL jitter
Set the TTL to a base value plus a random jitter, so keys do not all expire at the same instant.
Expected state: Expiries are spread across a jitter window, not aligned.
2. Add early refresh
Refresh the key before expiry if it is being read, so the refresh happens under normal load not a miss storm.
Expected state: The key is refreshed before it expires, avoiding a synchronous miss.
3. Add single-flight loader
Deduplicate concurrent loads for the same key so only one backend call happens, and the rest wait for its result.
Expected state: One miss causes one backend call, not N.
4. Verify under load
Run a load test that reads the key concurrently at expiry and assert the backend sees one call, not N.
Expected state: The stampede is prevented: one call at expiry.
Decision points
Jitter window size?
- Small (10% of TTL)
- Large (50% of TTL)
Small, enough to spread expiries without making the effective TTL unpredictable.
Early refresh threshold?
- 80% of TTL
- 95% of TTL
80%, so the refresh happens under normal load with time to complete before expiry.
Expected outcome
A cache key expiry causes one backend load, not a stampede, and the key is refreshed before expiry under normal load.
Verification
Run a load test of 100 concurrent readers at the key expiry and confirm the backend sees exactly 1 load, and the cache serves the rest.
Cleanup
Keep the jitter, early refresh, and single-flight; remove the load-test artifacts.
Why keys stampede
A cache key with a fixed TTL expires at one instant, and every concurrent reader at that instant misses and loads the backend simultaneously. The backend sees a burst of N calls for one key, which is a stampede.
Add TTL jitter
Spread the expiries by adding a random jitter to the TTL, so keys expire across a window instead of at one instant. A jitter of 10% of the TTL is enough to spread a population of keys without making the effective TTL unpredictable.
TTL = BASE + random(0, BASE * 0.1)Add early refresh
Refresh the key before it expires if it is being read, so the refresh happens under normal load, not a miss storm. A threshold of 80% of the TTL gives the refresh time to complete before expiry.
if (cache.ttl(key) < TTL * 0.2) refresh(key); // refresh when 20% of TTL remainsAdd single-flight loader
Deduplicate concurrent loads for the same key. The first miss starts the load and stores the promise; subsequent misses await the same promise, so the backend sees one call.
if (!inflight.has(key)) inflight.set(key, load(key));
return inflight.get(key);Verification
A load test of 100 concurrent readers at the key expiry sees exactly 1 backend load. The jitter spreads expiries, the early refresh avoids the synchronous miss, and the single-flight serializes the concurrent load.