Implement Stale-While-Revalidate Caching

Serve a cached response immediately while refreshing it in the background, and cap staleness so users get fast reads without permanently stale data.

~35 mincachingperformancereliability

Task

Add stale-while-revalidate semantics to an expensive read path with a bounded staleness window.

When to use this

Every read blocks on a slow origin, but caching with a fixed TTL means the first request after expiry is slow and risky.

Prerequisites

A cache store with atomic writes
A way to run background refreshes

Steps

1. Return the cached value if present and mark it for background refresh.

Expected output: Reads are served without waiting for the origin.

2. Refresh asynchronously and store the new value with a fresh timestamp.

curl -s -H "Cache-Control: max-age=60, stale-while-revalidate=300" https://api.example.com/profile

Expected output: The next read sees updated data.

3. Enforce a maximum stale age after which the request blocks on the origin.

Expected output: Data older than the cap is never served.

4. Single-flight the refresh so concurrent readers share one origin call.

Expected output: A burst of reads causes at most one refresh.

Troubleshooting

Problem

Thundering herd on expiry

Solution

Use an in-process lock or a distributed lock so only one reader refreshes.

Problem

Serving data that is too old

Solution

Store a fetchedAt timestamp and check the age against the hard cap on every read.

Limitations

Stale-while-revalidate trades consistency for latency, so it is wrong for data that must be exact on read.
It requires a background mechanism; without one it degrades into plain caching.

Context

Add stale-while-revalidate semantics to an expensive read path with a bounded staleness window. Every read blocks on a slow origin, but caching with a fixed TTL means the first request after expiry is slow and risky.

Steps

Return the cached value if present and mark it for background refresh. Expected: Reads are served without waiting for the origin.
Refresh asynchronously and store the new value with a fresh timestamp. Expected: The next read sees updated data.
Enforce a maximum stale age after which the request blocks on the origin. Expected: Data older than the cap is never served.
Single-flight the refresh so concurrent readers share one origin call. Expected: A burst of reads causes at most one refresh.

async function get(k) {
  const hit = await cache.get(k);
  if (hit && Date.now() - hit.at < HARD_CAP) { void refresh(k); return hit.value; }
  return load(k);
}

Verify

Reads are served without waiting for the origin.
The next read sees updated data.
Data older than the cap is never served.
A burst of reads causes at most one refresh.

Limits

Stale-while-revalidate trades consistency for latency, so it is wrong for data that must be exact on read.
It requires a background mechanism; without one it degrades into plain caching.