Request Coalescing
Request coalescing merges concurrent identical requests into one upstream fetch, with the first caller computing and the rest awaiting the shared result, at proxy or runtime level.
Definition
Request coalescing deduplicates in-flight identical requests: when a request for key K is already in progress, subsequent requests for K wait on the same computation instead of starting their own. Unlike single-flight inside one process, proxy-level coalescing — nginx proxy_cache_lock, CDN request collapsing — merges across clients.
Context and scope
Applied at CDNs collapsing edge requests, at API gateways deduplicating concurrent fan-outs, and in-process as single-flight around expensive computation.
Common misconceptions
- Coalescing batches requests over time — it merges concurrent requests only; time-window batching is a different technique
- Errors are naturally coalesced safely — one failing fetch fans the same error to all waiters unless the design varies retry policy per waiter
- It requires distributed locks — within a process a promise map suffices; cross-process designs use locks or lease TTLs
Anti-patterns
- Coalescing requests with per-user context under one key and returning one user data to another
- Holding waiters unboundedly when the leader stalls — the coalescer needs timeout and leader eviction
- Coalescing non-idempotent operations because they arrived concurrently
Related concepts
Related examples
Authoritative source
https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_cache_lock
What it is
Cache misses arrive in bursts. Without merging, each waiting request fetches independently; the origin executes the same query N times. Coalescing elects one leader to fetch while followers subscribe to the leader outcome, collapsing N identical upstream calls into one.
How it works
In-process: a map from key to in-flight promise; arrivals join the existing promise, and completion removes the entry. Proxy-level: cache lock semantics serialize fill attempts per key, with a timeout releasing waiters to fetch themselves if the leader is stuck. Cross-process coalescing leans on distributed locks or lease-based election.
When to use it
Hot keys after expiry or deploy, expensive aggregations, and cache fill paths anywhere. It complements TTL jitter: jitter smears expirations, coalescing absorbs what aligns anyway.
Pitfalls
The key must capture all request semantics — coalescing by URL while ignoring auth or vary headers serves wrong responses. Leader stall needs bounded wait plus fallback fetch. Error fan-out decides whether all waiters receive the failure or a fresh leader retries immediately, recreating the burst.