Hedged Requests
Hedged requests issue a duplicate request after a delay threshold to another replica and take the first response, cutting tail latency at the cost of duplicate load.
Definition
Request hedging sends the primary request, then — if it exceeds a delay percentile — fires a hedge copy to a different replica, taking whichever finishes first and canceling the loser. It converts variance-dominated tails into races where the fastest replica wins, at the cost of extra load that itself worsens queues if overused.
Context and scope
Used in storage systems (Dynamo-style), frontend-to-backend fan-outs, and DNS resolution. Tuned by hedge delay relative to the latency distribution and by hedged fraction limits.
Common misconceptions
- Hedging always reduces latency — hedging too early doubles load for near-zero gain, and the added queueing can raise the tail it meant to cut
- Hedging requires idempotency everywhere — the hedge must be safe to execute twice: reads are natural; writes need idempotency keys
- Only p99 matters in choosing the delay — delay should sit near the point where waiting longer is worse than racing, typically p95 of the target distribution
Anti-patterns
- Firing hedges immediately rather than after a delay and doubling backend traffic at every layer
- Hedging against the same overloaded replica pool and amplifying the very queue causing the tail
- Hedging non-idempotent operations and occasionally executing them twice
Related concepts
Related examples
Authoritative source
What it is
Tail latency is often variance, not uniform slowness: one replica hit a bad event. Hedging races replicas against each other so a single straggler no longer defines the response time.
How it works
Send the request; arm a timer at a chosen percentile of typical latency; on expiry, send a copy to a different replica; first response wins, and the straggler response is discarded or its cancellation best-effort. Bounded hedging caps the fraction of requests that may hedge so the extra load stays a minority.
When to use it
Reads against replicated storage, cache lookups across regions, and latency-critical fan-outs where a straggler ruins the whole page. Combine with per-request deadline propagation so hedges respect the caller budget.
Pitfalls
Hedge load lands on the same fleet: a 10 percent hedge fraction raises backend load 10 percent, which raises queues, which raises tails — monitor the loop. Cancellation must actually stop work, or hedges burn CPU invisibly. Overlapping hedges with retries requires care to avoid exponential request trees during outages.