Single Flight
Single flight deduplicates concurrent calls for the same key so only one expensive operation runs while all callers share its result.
Definition
Single flight is a concurrency pattern that collapses multiple in-flight requests for the same key into one execution. The first caller starts the work; subsequent callers for the same key wait for that result instead of starting duplicate work. When the operation completes, all waiters receive the same value or error, and the in-flight entry is removed.
Context and scope
Single flight is useful for cache misses, expensive database reads, token refreshes, and metadata lookups under high concurrency. Without it, a cold cache with 1,000 concurrent requests can trigger 1,000 identical backend calls. It is a per-process optimization; across processes, a distributed lock or cache is needed.
Common misconceptions
- Single flight caches results — it deduplicates only concurrent calls; caching is a separate concern
- It eliminates all duplicate work — calls that arrive after completion start a new operation unless a cache is added
- Sharing errors is wrong — sharing the same failure with all waiters is usually correct and avoids a retry storm
Anti-patterns
- Using single flight for operations with side effects that must run per caller
- Keeping a failed in-flight entry forever instead of clearing it on completion
- Sharing mutable result objects that callers then modify
Related concepts
Related examples
Authoritative source
The thundering herd
A cache miss under load is a classic failure mode. Every request checks the cache, misses, and calls the database. The database sees a spike exactly when the cache is cold. Single flight ensures that only the first request performs the lookup; the rest wait for its result.
type Entry<T> = { promise: Promise<T> };
class SingleFlight {
private inflight = new Map<string, Entry<unknown>>();
async run<T>(key: string, operation: () => Promise<T>): Promise<T> {
const existing = this.inflight.get(key) as Entry<T> | undefined;
if (existing) return existing.promise;
const entry: Entry<T> = { promise: operation() };
this.inflight.set(key, entry as Entry<unknown>);
try {
return await entry.promise;
} finally {
this.inflight.delete(key);
}
}
}Scope and lifecycle
The key must identify the work precisely: a cache key, a user ID, or a token refresh identity. Remove the entry in finally so a failure does not poison future calls. If the operation can take longer than the caller’s deadline, support cancellation without cancelling the shared work for other waiters.
When to use it
Use single flight for read-through caches, DNS or metadata resolution, and token refresh. Compared to caching, single flight reduces concurrent duplicate work but does not serve future requests faster. Compared to a lock, it allows all callers to receive the result without serializing repeated executions.
Pitfalls
Side-effecting operations should not be deduplicated unless the side effect is idempotent. Returning the same mutable object to every caller invites accidental shared-state bugs. In multi-instance deployments, a per-process map still allows one call per instance; combine it with a distributed cache or lock when the backend cannot tolerate that fan-out.