Backpressure
Backpressure is a flow-control signal that tells a fast producer to slow down when a downstream consumer cannot keep up, preventing unbounded queues and memory growth.
Definition
Backpressure is the mechanism by which a consumer communicates its limited capacity to a producer. Instead of buffering without limit, the producer waits, drops work, or rejects new items according to an explicit policy. In streams, backpressure is expressed by the consumer not reading and the producer receiving a false return value or a drained event. In queues, it is expressed as a bounded capacity with blocking or rejection.
Context and scope
Backpressure matters whenever production and consumption rates can diverge, such as network uploads, log pipelines, database writes, and UI event processing. Without it, the fastest component determines memory usage and the slowest component only delays an eventual out-of-memory failure.
Common misconceptions
- Adding an unbounded queue fixes slow consumers — it only moves the failure from latency to memory exhaustion
- Backpressure means dropping data — it can also mean pausing the producer or applying a bounded buffer
- Node.js streams always apply backpressure automatically — producers must respect write() return values and drain events
Anti-patterns
- Pushing every event into an in-memory array without a size limit
- Ignoring the boolean return value of a stream write and continuing to write
- Using backpressure as a substitute for capacity planning instead of measuring consumer throughput
Related concepts
Related examples
Authoritative source
Why unbounded buffering fails
When a producer generates work faster than a consumer processes it, the difference has to go somewhere. An unbounded queue accepts it into memory. Latency grows, then memory grows, and eventually the process crashes. Backpressure makes the mismatch visible at the producer instead of hiding it in a queue.
Backpressure in streams
Node.js streams signal backpressure through write() returning false and emitting drain when the internal buffer empties. A correct producer waits for drain before writing more:
import { createReadStream, createWriteStream } from 'node:fs';
import { pipeline } from 'node:stream/promises';
await pipeline(
createReadStream('large-input.log'),
createWriteStream('copy.log'),
);pipeline() propagates errors and respects backpressure across stages. Manually calling write() in a loop without checking its return value can buffer the whole file in memory.
In queues and services
A bounded queue makes the policy explicit: wait, reject, or drop. The example below waits for capacity before accepting a new item:
class BoundedQueue {
constructor(capacity) {
this.capacity = capacity;
this.items = [];
this.waiters = [];
}
async push(item) {
while (this.items.length >= this.capacity) {
await new Promise((resolve) => this.waiters.push(resolve));
}
this.items.push(item);
}
}When to use it
Use backpressure at every boundary where producer and consumer rates can diverge: HTTP uploads, database batches, log shipping, and UI event queues. Compared to a rate limiter, backpressure is a per-consumer capacity signal, while a rate limiter is a shared admission policy. They compose: a rate limiter protects the service, and backpressure protects memory inside the service.
Pitfalls
Backpressure without a timeout can deadlock if a consumer never progresses. Pair bounded waits with deadlines and metrics. Also measure queue depth and consumer latency; backpressure is a safety mechanism, not a substitute for adequate capacity.