Thread Pool Sizing
Thread pool sizing matches worker count to the work shape: CPU-bound pools want core-count threads, while I/O-bound pools size by concurrency demand and downstream tolerance, not CPU.
Definition
Pool sizing is derived from the work profile. For CPU-bound tasks, optimal threads approach core count — more adds context switching without throughput. For I/O-bound tasks, threads mostly wait, so the pool sizes to desired concurrency and downstream capacity. Little-style formulas estimate threads from wait ratio and target utilization.
Context and scope
Tuned for executors, web container worker pools, and scheduled jobs. Sits upstream of connection pool sizing and downstream queue policy.
Common misconceptions
- More threads mean more throughput — past core count on CPU-bound work, throughput drops while latency rises
- Pool sizing depends only on the machine — it depends on the work mix, downstream limits, and memory per thread
- An unbounded queue is the safe default — it converts overload into unbounded latency and hidden memory growth
Anti-patterns
- Configuring hundreds of CPU-bound workers on an eight-core box and thrashing the scheduler
- Sharing one fixed pool across fast CPU tasks and slow remote calls, starving the CPU work behind waits
- Sizing pools without a rejection policy, so bursts queue invisibly until timeouts cascade
Related concepts
Related examples
Authoritative source
What it is
A pool converts arrival streams into parallel execution. The knob — worker count — trades context-switch overhead and memory against concurrency. The right value depends on whether workers compute or wait.
How it works
For CPU-bound work, threads = cores times target utilization. For I/O-bound work, estimate threads = cores times (1 + wait time over service time), then cap by downstream tolerance. The queue policy decides overload behavior: bounded queues with rejection turn bursts into visible errors; unbounded queues hide them as latency.
When to resize
Profile the wait-compute ratio before guessing. Separate pools for different work shapes — one for CPU-bound, one for remote calls — so waits cannot starve compute. Load-test at the intended concurrency and watch p99 plus queue depth.
Pitfalls
Fixed pools tuned for average load fail at peaks; consider adaptive sizing or staged queues. Thread stack memory caps pool sizes on constrained runtimes. Sizing one pool in isolation destabilizes the chain: workers waiting on a saturated connection pool look like a thread shortage.