Memory Pool Sizing
Memory pool sizing pre-allocates a fixed arena for a known allocation pattern, so the hot path allocates from the pool without entering the general allocator or fragmenting the heap.
Definition
Memory pool sizing is the practice of pre-allocating a contiguous arena for a class of allocations — per-request buffers, per-connection state — so that allocation on the hot path is a bump pointer within the pool, avoiding the general allocator overhead and the fragmentation that variable-size allocation produces.
Context and scope
Applied in network servers, game engines, and any system with a predictable allocation pattern where the general allocator is a measured bottleneck.
Common misconceptions
- A pool always reduces allocation cost — it reduces it for the pooled pattern; for patterns that do not fit the pool, it adds indirection and waste
- Pools eliminate fragmentation — they eliminate it within the pool; the pool itself is one allocation, and the arena can still fragment if the pool is variable-size
Anti-patterns
- Sizing the pool for the peak load and holding the memory forever, wasting RAM when the load drops
- Pooling allocations of many different sizes in one arena, fragmenting the arena the pool was meant to keep defragmented
Related concepts
Authoritative source
https://www.boost.org/doc/libs/release/libs/pool/doc/html/boost_pool/pool.html
What it is
Memory pool sizing is the practice of pre-allocating a contiguous arena for a class of allocations — per-request buffers, per-connection state — so that allocation on the hot path is a bump pointer within the pool, avoiding the general allocator overhead and the fragmentation that variable-size allocation produces. Applied in network servers, game engines, and any system with a predictable allocation pattern where the general allocator is a measured bottleneck. Performance is a distribution, not a number: the tail determines user experience, and the mean hides the outliers that cause abandonment.
How it works
Measurement targets percentiles at the workload’s actual concurrency, not a single sequential run. The fast path is kept fast by deferring work that is not on it — lazy initialization, coalescing writes, avoiding allocation in hot loops. Profiling finds the bottleneck; micro-optimization without a profile optimizes the wrong thing. Constant factors matter, but algorithmic complexity dominates as scale grows.
When to optimize and pitfalls
Optimize when a measurement shows a concrete cost — latency budget burn, compute spend, user-visible jank. The pitfall is optimizing for the benchmark rather than the workload: a synthetic test that does not reproduce the production access pattern produces a local maximum that hurts globally. Another is trading memory for speed without accounting for the GC or eviction cost the extra allocation introduces.