Little's Law
Little's Law states that average concurrency equals throughput times average latency — L = lambda W — a three-variable identity that turns queue observations into capacity estimates.
Definition
Little's Law relates the average number of items in a system (L), the arrival or throughput rate (lambda), and the average time each item spends (W): L = lambda times W. It holds for any stable system regardless of distribution, making it the universal bridge between observed queue depth and required capacity.
Context and scope
Used for capacity planning (threads from target latency and throughput), queue sizing, connection pool sizing, and diagnosing which variable moved when latency rises.
Common misconceptions
- It assumes exponential distributions — it holds for any arrival and service distributions in a stable system; stability is the requirement
- It gives exact numbers for complex systems — it gives exact relationships between averages; detailed prediction still needs distribution modeling
- More threads always increase throughput — the law shows concurrency rising with latency: when saturation raises W, L grows without any throughput gain
Anti-patterns
- Sizing pools from peak throughput without accounting for the latency spike under load, undersizing concurrency
- Reading queue depth without rate context and declaring a system overloaded when the depth matches a healthy lambda W product
- Applying the law to an unstable system — queues growing unbounded — where the identity no longer binds the variables
Related concepts
Authoritative source
What it is
Three numbers constrain each other everywhere: how many items are inside, how fast they arrive, how long they stay. Change one and another must follow. The identity needs almost no assumptions, which is why it appears in every capacity conversation.
How it works
Observe requests in flight (L) and service latency (W): throughput capacity is L over W. Or invert: to sustain lambda at target latency W, the system must hold lambda times W concurrent requests. A queue growing at fixed lambda means W is rising — the system slowed rather than traffic spiking.
When to apply
Thread and connection pool sizing, autoscaling thresholds from concurrency metrics, and triage: separate traffic growth (lambda up) from degradation (W up) by checking which variable moved. It also sanity-checks queue dashboards — a depth that matches lambda times W is healthy, not alarming.
Pitfalls
Averages hide burst overload: an identity on means says nothing about p99. Stability is a precondition — a saturating system has unbounded L with lambda capped. Response time versus time-in-system must be defined consistently; mixing queue wait into W and not into L breaks the accounting.