Diagnose Container CPU Throttling
Read cgroup throttling counters to prove whether a container's latency comes from CPU limits rather than slow code.
Task
Check CPU throttling metrics to decide whether to raise limits or optimise code.
When to use this
A service shows latency spikes under load, and the team suspects code but the container may be CPU-throttled.
Prerequisites
- A Linux container with cgroup v2
- Access to container metrics
- A load profile
Steps
1. Read the throttling counters from the cgroup.
cat /sys/fs/cgroup/cpu.stat
# look at nr_throttled and throttled_usecExpected output: A high nr_throttled with rising throttled_usec confirms CPU throttling.
2. Compare requested CPU with actual usage.
kubectl top pod app --containersExpected output: Usage near the limit correlates with the throttling counters.
Troubleshooting
Problem
Throttled despite low average usage
Solution
Bursts hitting the quota in a 100 ms period throttle even when averages look fine; check per-period usage.
Problem
No metrics access
Solution
Expose cgroup counters through your metrics pipeline before incidents.
Limitations
- Counters are cumulative; compare deltas over time.
- Does not replace profiling for algorithmic issues.
- Raising limits costs capacity.
What this recipe does
Check CPU throttling metrics to decide whether to raise limits or optimise code.
A service shows latency spikes under load, and the team suspects code but the container may be CPU-throttled.
Prerequisites
- A Linux container with cgroup v2
- Access to container metrics
- A load profile
Procedure
- Read the throttling counters from the cgroup.
cat /sys/fs/cgroup/cpu.stat
# look at nr_throttled and throttled_usecExpected result: A high nr_throttled with rising throttled_usec confirms CPU throttling.
- Compare requested CPU with actual usage.
kubectl top pod app --containersExpected result: Usage near the limit correlates with the throttling counters.
How to confirm it worked
Run the commands above and confirm every expected result in order. If a step does not produce the stated output, use the troubleshooting notes before continuing. A passing run shows either raising the limit removes throttling and latency, or profiling shows the real bottleneck is code.
Troubleshooting
Throttled despite low average usage — Bursts hitting the quota in a 100 ms period throttle even when averages look fine; check per-period usage.
No metrics access — Expose cgroup counters through your metrics pipeline before incidents.
Limitations
- Counters are cumulative; compare deltas over time.
- Does not replace profiling for algorithmic issues.
- Raising limits costs capacity.