I/O-Bound Throughput Workflow
Increase throughput for I/O-bound work by raising concurrency, batching, and using async I/O instead of adding CPU.
Objective
Improve throughput of I/O-bound workloads by addressing blocking and concurrency.
Environment
- Metrics for I/O wait
- A profiler or tracer
- A workload with I/O calls
Steps
1. Confirm the bottleneck is I/O
Check whether threads spend most time waiting on I/O rather than computing.
Expected state: The bottleneck is identified as I/O, not CPU.
2. Introduce async or more concurrency
Replace blocking calls with async or raise concurrency within safe limits.
Expected state: I/O waits overlap instead of serialising.
3. Batch small operations
Combine many small I/O operations into fewer larger ones.
Expected state: Round trips and syscalls drop.
4. Verify throughput
Measure throughput under load and confirm it rises without saturating the dependency.
Expected state: Throughput improves with the dependency healthy.
Decision points
More threads or async I/O?
- Async where the runtime supports it
- More threads
Prefer async; thread-per-request scales poorly for high concurrency.
Batch or keep per-item calls?
- Batch where possible
- Per-item calls
Batch small operations to cut round-trip overhead.
Expected outcome
I/O-bound workloads achieve higher throughput through concurrency and batching rather than more CPU.
Verification
Confirm throughput rises, the I/O dependency is not saturated, and CPU stays within budget.
Cleanup
Document the concurrency settings and their safe upper bound.
Overview
Improve throughput of I/O-bound workloads by addressing blocking and concurrency.
This workflow is reusable across teams and environments; adapt the commands to your tooling while keeping the sequence and checkpoints intact.
Environment and prerequisites
- Metrics for I/O wait
- A profiler or tracer
- A workload with I/O calls
Procedure
1. Confirm the bottleneck is I/O
Check whether threads spend most time waiting on I/O rather than computing.
Expected state: The bottleneck is identified as I/O, not CPU.
2. Introduce async or more concurrency
Replace blocking calls with async or raise concurrency within safe limits.
Expected state: I/O waits overlap instead of serialising.
3. Batch small operations
Combine many small I/O operations into fewer larger ones.
Expected state: Round trips and syscalls drop.
4. Verify throughput
Measure throughput under load and confirm it rises without saturating the dependency.
Expected state: Throughput improves with the dependency healthy.
# measure I/O wait share
pidstat -p $(pgrep -f server) 1 5Verification
Confirm throughput rises, the I/O dependency is not saturated, and CPU stays within budget.
Decision points
More threads or async I/O?
- Async where the runtime supports it
- More threads
Recommendation: Prefer async; thread-per-request scales poorly for high concurrency.
Batch or keep per-item calls?
- Batch where possible
- Per-item calls
Recommendation: Batch small operations to cut round-trip overhead.
Common failures
- Adding CPU to I/O-bound work wastes resources.
- Unbounded concurrency saturates the dependency.
- Per-item calls multiply overhead.