SIMD Vectorization
SIMD vectorization applies one instruction to multiple data elements, so a loop that processes four floats per iteration does one quarter the iterations and one quarter the loop overhead.
Definition
SIMD vectorization is the optimization of rewriting a scalar loop to use single-instruction-multiple-data instructions that process a vector of elements per instruction, so that the iteration count and the per-element overhead drop by the vector width, applied by the compiler auto-vectorizer or by hand with intrinsics.
Context and scope
Applied in image processing, numerical kernels, string operations, and any data-parallel loop where the elements are independent and the vector width is a meaningful speedup.
Common misconceptions
- Auto-vectorization handles everything — it handles straightforward loops; reductions, conditionals, and dependencies often need restructuring or intrinsics
- SIMD always speeds up the loop — it speeds up the data-parallel part; the scalar setup and the tail handling can offset the gain on short loops
Anti-patterns
- Hand-vectorizing a loop the compiler already vectorizes, adding intrinsic maintenance for no gain
- Vectorizing a loop with a data dependency between iterations, producing wrong results at vector width
Related concepts
Authoritative source
What it is
SIMD vectorization is the optimization of rewriting a scalar loop to use single-instruction-multiple-data instructions that process a vector of elements per instruction, so that the iteration count and the per-element overhead drop by the vector width, applied by the compiler auto-vectorizer or by hand with intrinsics. Applied in image processing, numerical kernels, string operations, and any data-parallel loop where the elements are independent and the vector width is a meaningful speedup. Performance is a distribution, not a number: the tail determines user experience, and the mean hides the outliers that cause abandonment.
How it works
Measurement targets percentiles at the workload’s actual concurrency, not a single sequential run. The fast path is kept fast by deferring work that is not on it — lazy initialization, coalescing writes, avoiding allocation in hot loops. Profiling finds the bottleneck; micro-optimization without a profile optimizes the wrong thing. Constant factors matter, but algorithmic complexity dominates as scale grows.
When to optimize and pitfalls
Optimize when a measurement shows a concrete cost — latency budget burn, compute spend, user-visible jank. The pitfall is optimizing for the benchmark rather than the workload: a synthetic test that does not reproduce the production access pattern produces a local maximum that hurts globally. Another is trading memory for speed without accounting for the GC or eviction cost the extra allocation introduces.