Test Impact Analysis
Test impact analysis selects the subset of tests affected by a code change via dependency mapping, cutting CI time from all-tests to relevant-tests without abandoning safety.
Definition
Test impact analysis maps tests to the production code they exercise — statically or through runtime coverage traces — and runs only tests whose footprint intersects the change. Correct selection requires conservative mapping: unknown or unmapped dependencies trigger running more, not fewer, tests. It trades compute for latency on every pull request.
Context and scope
Deployed as selective test runners in monorepo CI, remote execution caches (Bazel, Turborepo), and IDE fast-feedback loops.
Common misconceptions
- Skipping tests always risks missing bugs — conservative impact maps run every test that might be affected; the risk is bounded by mapping quality, not eliminated
- Static dependency graphs are sufficient — dynamic features — reflection, DI wiring, plugin loading — defeat static analysis and require runtime traces
- Impact analysis only helps huge monorepos — any suite over minutes benefits; the mapping pays for itself once CI queue time hurts developers
Anti-patterns
- Trimming the affected set until it is fast and shipping regressions the trimmed tests would have caught
- Trusting a stale coverage map after refactors moved code between modules
- Running selective suites on PRs but never running the full suite, letting slow rot accumulate
Related concepts
Related examples
Authoritative source
What it is
CI runs everything to catch anything. Impact analysis runs the tests that could catch this change. The difference is the mapping between code and the tests that observe it.
How it works
Static analysis derives module dependency graphs; runtime coverage records which tests executed which files. On a diff, the runner intersects changed files with test footprints and schedules the union — plus conservative expansion for reflection, generics, and generated code. Caches key on input hashes so unchanged scopes skip entirely.
When to adopt
When CI queue time dominates developer feedback loops, in monorepos where most tests are unaffected by most changes, and for pre-merge fast paths paired with scheduled full runs. Startup cost is real: the mapping infrastructure earns its keep only above a suite size where full runs no longer fit the merge window.
Pitfalls
Mappings rot as code moves; without periodic full-run validation, silent staleness becomes missed regressions. Cross-cutting changes — dependency upgrades, config, runtime versions — must invalidate broad scopes. A selective runner without a periodic whole-suite backstop optimizes latency while accumulating blind spots.