Add Mutation Testing to Critical Modules

Run Stryker on money and authorization code, set a mutation score threshold, and strengthen tests for the mutants that survive.

~40 mintestingqualityci

Task

Measure whether the test suite actually detects faults in critical business logic, not just whether tests execute.

When to use this

Coverage reports 92 percent on the pricing module, yet a mutant that flips a discount comparison survives because tests only assert the happy path.

Prerequisites

Node.js 22 with TypeScript
StrykerJS 8 and the Vitest runner
A small, fast test suite for the module under test

Steps

1. Install the Stryker core package and the Vitest runner.

pnpm add -D @stryker-mutator/core @stryker-mutator/vitest-runner

Expected output: Both packages appear in devDependencies and a stryker.conf.json can reference the vitest runner.

2. Scope mutation to the critical sources and set a score threshold that breaks the build.

{
  "$schema": "./node_modules/@stryker-mutator/core/schema/stryker-schema.json",
  "testRunner": "vitest",
  "mutate": ["src/pricing/**/*.ts", "!src/pricing/**/*.test.ts"],
  "thresholds": { "high": 85, "low": 70, "break": 70 },
  "reporters": ["html", "clear-text", "progress"]
}

Expected output: Only pricing sources are mutated, and the command exits non-zero when the score drops below 70.

3. Run mutation testing and read the surviving mutants before touching the tests.

pnpm stryker run

Expected output: A score such as 78.4 percent (142 killed, 39 survived) plus an HTML report under reports/mutation/.

4. Add boundary assertions for real survivors and document genuinely equivalent mutants.

// Stryker disable next-line ConditionalExpression: equivalent mutant, both branches return 0
const discount = total > 0 ? total * 0.1 : 0;

Expected output: The score rises after adding tests for the discount threshold; every ignore comment states why the mutant cannot change behaviour.

5. Gate pull requests with incremental mode so only changed code is mutated.

pnpm stryker run --incremental --incrementalFile reports/stryker-incremental.json

Expected output: Repeated runs reuse previous results and finish in minutes instead of an hour, while the full nightly run stays authoritative.

Troubleshooting

Problem

The first run takes hours

Solution

Restrict the mutate glob to money, auth, and state-machine code, enable incremental mode, and keep the exhaustive run in a nightly job.

Problem

Many survivors look impossible to kill

Solution

Some are equivalent mutants that cannot change behaviour. Do not chase 100 percent; document each ignore with a reason and focus on boundary and conditional mutants.

Problem

The runner picks up end-to-end tests and hangs

Solution

Point Stryker at a dedicated Vitest config that excludes e2e and integration suites, and keep unit tests for the mutated module fast.

Limitations

Mutation testing is slow; apply it to critical modules rather than the whole repository.
The score is a proxy for test strength, not a proof of correctness.
Equivalent mutants inflate survivor counts and cannot always be eliminated.

Coverage measures execution, not detection

A line covered by a test is a line that ran, not a line whose failure would be noticed. Mutation testing makes that distinction concrete: it modifies the code in small ways, such as flipping a comparison or removing a return, and checks whether any test fails. A mutant that survives means the suite executed the line but never asserted the behaviour that the line controls.

Start with the code that hurts

Mutating an entire repository is expensive and produces noise. Begin with pricing, permissions, tax, and state transitions, where a subtle inversion costs money or access. Exclude generated code, configuration, TypeScript type-only files, and tests themselves. A focused run finishes in minutes, which means it can run on every pull request instead of once a quarter.

{
  "testRunner": "vitest",
  "mutate": ["src/pricing/**/*.ts", "!src/pricing/**/*.test.ts"],
  "thresholds": { "high": 85, "low": 70, "break": 70 }
}

Read survivors as test requirements

Each surviving mutant points at a missing assertion. A flipped >= usually means no test covers the exact boundary. A removed await means no test checks ordering or rejection. Fix the tests, not the score. If a mutant truly cannot change behaviour, such as a branch that returns the same value on both paths, disable it with a comment explaining why; blind ignores hide real gaps.

Set the gate where it helps

A high threshold on day one will fail the build and tempt the team to disable the tool. Start by measuring, then set break slightly below the current score and ratchet it up as tests improve. Use incremental mode on pull requests and a full run nightly. Report the score trend next to coverage, because the two moving in opposite directions is a useful signal.

Combine with other signals

Mutation testing complements, but does not replace, integration tests, contract tests, and property-based tests. It says nothing about a missing database constraint or a wrong API schema. Use it to answer one narrow question: if this logic were wrong, would the suite notice? For the code that handles money and access, that question is worth asking continuously.