Gate on Coverage Diff Not Absolute Number
Compare coverage against the base branch so a PR that drops coverage by 2 percent fails even if the total is still above 80 percent.
Task
Add npx c8 report --compare-with ./base-coverage/lcov.info --threshold-diff -2 to the test suite so regressions are caught automatically on every pull request.
When to use this
A refactor passes existing tests but breaks npx c8 report --compare-with ./base-coverage/lcov.info --threshold-diff -2, shipping a silent regression to users.
Prerequisites
- A test runner configured (Vitest or Jest)
- Source code with testable units
Steps
1. Configure npx c8 report --compare-with ./base-coverage/lcov.info --threshold-diff -2 in the project configuration file with the recommended defaults.
npx c8 report --compare-with ./base-coverage/lcov.info --threshold-diff -2
Expected output: The configuration is accepted and npx c8 report --compare-with ./base-coverage/lcov.info --threshold-diff -2 runs without errors on the first invocation.
2. Run the check locally to verify npx c8 report --compare-with ./base-coverage/lcov.info --threshold-diff -2 produces the expected result before pushing.
pnpm test
Expected output: Local run matches CI behavior; no surprises after push.
3. Add the gate to the CI pipeline so npx c8 report --compare-with ./base-coverage/lcov.info --threshold-diff -2 blocks merge on failure.
- name: Run add coverage diff gate
run: pnpm test
Expected output: Pull requests with failing npx c8 report --compare-with ./base-coverage/lcov.info --threshold-diff -2 are blocked from merging.
Troubleshooting
Problem
The check passes locally but fails in CI
Solution
Pin dependency versions and the runtime to match CI exactly; reproduce with the same Docker image or act.
Problem
Configuration is rejected with a schema error
Solution
Check the tool version matches the config schema; older versions may not support newer fields.
Limitations
- The gate only runs on events that trigger the workflow; locally pushed commits bypass it.
- Configuration drift is possible if someone edits settings outside version control.
Cover the regression path
Start by measuring the problem. Compare coverage against the base branch so a PR that drops coverage by 2 percent fails even if the total is still above 80 percent. The most common mistake is adding the gate without first confirming it catches the issue you care about. Run npx c8 report –compare-with ./base-coverage/lcov.info –threshold-diff -2 against the current codebase and inspect the output before wiring it into CI.
npx c8 report --compare-with ./base-coverage/lcov.info --threshold-diff -2Keep tests maintainable
Once the gate is in place, make failures actionable. A failing npx c8 report –compare-with ./base-coverage/lcov.info –threshold-diff -2 should point to the exact file and line, not a generic error. Pin the tool version in the lockfile so a upstream release does not silently change behavior. Review the gate output in every pull request just like a test result.