Verify Test Environment Parity with Production
Compare the test and production dependency versions and config so a bug that only reproduces on a version skew is caught early.
Task
Add diff <(jq '.dependencies' test-package.json) <(jq '.dependencies' prod-package.json) to the test suite so regressions are caught automatically on every pull request.
When to use this
A refactor passes existing tests but breaks diff <(jq '.dependencies' test-package.json) <(jq '.dependencies' prod-package.json), shipping a silent regression to users.
Prerequisites
- A test runner configured (Vitest or Jest)
- Source code with testable units
Steps
1. Configure diff <(jq '.dependencies' test-package.json) <(jq '.dependencies' prod-package.json) in the project configuration file with the recommended defaults.
diff <(jq '.dependencies' test-package.json) <(jq '.dependencies' prod-package.json)
Expected output: The configuration is accepted and diff <(jq '.dependencies' test-package.json) <(jq '.dependencies' prod-package.json) runs without errors on the first invocation.
2. Run the check locally to verify diff <(jq '.dependencies' test-package.json) <(jq '.dependencies' prod-package.json) produces the expected result before pushing.
pnpm test
Expected output: Local run matches CI behavior; no surprises after push.
3. Add the gate to the CI pipeline so diff <(jq '.dependencies' test-package.json) <(jq '.dependencies' prod-package.json) blocks merge on failure.
- name: Run add test environment parity
run: pnpm test
Expected output: Pull requests with failing diff <(jq '.dependencies' test-package.json) <(jq '.dependencies' prod-package.json) are blocked from merging.
Troubleshooting
Problem
The check passes locally but fails in CI
Solution
Pin dependency versions and the runtime to match CI exactly; reproduce with the same Docker image or act.
Problem
Configuration is rejected with a schema error
Solution
Check the tool version matches the config schema; older versions may not support newer fields.
Limitations
- The gate only runs on events that trigger the workflow; locally pushed commits bypass it.
- Configuration drift is possible if someone edits settings outside version control.
Cover the regression path
Start by measuring the problem. Compare the test and production dependency versions and config so a bug that only reproduces on a version skew is caught early. The most common mistake is adding the gate without first confirming it catches the issue you care about. Run diff <(jq ‘.dependencies’ test-package.json) <(jq ‘.dependencies’ prod-package.json) against the current codebase and inspect the output before wiring it into CI.
diff <(jq '.dependencies' test-package.json) <(jq '.dependencies' prod-package.json)Keep tests maintainable
Once the gate is in place, make failures actionable. A failing diff <(jq ‘.dependencies’ test-package.json) <(jq ‘.dependencies’ prod-package.json) should point to the exact file and line, not a generic error. Pin the tool version in the lockfile so a upstream release does not silently change behavior. Review the gate output in every pull request just like a test result.