Add Property-Based Testing with fast-check
Replace hand-written examples with property-based generators that explore the input space and shrink failures to a minimal counterexample.
Task
Add npx fast-check --num-runs 1000 to the test suite so regressions are caught automatically on every pull request.
When to use this
A refactor passes existing tests but breaks npx fast-check --num-runs 1000, shipping a silent regression to users.
Prerequisites
- A test runner configured (Vitest or Jest)
- Source code with testable units
Steps
1. Configure npx fast-check --num-runs 1000 in the project configuration file with the recommended defaults.
npx fast-check --num-runs 1000
Expected output: The configuration is accepted and npx fast-check --num-runs 1000 runs without errors on the first invocation.
2. Run the check locally to verify npx fast-check --num-runs 1000 produces the expected result before pushing.
pnpm test
Expected output: Local run matches CI behavior; no surprises after push.
3. Add the gate to the CI pipeline so npx fast-check --num-runs 1000 blocks merge on failure.
- name: Run add property based testing
run: pnpm test
Expected output: Pull requests with failing npx fast-check --num-runs 1000 are blocked from merging.
Troubleshooting
Problem
The check passes locally but fails in CI
Solution
Pin dependency versions and the runtime to match CI exactly; reproduce with the same Docker image or act.
Problem
Configuration is rejected with a schema error
Solution
Check the tool version matches the config schema; older versions may not support newer fields.
Limitations
- The gate only runs on events that trigger the workflow; locally pushed commits bypass it.
- Configuration drift is possible if someone edits settings outside version control.
Cover the regression path
Start by measuring the problem. Replace hand-written examples with property-based generators that explore the input space and shrink failures to a minimal counterexample. The most common mistake is adding the gate without first confirming it catches the issue you care about. Run npx fast-check –num-runs 1000 against the current codebase and inspect the output before wiring it into CI.
npx fast-check --num-runs 1000Keep tests maintainable
Once the gate is in place, make failures actionable. A failing npx fast-check –num-runs 1000 should point to the exact file and line, not a generic error. Pin the tool version in the lockfile so a upstream release does not silently change behavior. Review the gate output in every pull request just like a test result.