Property-Based Testing
Property-based testing states universal invariants of the code and lets the framework search for a counterexample input, finding edge cases the example-based tester did not think to write.
Definition
Property-based testing is a technique where the test states a property that should hold for all inputs in a domain — e.g., round-trip equality, monotonicity, idempotence — and the framework generates hundreds of random inputs seeking a counterexample, so the test discovers edge cases the hand-written examples missed.
Context and scope
Applied in pure function testing, codec round-trips, state machine properties, and any code where the input space is too large to enumerate but the invariant is concise.
Common misconceptions
- Property-based testing replaces example-based testing — it complements it; examples document intent, properties verify universality
- If no counterexample is found the property is proven — it is tested for the generated inputs; a proof requires exhaustive or symbolic checking
Anti-patterns
- Stating a property so weak it holds for any implementation, finding no bugs but verifying nothing
- Shrinking a counterexample to a non-minimal case, leaving the developer to dig through a large failing input to find the actual trigger
Related concepts
Authoritative source
https://fsharpforfunandprofit.com/posts/property-based-testing/
What it is
Property-based testing is a technique where the test states a property that should hold for all inputs in a domain — e.g., round-trip equality, monotonicity, idempotence — and the framework generates hundreds of random inputs seeking a counterexample, so the test discovers edge cases the hand-written examples missed. Applied in pure function testing, codec round-trips, state machine properties, and any code where the input space is too large to enumerate but the invariant is concise. The distinguishing question is what counts as evidence: a test that passes by accident is worse than no test, because it manufactures false confidence that propagates into release decisions.
How it works
Sound tests exercise a specific behavior contract, isolate the unit under test from environmental noise, and fail with a diagnostic that points at the cause rather than the symptom. Determinism is engineered, not assumed: clocks are fixed, randomness is seeded, and external state is replaced with doubles whose interactions are asserted, not merely stubbed. Coverage measures which behaviors were exercised, not which lines were visited.
When to apply and pitfalls
Apply where the cost of an undetected regression exceeds the cost of writing and maintaining the test. The recurring pitfall is testing the implementation rather than the behavior — such tests break on every refactor and teach nothing. Equally damaging are tests that assert only the happy path, leaving the error branches — where most production bugs live — unexercised.