Capture-Replay Testing
Capture-replay testing records production traffic and replays it against a candidate build, testing real-world input distributions without the labor of synthesizing representative fixtures by hand.
Definition
Capture-replay testing is a technique where production requests (with sensitive fields scrubbed) are recorded into a fixture set and replayed against a new build, so the test exercises the actual input distribution, edge cases, and ordering the production workload exhibits rather than a hand-authored sample.
Context and scope
Applied in shadow traffic testing, database migration validation, and API refactor verification where the risk is behavior change under real inputs the hand-written tests did not imagine.
Common misconceptions
- Capture-replay replaces unit tests — it complements them; unit tests specify behavior, replay tests discover regressions in unspecified behavior
- Recorded traffic is representative forever — the distribution drifts as the product evolves, so the capture must be refreshed
Anti-patterns
- Replaying captured traffic against a build that has schema changes, producing failures from the schema mismatch rather than real regressions
- Scrubbing only the obvious PII and replaying traffic that contains correlated identifiers, leaking production data into the test environment
Related concepts
Authoritative source
What it is
Capture-replay testing is a technique where production requests (with sensitive fields scrubbed) are recorded into a fixture set and replayed against a new build, so the test exercises the actual input distribution, edge cases, and ordering the production workload exhibits rather than a hand-authored sample. Applied in shadow traffic testing, database migration validation, and API refactor verification where the risk is behavior change under real inputs the hand-written tests did not imagine. The distinguishing question is what counts as evidence: a test that passes by accident is worse than no test, because it manufactures false confidence that propagates into release decisions.
How it works
Sound tests exercise a specific behavior contract, isolate the unit under test from environmental noise, and fail with a diagnostic that points at the cause rather than the symptom. Determinism is engineered, not assumed: clocks are fixed, randomness is seeded, and external state is replaced with doubles whose interactions are asserted, not merely stubbed. Coverage measures which behaviors were exercised, not which lines were visited.
When to apply and pitfalls
Apply where the cost of an undetected regression exceeds the cost of writing and maintaining the test. The recurring pitfall is testing the implementation rather than the behavior — such tests break on every refactor and teach nothing. Equally damaging are tests that assert only the happy path, leaving the error branches — where most production bugs live — unexercised.