Make Randomized Tests Reproducible with Seeds

1. Audit where randomness enters the suite.

rg -n "Math\.random|crypto\.randomUUID|crypto\.getRandomValues" src/ tests/ | head -10

Expected output: Every call site prints; direct randomness calls outside a seeded generator are the reproducibility leaks to fix.

2. Route randomness through one seeded generator.

// tests/helpers/rand.ts
import { createRequire } from 'node:module';
const seed = Number(process.env.TEST_SEED ?? Date.now());
console.error(`[rand] seed=${seed}  (replay with TEST_SEED=${seed})`);

// xorshift128: tiny, fast, seedable
let s = seed >>> 0 || 1;
export const rand = () => {
  s ^= s << 13; s >>>= 0;
  s ^= s >> 17;
  s ^= s << 5;  s >>>= 0;
  return s / 4294967296;
};
export const pick = <T>(xs: readonly T[]) => xs[Math.floor(rand() * xs.length)];

Expected output: All randomized helpers derive from one seed printed at startup; any failing run can be replayed with the logged TEST_SEED value.

3. Seed everything else that is implicitly random: time, ids, order.

// before: hidden randomness
const id = crypto.randomUUID();
const at = new Date();
tests.sort();

// after: derived from the same seed
const id = `id-${rand().toString(36).slice(2, 10)}`;
const at = new Date(seed);
tests.sort((a, b) => hash(seed, a.name) - hash(seed, b.name));

Expected output: UUIDs, timestamps, and execution order become functions of the seed, so replay reproduces them exactly instead of merely approximately.

4. Prove replay works by failing on purpose at a seed.

TEST_SEED=777 npx vitest run tests/checkout.test.ts 2>&1 | grep -E "seed=|FAIL|✓|×" | head -8

Expected output: The output shows the seed line; injecting a temporary assertion that fails only under seed 777 demonstrates identical replay on rerun.

5. Make CI feed its seed back into reruns.

# ci.yml
- name: Test
  run: npx vitest run 2>&1 | tee test.log
- name: Rerun failures with the same seed
  if: failure()
  run: |
    SEED=$(grep -oP "seed=\K[0-9]+" test.log | head -1)
    echo "replaying with seed=$SEED"
    TEST_SEED=$SEED npx vitest run --rerun-tried || true

Expected output: On failure, CI logs the seed and immediately reruns with it, so the log itself contains a locally reproducible trace.

Problem

Test still not reproducible after seeding

Solution

Something random remains: async timing, network, or a library internal; search for remaining direct random calls, then stub Date and Math.random globally in test setup.

Problem

Library has its own seed mechanism

Solution

Forward your seed into it (fast-check seed option, jest randomSeed) rather than maintaining two systems; print both if the library seed differs from yours.

Problem

Parallel workers each got a different seed

Solution

Derive per-worker seeds deterministically from the main seed and the worker index, and log worker plus seed, so any single failure replays in that worker context.