← Recipes

阅读中文版本

Write Tests That Actually Catch Concurrency Races

Use stress loops, synchronization barriers, and race detectors to make latent races fail deterministically in the test suite.

~35 mintestingconcurrencyunit-testing

Task

Expose a suspected race condition in shared-state code with a test that reliably reproduces it and keeps guarding the fix.

When to use this

A counter occasionally loses increments under load. The bug report is probabilistic, and every attempt to reproduce it by hand fails.

Prerequisites

  • A unit test runner with async support
  • The suspected code with shared mutable state
  • A race detector for your runtime if available, such as Go race or TSAN

Steps

1. Write a stress loop with many concurrent actors.

it("loses no increments under contention", async () => {
  const counter = new Counter();
  await Promise.all(
    Array.from({ length: 200 }, () => counter.increment())
  );
  expect(counter.value).toBe(200);
});

Expected output: Two hundred concurrent increments either hold the invariant or fail reproducibly enough to become a fixed test.

2. Force the interleaving with a barrier.

it("reads stale only after writes complete", async () => {
  const barrier = new Barrier(2);
  const first = writeThenAwait("a", barrier);
  const second = writeThenAwait("b", barrier);
  await Promise.all([first, second]);
  expect(store.consistent()).toBe(true);
});

Expected output: The barrier aligns both tasks at the vulnerable point, turning a lucky timing into a deterministic schedule.

3. Run the runtime race detector on the same code.

go test -race ./store/ 2>/dev/null || node --experimental-test-isolation=none test/race.test.mjs

Expected output: The detector reports the exact memory accesses that conflict, with both stack traces, no luck required.

4. Fix with atomicity, not with timing.

class Counter {
  #mutex = new Mutex();
  async increment() {
    return this.#mutex.runExclusive(() => {
      this.#value += 1;
      return this.#value;
    });
  }
}

Expected output: The increment becomes atomic; the stress test now passes on every run and guards the invariant permanently.

5. Keep the stress test in the fast suite.

it("loses no increments", { timeout: 2000 }, async () => {
  // 200 actors, well under 50ms in practice
});

Expected output: Race guards that take milliseconds run on every commit instead of being exiled to nightly jobs.

Troubleshooting

Problem

Race reproduces in CI but not locally

Solution

CI scheduling differs; add actors or iterations until the failure rate is high enough to be a reliable detector, and seed any randomness.

Problem

Stress test is flaky after the fix

Solution

It still asserts timing rather than the invariant; assert final state and consistency, never elapsed durations.

Problem

Race detector not available for the runtime

Solution

Barriers plus high-iteration stress loops remain effective; align actors at the suspected window and assert the invariant repeatedly.

Limitations

  • Stress tests show a race exists, not every interleaving; the detector covers what the schedule never hits.
  • Fixes add synchronization cost; measure hot paths after serializing them.

Make the unlucky schedule happen on purpose

Races hide because the bad interleaving is rare, and hand-reproduction is gambling. Tests that catch races engineer the luck out: stress loops raise the number of attempts, barriers align tasks at the vulnerable moment, and race detectors observe every memory access schedule. Once the failure is deterministic, the fix is verified by the same test that once failed it.

// Stress: many actors, one invariant
await Promise.all(Array.from({ length: 200 }, () => counter.increment()));
expect(counter.value).toBe(200);

// Barrier: align the adversaries at the window
const barrier = new Barrier(2);

Invariants, not timings

A race test that asserts durations is itself timing-dependent. Assert the invariant: the total count, the consistency of paired fields, the absence of duplicate ids. The schedule varies freely between runs while the property stays fixed, which is what makes the test a detector rather than another flake.

Detectors see the schedules you never run

Stress testing samples schedules; a race detector enumerates conflicting accesses under a happens-before analysis, catching races that a million random runs might miss. Where your runtime has one, the detector belongs in CI permanently: it turns silent corruption into build failures with stack traces on both sides of the conflict.