Test Both Sides of a Feature Flag

Run the same behaviour tests with the flag on and off, assert the default and kill switch, and retire the flag when it expires.

~30 mintestingfeature-flagsrelease

Task

Cover both flag states in automated tests so a rollout cannot silently break the disabled path.

When to use this

A new checkout flow sits behind a flag. Tests only run with the flag on, and the off path breaks because nobody exercises it after launch.

Prerequisites

OpenFeature server SDK or an equivalent flag abstraction
A test runner with parameterized tests
An in-memory provider so tests do not call a remote flag service

Steps

1. Depend on a small flag interface instead of importing a global client into handlers.

export interface Flags {
  isEnabled(key: string, context: FlagContext): boolean;
}

Expected output: Handlers receive Flags through their constructor, so tests can supply any implementation.

2. Configure an in-memory provider with explicit on and off variants for the test flag.

import { OpenFeature, InMemoryProvider } from '@openfeature/server-sdk';

function providerFor(enabled) {
  return new InMemoryProvider({
    'checkout-v2': {
      variants: { on: true, off: false },
      defaultVariant: enabled ? 'on' : 'off',
      disabled: false,
    },
  });
}

Expected output: Flag evaluation is deterministic and local; no network call or remote state affects the result.

3. Run the same journey for both states and require both to pass.

for (const enabled of [true, false]) {
  test(`checkout works with checkout-v2=${enabled}`, async () => {
    await OpenFeature.setProviderAndWait(providerFor(enabled));
    const res = await request('/api/checkout', { method: 'POST', body: cart });
    assert.equal(res.status, 200);
  });
}

Expected output: Two cases exercise the old and new checkout implementations; a failure in either fails the suite.

4. Assert safe defaults for a missing flag and a provider error.

const client = OpenFeature.getClient();
await OpenFeature.setProviderAndWait(new InMemoryProvider({}));
assert.equal(await client.getBooleanValue('checkout-v2', false), false);
assert.equal(await client.getBooleanValue('unknown-flag', false), false);

Expected output: Unknown or missing flags evaluate to the caller's default instead of throwing or enabling the feature.

5. Track expiry dates and fail a scheduled check when a flag outlives its rollout.

const expired = registry.filter((f) => new Date(f.expiresAt) < new Date());
if (expired.length > 0) {
  console.error('expired flags:', expired.map((f) => f.key));
  process.exitCode = 1;
}

Expected output: The nightly job lists expired flags, and the next release removes the flag plus the dead branch.

Troubleshooting

Problem

Flag state leaks between tests

Solution

The provider is global. Set it in beforeEach, run tests serially for the flag suite, and reset it in afterEach, or inject a per-test client.

Problem

Only the enabled path is covered

Solution

Parameterize every behaviour test by flag state, and make both cases required in CI rather than an optional job.

Problem

Removing the flag breaks many tests

Solution

Tests asserted flag calls instead of behaviour. Keep flag assertions only in the default and kill-switch tests, and delete them with the flag.

Limitations

Two states per flag multiply the matrix; with many flags, test pairs rather than the full combination.
An in-memory provider does not validate targeting rules, segments, or percentage rollouts.
Expiry tracking is a process control; nothing in the runtime prevents leaving a flag in place forever.

Flags create two products

A feature flag is not a configuration detail; it means two code paths are shipped and one of them is live for some users. If tests only exercise the enabled path, the disabled path rots silently. The first time it runs may be during an incident, when the flag is turned off to stop an outage and the old path fails for a different reason. Both paths are production code and both need coverage.

Make evaluation injectable

Handlers that import a global flag client are hard to test and easy to couple to a vendor. A one-method interface is enough: isEnabled(key, context). Production wires the real provider; tests wire an in-memory implementation with explicit variants. The same tests then run against both states without mocks of internal calls, which keeps them honest.

for (const enabled of [true, false]) {
  test(`checkout works with checkout-v2=${enabled}`, async () => {
    await OpenFeature.setProviderAndWait(providerFor(enabled));
    assert.equal((await request('/api/checkout', { method: 'POST', body: cart })).status, 200);
  });
}

Test the defaults explicitly

Most flag SDKs accept a default value for a reason: the flag may be missing, the provider may be unreachable, or the key may be misspelled. Assert that an unknown flag yields the safe default and never throws. The kill switch deserves its own test: when the flag is off, the new dependency must not be called at all, not merely return an error.

Avoid a combinatorial explosion

Three flags already mean eight combinations, and most teams have dozens. Test each flag in isolation with the others pinned to their production defaults, and add a small number of end-to-end cases for combinations that actually coexist. This catches the common failure, a branch that is never exercised, without turning the suite into an exponential matrix.

Retire flags on a schedule

A flag that stays for a year becomes permanent complexity with a mysterious boolean. Give every flag an owner and an expiry date in a registry, and run a scheduled check that fails when the date passes. Removal should delete the flag, the dead branch, and the tests that only existed for the rollout. That last step is what keeps the codebase from accumulating dormant paths.