How to Build Config from Many Sources

Introduction

The hardest part of merge many sources is not writing the code — it is deciding what the code is allowed to reject. A multi-source config setup that accepts everything is easy to ship and expensive to operate, because every malformed input becomes a production surprise. The discipline is to enumerate the shapes the system actually receives, encode them in multi-source config, and let the library reject the rest loudly. This keeps the happy path fast and the failure path visible.

A multi-source config guard for merge many sources should be proportional to the risk. A ten-field form does not need a hundred-line schema; it needs the five checks that catch the five mistakes the users actually make. Ship the five, measure the rejections, and add the sixth only when the traffic proves it exists. Speculative generality is the bug you write on purpose, and multi-source config makes it easy to write, which is why discipline matters.

The sections below walk through the concrete decisions: where to put the multi-source config check, how to keep it fast under load, what to do when the contract changes, and how to test the failure paths so they degrade on purpose instead of by accident.

List the sources

List the sources is the first decision because it determines what the rest of the multi-source config pipeline can assume. A wrong choice here turns into merge many sources under load, and the invariant you thought multi-source config guaranteed no longer holds. Measure the impact before committing, because the cost of a bad choice is paid in incidents, not in a code review.

The test for List the sources is the one most teams skip: drive multi-source config to the failing condition and assert the system degrades gracefully. Write it with a fake clock for time-based logic, a stub for the failure path, and an assertion on the user-visible outcome. A test that passes by timing out is a hope, not a test.

const cfg = CfgSchema.parse({
  ...defaults, ...file, ...env, ...flags, ...await remote.get(),
});

The typescript above is the shape to ship for merge many sources. It is small enough to audit and specific enough that the failure behavior is obvious. Adapt the constants to your traffic and latency budget, and resist the urge to handle cases the system has not seen yet.

Order by precedence

The mechanism behind Order by precedence is straightforward in multi-source config, but the consequences ripple. A wrong choice here turns into merge many sources under load, and the invariant you thought multi-source config guaranteed no longer holds. Measure the impact before committing, because the cost of a bad choice is paid in incidents, not in a code review.

The test for Order by precedence is the one most teams skip: drive multi-source config to the failing condition and assert the system degrades gracefully. Write it with a fake clock for time-based logic, a stub for the failure path, and an assertion on the user-visible outcome. A test that passes by timing out is a hope, not a test.

Merge in order

Merge in order is where most multi-source config implementations leak performance. A wrong choice here turns into merge many sources under load, and the invariant you thought multi-source config guaranteed no longer holds. Measure the impact before committing, because the cost of a bad choice is paid in incidents, not in a code review.

The test for Merge in order is the one most teams skip: drive multi-source config to the failing condition and assert the system degrades gracefully. Write it with a fake clock for time-based logic, a stub for the failure path, and an assertion on the user-visible outcome. A test that passes by timing out is a hope, not a test.

test('remote overrides flags', () => {
  expect(cfg.feature).toBe(remoteValue);
});

Pin the behavior with this test. A multi-source config setup that passes this test degrades on purpose instead of by accident, which is the difference between a managed failure and an incident.

Validate

The hard part of Validate is not the happy path — it is what happens when multi-source config fails halfway. A wrong choice here turns into merge many sources under load, and the invariant you thought multi-source config guaranteed no longer holds. Measure the impact before committing, because the cost of a bad choice is paid in incidents, not in a code review.

The test for Validate is the one most teams skip: drive multi-source config to the failing condition and assert the system degrades gracefully. Write it with a fake clock for time-based logic, a stub for the failure path, and an assertion on the user-visible outcome. A test that passes by timing out is a hope, not a test.

Cache the remote

Cache the remote should be observable without a debugger. A wrong choice here turns into merge many sources under load, and the invariant you thought multi-source config guaranteed no longer holds. Measure the impact before committing, because the cost of a bad choice is paid in incidents, not in a code review.

The test for Cache the remote is the one most teams skip: drive multi-source config to the failing condition and assert the system degrades gracefully. Write it with a fake clock for time-based logic, a stub for the failure path, and an assertion on the user-visible outcome. A test that passes by timing out is a hope, not a test.

Test the merge

Test the merge is a trade, not a winner. A wrong choice here turns into merge many sources under load, and the invariant you thought multi-source config guaranteed no longer holds. Measure the impact before committing, because the cost of a bad choice is paid in incidents, not in a code review.

The test for Test the merge is the one most teams skip: drive multi-source config to the failing condition and assert the system degrades gracefully. Write it with a fake clock for time-based logic, a stub for the failure path, and an assertion on the user-visible outcome. A test that passes by timing out is a hope, not a test.

Common pitfalls

The common pitfalls in merge many sources all reduce to one mistake: treating multi-source config as a formality instead of a contract. The contract is the thing the rest of the system relies on, and a weak contract propagates weakness everywhere.

• Skipping the multi-source config boundary so merge many sources enters unvalidated.
• Assuming multi-source config is fast without measuring the tail latency under load.
• Letting merge many sources fail silently instead of failing loudly with a metric.
• Testing only the happy path so the multi-source config failure surfaces in production.
• Sharing mutable state across requests so one bad input poisons the next.

Conclusion

Do not conflate the conclusion with the implementation. They look similar from the outside but the failure modes differ: multi-source config fails fast and the caller knows, while an unchecked path fails slow and the caller guesses. Choose the one whose failure mode your system can absorb, and document the choice at the call site.

Self-check: write a test that drives multi-source config to the failing condition and assert the system degrades gracefully. Then merge many sources is a property you maintain, not a feature you hope for.

Sources: Config sources, Zod

Report a problem with this articleOpens a pre-filled issue on GitHub.