阅读中文版本

Build a Config Center with Layered Overrides

A configuration service that merges file, environment, and database layers with explicit precedence, exposes merged values over HTTP, and validates every layer against a schema.

config-center/
├── src/
│   ├── server.js
│   ├── merge.js
│   ├── layers/
│   │   ├── fileLayer.js
│   │   ├── envLayer.js
│   │   └── dbLayer.js
│   └── validate.js
├── schemas/app.json
├── base/app.json
├── test/
│   ├── merge.test.js
│   └── api.test.js
└── package.json

function mergeWithOrigins(layers) {
  const merged = {};
  const origins = {};
  for (const { name, values } of layers) {
    walk(values, [], (path, value) => {
      setPath(merged, path, value);
      setPath(origins, path, name);
    });
  }
  return { merged, origins };
}

function fromEnv(prefix = 'APP_', separator = '__') {
  const out = {};
  for (const [rawKey, rawValue] of Object.entries(process.env)) {
    if (!rawKey.startsWith(prefix)) continue;
    const path = rawKey.slice(prefix.length).toLowerCase().split(separator);
    setPath(out, path, parseValue(rawValue));
  }
  return out;
}

const validate = ajv.compile(schema);
if (!validate(merged)) {
  const err = validate.errors[0];
  res.writeHead(503).end(JSON.stringify({
    error: 'invalid configuration',
    path: err.instancePath || '/',
    message: err.message,
  }));
  return;
}

const next = buildMergedTree(app);
const check = ajv.compile(schemas[app]);
if (!check(next.merged)) throw new ConfigValidationError(check.errors[0]);
cache.set(app, next); // reference swap: readers finish on the old tree

Project overview

Configuration bugs are the class of bug that only appears in production, because only production has the real environment variables. A config center makes precedence explicit, validates the final tree before anyone consumes it, and tells every consumer exactly which layer a value came from.

The merge order is the contract at the heart of this project: file defaults, environment overrides for deployment differences, database overrides for runtime tuning. Deep merging objects while replacing scalars and arrays keeps the common case intuitive, and the origin report keeps the surprising cases debuggable.

The finished service delivers five capabilities:

• Layered merge — file values under environment variables under database rows, deep-merged for objects, arrays replaced whole.
• GET /config/:app — returns the validated merged tree plus a per-key origin report.
• Environment mapping — variables map through an explicit prefix and separator convention into nested keys, with typed value parsing.
• Schema gate — validation runs on the merged result and answers 503 with a precise error path instead of serving bad config.
• PUT /admin/:app/override — writes a database override and invalidates the in-memory cache atomically, rejecting schema-breaking writes with 422.

The architecture is a pipeline of small pure stages: three loader modules each produce a plain object, a merger folds them left to right with deep object merge, a validator runs the result through a JSON schema, and a small HTTP facade serves the cached merged tree. The cache is invalidated by replacing the object reference after a full reload, so readers either see the old or the new complete tree and never a half-merged one.

Setup

Verify the prerequisites before starting: Node.js 20 LTS or later, comfort with JSON Pointer style paths and deep merge semantics, and a basic understanding of twelve-factor configuration.

The dependency footprint is three runtime packages plus the test runner:

Create the project and install:

mkdir config-center && cd config-center
npm init -y
npm install ajv@^8.17.0 better-sqlite3@^11.0.0
npm install -D vitest@^2.1.0

Configure the environment in package.json — the server entry point and the test script; base files live in base/, schemas in schemas/, and the SQLite database path is set by the CONFIG_DB environment variable:

{
  "name": "config-center",
  "scripts": { "start": "node src/server.js", "test": "vitest run" }
}

The directory structure mirrors the pipeline: one loader module per layer, the merger and validator as pure functions, and the server as the only module that owns state.

Phase 1: Deep merge with origins

The merger folds layer objects left to right, deep-merging plain objects, replacing other types, and tracking which layer won each key path.

// src/merge.js
export function mergeWithOrigins(layers) {
  const merged = {};
  const origins = {};
  for (const { name, values } of layers) {
    walk(values, [], (path, value) => {
      setPath(merged, path, value);
      setPath(origins, path, name);
    });
  }
  return { merged, origins };
}

function walk(node, path, visit) {
  if (node !== null && typeof node === 'object' && !Array.isArray(node)) {
    for (const [key, value] of Object.entries(node)) walk(value, [...path, key], visit);
  } else {
    visit(path, node);
  }
}

function setPath(target, path, value) {
  let cursor = target;
  for (const key of path.slice(0, -1)) {
    cursor = cursor[key] ??= {};
  }
  cursor[path[path.length - 1]] = value;
}

Two semantic decisions are load-bearing. Plain objects deep-merge, everything else replaces: arrays replace whole rather than merging element-wise, because concatenating two arrays from different layers produces a value neither layer specified — and a replacement rule that never invents values is a rule an operator can reason about. Origins are tracked per leaf, not per subtree: when the database overrides exactly one nested key, the origin map names the database for that leaf and the file for its siblings, which is the granularity the debugging question actually needs.

The fold is left to right in layer order — file, then environment, then database — and the same walk that writes values writes origins, so the two trees can never disagree about shape.

Acceptance for this phase: merge tests assert deep objects combine, scalars override, arrays replace rather than merge, and every leaf origin names the winning layer.

Phase 2: Environment layer mapping

The environment loader translates prefixed variables into nested keys with a documented separator, parsing booleans, integers, and JSON values where possible.

// src/layers/envLayer.js
export function fromEnv(env = process.env, prefix = 'APP_', separator = '__') {
  const out = {};
  for (const [rawKey, rawValue] of Object.entries(env)) {
    if (!rawKey.startsWith(prefix)) continue;
    const path = rawKey.slice(prefix.length).toLowerCase().split(separator);
    setPath(out, path, parseValue(rawValue));
  }
  return out;
}

function parseValue(raw) {
  if (raw === 'true') return true;
  if (raw === 'false') return false;
  if (/^-?\d+(\.\d+)?$/.test(raw)) return Number(raw);
  try { return JSON.parse(raw); } catch { return raw; }
}

The convention is the whole interface: APP_DATABASE__POOL__MAX=20 becomes database.pool.max = 20, lowercase, nested at the double underscore. The prefix must include the app name in real deployments — APP_PAY__DATABASE rather than a shared APP_ — or two tenants’ variables merge into one tree, which is the quietest possible failure mode and the reason the prefix is a constructor argument the tests can pin.

Value parsing runs before the value lands in the tree, and it is deliberately conservative: the exact strings true and false become booleans, strict number patterns become numbers, anything that parses as JSON (including quoted strings) becomes its parsed value, and everything else stays a string. The classic bug this ordering prevents is the string "false" flowing into a boolean-typed schema field and passing validation as a truthy string.

Acceptance for this phase: APP_DATABASE__POOL__MAX maps to database.pool.max with numeric parsing, and unprefixed variables are ignored entirely.

Phase 3: Schema gate and serving

The validator runs the merged tree through Ajv, and the server serves it with origins, answering 503 with the exact failing path when validation fails.

// src/server.js — the gate inside the request handler
const validate = ajv.compile(schema);
if (!validate(merged)) {
  const err = validate.errors[0];
  res.writeHead(503).end(JSON.stringify({
    error: 'invalid configuration',
    path: err.instancePath || '/',
    message: err.message,
  }));
  return;
}
res.writeHead(200, { 'content-type': 'application/json' });
res.end(JSON.stringify({ values: merged, origins }));

Why validate the merged tree rather than each layer? Because a valid file plus valid environment values can still merge into a tree that violates a cross-field constraint or misses a key the environment was supposed to provide — validation on the final product is the only place all three layers are visible at once. And failure is a 503 rather than a best-effort tree, because serving half-valid configuration converts a loud failure into quiet misbehavior.

Ajv errors carry the instance path, and the server forwards it verbatim. An operator sees database.pool.max must be number with the path /database/pool/max, not a 500 with a stack trace — and fixing the offending environment variable takes a minute instead of an afternoon. Nothing invalid is ever cached, so the next request recomputes from the layers rather than serving a poison pill.

Acceptance for this phase: a valid merge serves 200 with the tree and origins; a missing required key serves 503 naming the path.

Phase 4: Override writes and cache swap

The admin route persists an override to SQLite, reloads all layers, validates, and swaps the cached tree only on success.

// src/server.js — admin route handler
const db = require('./layers/dbLayer');

async function applyOverride(app, key, value) {
  db.writeOverride(app, key, value);            // persisted first
  const next = buildMergedTree(app);            // reload all three layers
  const check = ajv.compile(schemas[app]);
  if (!check(next.merged)) {
    db.deleteOverride(app, key);                // roll back the row
    throw new ConfigValidationError(check.errors[0]);
  }
  cache.set(app, next); // reference swap: readers finish on the old tree
}

The cache discipline is the phase’s real lesson: readers of the config are concurrent request handlers, so the cached object is never mutated in place. Every reload builds a complete merged tree off to the side, validates it, and only then swaps the reference. In-flight readers keep the old tree for the rest of their request, which is exactly the consistency most consumers want.

A rejected override therefore has no effect on serving: the write is rolled back, the 422 response carries the same validation detail the periodic reload would log, and the previous tree keeps answering. The failure mode of a bad write is a loud no rather than a soft corruption.

Acceptance for this phase: a valid override is immediately visible in GET responses; a schema-breaking override is rejected with 422 and the previous tree keeps serving.

Design trade-offs

Three fixed layers over an arbitrary layer stack. A plugin architecture for loaders would be more flexible and less legible — the file/env/database order is a deployment convention the whole industry already shares, and hard-coding it makes the precedence diagram part of the type system rather than part of the documentation.

503 over best-effort serving. Refusing to serve beats serving wrong values, at the cost of availability coupling between the config center and schema discipline. The escape hatch is deliberate: fix the offending layer, and the next reload recovers automatically because nothing invalid was cached.

SQLite over a config service. For runtime tuning durability, a local database file beats an external service dependency — one fewer thing to be down. The limitation is honest: external database edits take up to the periodic reload interval to appear, which the timer trigger documents.

Per-leaf origins over per-subtree. Finer granularity costs a slightly larger response payload; in exchange, the debugging question “which layer is winning this key” is answered without ambiguity.

Testing strategy

Run npx vitest. Merge tests cover nested objects, scalar and array replacement, and origin tracking — pure functions, no I/O, fast enough to enumerate every combination. Env layer tests spawn the layer with an injected env object rather than process.env directly, asserting key mapping and value parsing for booleans, numbers, JSON, and strings — injection is what keeps the suite hermetic and parallel-safe.

API tests boot the server with temp layer files and a temp database, then assert three behaviors in order: the 200 tree matches a hand-computed merge; a broken base file produces a 503 naming the exact JSON Pointer path; and a rejected override leaves the previous configuration serving — the last one is the test that catches a cache-swap regression, because it fails only when someone starts mutating the cached tree in place.

The hand-computed merge in the first assertion is the suite’s anchor: a literal object written out in the test file, not a merge of the same layers through the same merger. Asserting against the implementation’s own output would make the test tautological — it would pass when the merger was consistently wrong. The literal is short by construction (three layers, five keys), and every future contributor who wants to know what “deep merge with replacement semantics” means can read the expected object and see the rule applied.

Extending the project

The layer interface is a function that returns a plain object, so new layers are cheap: a Consul or Kubernetes ConfigMap layer is one module, and the merge order in the pipeline definition is the only wiring. Client-side watching is an SSE stream on the cache-swap path — the reference swap already publishes a clean change point. Schema evolution benefits most from the origin report: when a key moves from the file layer to the environment, the origins diff shows the handoff explicitly in review.

For deployment, run one instance per environment behind the service mesh or as a sidecar. Ship base files and schemas inside the image, and mount database storage on a persistent volume. Front the admin route with authentication — the built-in route has none by design, and the deployment guidance is explicit that it must sit behind an authenticating proxy before it leaves localhost. Monitor the 503 counter: it is the earliest signal that an environment variable or override broke a required key.

Troubleshooting playbook

Booleans arrive as the string false. The schema says database.debug is a boolean, the environment variable is APP_DATABASE__DEBUG=false, and validation fails with “must be boolean.” The cause is a value that skipped typed parsing — environment values are strings by definition, and an unparsed "false" is a truthy string as far as JavaScript and the schema are concerned. The fix is the parseValue ordering from the environment layer phase: the exact strings true and false are converted before the value ever lands in the tree. The env-layer test suite includes both spellings with that exact assertion so the conversion cannot regress.

Environment variables leak into other apps. Two services report each other’s configuration keys after a shared deployment. The cause is a prefix that stops one level too early — a shared APP_ prefix means every tenant’s variables enter every tenant’s tree, and the merge silently unions them. The fix is prefix discipline: the prefix must include the app name, APP_PAY__DATABASE style, and the prefix is a constructor argument precisely so each app’s loader is constructed with its own. The test that guards this constructs two loaders with two prefixes against the same env object and asserts zero overlap.

Invalid configuration served after a bad override. A rejected override was visible in GET responses anyway. The cause is a cache written before validation — the admin route mutated the cached tree in place, so even a rolled-back database row had already leaked into serving. The fix is the phase 4 ordering, and it is worth restating as an invariant: the database row is written first, the full tree is rebuilt and validated second, the cache is swapped only on success, and the row is rolled back on failure. In-flight readers may finish on the old tree, and that is correct — the alternative is readers seeing a tree that never passed validation.