Multi-Environment Config Matrix Workflow
Model environment configuration as a base-plus-overlay matrix, validate every cell, and prevent the typo that only appears in prod.
Objective
Manage configuration across environments as a validated matrix where every cell is checked before deploy, ending the prod-only typo class.
Environment
- Config files per environment with shared structure
- A validation schema for the config type
- A pipeline that can render and validate all cells
Steps
1. Restructure into base plus overlays
Move shared values into one base config with per-environment overlays containing only true differences, making divergence explicit.
Expected state: Each environment's file contains only what actually differs from base.
2. Define the schema once
Write one validation schema for the config type covering types, ranges, required keys, and cross-field constraints.
Expected state: One schema validates every environment's rendered output.
3. Validate every cell in CI
In CI, render each environment's effective config and validate all of them, not just the one being deployed.
Expected state: A typo in any environment's overlay fails every build.
4. Diff the cell being deployed
On deploy, show the effective-config diff for the target environment as part of review, so changes are seen as they apply.
Expected state: Deploy reviews include the rendered effective config diff.
5. Guard the cross-environment invariants
Add checks for invariants that must hold across cells: prod never pointing at staging services, secrets never shared between environments.
Expected state: Invariant violations fail validation regardless of schema correctness.
Expected outcome
A configuration matrix where divergence is explicit, every cell is validated on every change, and cross-environment invariants are mechanically enforced.
Verification
An injected typo in any environment's overlay fails CI; an invariant violation (prod referencing staging) is rejected by the cross-check.
The prod-only typo is a structure problem
Full-copy config files mean every environment edit is a re-write of everything, and typos hide until the one deploy that matters. Base-plus-overlay makes divergence explicit, and validating every cell in CI catches the typo in any environment on every change.
structure:
config/base.yaml (shared truth)
overlays/staging.yaml (url, log level, flags)
overlays/prod.yaml (url, replicas, timeouts)
CI: render ALL cells, validate ALL:
staging: valid prod: INVALID
prod.replicas: "six" <- typo, expected int
(fails on ANY change, not just prod deploys)
deploy review includes rendered diff:
prod effective: - replicas: 6
+ replicas: 8 (seen as applied)
cross-invariants:
prod.service_url contains ".prod." PASS
staging.secret_hash != prod.secret_hash PASS
(secrets never shared across cells)Divergence should be visible or absent
When configs are full copies, the actual differences between environments are invisible — buried in ninety identical lines. Overlays invert this: a difference must be declared to exist, which makes accidental divergence impossible and intentional divergence reviewable.
Validate every cell on every change
Validating only the environment being deployed leaves prod typos dormant until the scariest moment. Rendering and validating all cells on every build means a staging-only change still fails CI if prod’s overlay is broken — the typo class dies completely.