Test API Schema Diff for Breaking Changes
Compare the OpenAPI spec before and after a PR so a removed field or narrowed type is flagged as a breaking change.
Task
Add npx @redocly/openapi-cli diff old.yaml new.yaml --fail-on breaking to the test suite so regressions are caught automatically on every pull request.
When to use this
A refactor passes existing tests but breaks npx @redocly/openapi-cli diff old.yaml new.yaml --fail-on breaking, shipping a silent regression to users.
Prerequisites
- A test runner configured (Vitest or Jest)
- Source code with testable units
Steps
1. Configure npx @redocly/openapi-cli diff old.yaml new.yaml --fail-on breaking in the project configuration file with the recommended defaults.
npx @redocly/openapi-cli diff old.yaml new.yaml --fail-on breaking
Expected output: The configuration is accepted and npx @redocly/openapi-cli diff old.yaml new.yaml --fail-on breaking runs without errors on the first invocation.
2. Run the check locally to verify npx @redocly/openapi-cli diff old.yaml new.yaml --fail-on breaking produces the expected result before pushing.
act -j build # or: pnpm test
Expected output: Local run matches CI behavior; no surprises after push.
3. Add the gate to the CI pipeline so npx @redocly/openapi-cli diff old.yaml new.yaml --fail-on breaking blocks merge on failure.
- name: Run add api schema diff test
run: pnpm test
Expected output: Pull requests with failing npx @redocly/openapi-cli diff old.yaml new.yaml --fail-on breaking are blocked from merging.
Troubleshooting
Problem
The check passes locally but fails in CI
Solution
Pin dependency versions and the runtime to match CI exactly; reproduce with the same Docker image or act.
Problem
Configuration is rejected with a schema error
Solution
Check the tool version matches the config schema; older versions may not support newer fields.
Limitations
- The gate only runs on events that trigger the workflow; locally pushed commits bypass it.
- Configuration drift is possible if someone edits settings outside version control.
Cover the regression path
Start by measuring the problem. Compare the OpenAPI spec before and after a PR so a removed field or narrowed type is flagged as a breaking change. The most common mistake is adding the gate without first confirming it catches the issue you care about. Run npx @redocly/openapi-cli diff old.yaml new.yaml –fail-on breaking against the current codebase and inspect the output before wiring it into CI.
npx @redocly/openapi-cli diff old.yaml new.yaml --fail-on breakingKeep tests maintainable
Once the gate is in place, make failures actionable. A failing npx @redocly/openapi-cli diff old.yaml new.yaml –fail-on breaking should point to the exact file and line, not a generic error. Pin the tool version in the lockfile so a upstream release does not silently change behavior. Review the gate output in every pull request just like a test result.