← Workflows

阅读中文版本

Release Rollback Readiness Workflow

Define and rehearse the rollback path before releasing so a fast revert is possible when a change goes wrong.

advanced~75 minreleaserollbackreliability

Objective

Ensure a tested rollback path exists before any risky release proceeds.

Environment

  • A deploy pipeline with rollback support
  • Database migration rollback plan
  • A staging environment

Steps

1. Define the rollback steps

Write the exact steps to revert code and any schema changes for this release.

Expected state: A concrete, ordered rollback plan exists.

2. Rehearse in staging

Deploy the release and its changes to staging, then execute the rollback plan.

Expected state: The rollback returns staging to the previous healthy state.

3. Verify data compatibility

Confirm the old code works with any schema the new release introduced.

Expected state: Forward-compatible schema means rollback will not break data.

4. Publish readiness criteria

State the conditions that will trigger rollback during the release.

Expected state: Operators know when and how to roll back.

Decision points

Roll back or roll forward?

  • Roll back when data-compatible
  • Roll forward with a fix

Roll back when the schema is compatible; roll forward when a migration cannot be reverted.

Rehearse every release?

  • Rehearse risky releases
  • Rehearse everything

Rehearse releases with schema or infra changes; simple ones need only a documented path.

Expected outcome

A rehearsed rollback path exists so a failed release can be reverted quickly and safely.

Verification

Confirm the staging rollback restored service, schema compatibility holds, and operators can state the trigger criteria.

Cleanup

Document the rehearsal outcome and update the runbook with any gaps found.

Overview

Ensure a tested rollback path exists before any risky release proceeds.

This workflow is reusable across teams and environments; adapt the commands to your tooling while keeping the sequence and checkpoints intact.

Environment and prerequisites

  • A deploy pipeline with rollback support
  • Database migration rollback plan
  • A staging environment

Procedure

1. Define the rollback steps

Write the exact steps to revert code and any schema changes for this release.

Expected state: A concrete, ordered rollback plan exists.

2. Rehearse in staging

Deploy the release and its changes to staging, then execute the rollback plan.

Expected state: The rollback returns staging to the previous healthy state.

3. Verify data compatibility

Confirm the old code works with any schema the new release introduced.

Expected state: Forward-compatible schema means rollback will not break data.

4. Publish readiness criteria

State the conditions that will trigger rollback during the release.

Expected state: Operators know when and how to roll back.

migrate status
# rehearse: deploy previous release and confirm health
kubectl rollout undo deploy/app

Verification

Confirm the staging rollback restored service, schema compatibility holds, and operators can state the trigger criteria.

Decision points

Roll back or roll forward?

  • Roll back when data-compatible
  • Roll forward with a fix

Recommendation: Roll back when the schema is compatible; roll forward when a migration cannot be reverted.

Rehearse every release?

  • Rehearse risky releases
  • Rehearse everything

Recommendation: Rehearse releases with schema or infra changes; simple ones need only a documented path.

Common failures

  • Untested rollback fails when it matters most.
  • Schema changes that are not backward compatible block rollback.
  • No trigger criteria leads to slow decisions.