Rotate a Leaked Secret Safely

Revoke a leaked credential, issue a replacement, deploy it through a secret manager, and purge the old value from history where your threat model requires it.

~30 minsecuritysecretsgit

Task

Revoke and replace a credential that was committed to a repository or exposed in logs, then verify the old value no longer grants access.

When to use this

A database password, API token, or signing key was committed to Git, printed in CI logs, or included in a client bundle. Assume it is compromised and act before cleaning history.

Prerequisites

Access to the provider console or secret manager to revoke and create credentials
Repository admin rights if history rewriting is required
Ability to redeploy every service that uses the secret

Steps

1. Revoke the exposed credential at the provider first. Do not wait for history cleanup; revocation immediately removes the attacker's access.

openssl rand -base64 32   # generate the replacement value

Expected output: A new random secret that has never appeared in the repository or logs.

2. Store the replacement in the secret manager, update the deployment environment, and redeploy all consumers. Confirm the new value works before removing the old one entirely.

Expected output: Services authenticate successfully with the new credential and the old credential is rejected.

3. Remove the value from the working tree and repository history. Rotate anything derived from it (sessions, tokens, webhooks) as well.

git filter-repo --path path/to/exposed-file --invert-paths

Expected output: The file is absent from rewritten history; all collaborators must re-clone or reset.

4. Search the current tree, CI configuration, and build artifacts for the old value. Update incident notes with revocation time, replacement time, and any access observed.

git grep -n 'OLD_SECRET_PREFIX' $(git rev-list --all)  # audit before and after rewrite

Expected output: No live secret remains; only historical audit records may reference the incident.

Troubleshooting

Problem

Services fail after rotation

Solution

Check that every consumer reads from the secret manager, not a cached .env file or baked image. Redeploy rather than editing running containers.

Problem

History rewrite breaks collaborators

Solution

Announce a freeze, require re-clone, and never force-push over unmerged work without coordination.

Limitations

History rewriting cannot remove copies already cloned, cached by CI, or indexed by third parties; revocation is the only reliable control.
Some providers require a staged rotation with two valid credentials; follow the provider's rotation procedure.

Why revocation comes first

Cleaning Git history is slow and incomplete: clones, CI caches, and logs may already contain the secret. Revoking at the provider is the only step that reliably stops abuse. Treat history rewriting as cleanup, not containment.

Generate a replacement and verify it through the real deployment path:

openssl rand -base64 32
# Store the value in your secret manager, never in a committed file.
aws secretsmanager update-secret \
  --secret-id prod/app \
  --secret-string "$(openssl rand -base64 32)"

Then remove the old value from history:

git filter-repo --path .env.production --invert-paths
git push --force-with-lease origin main

Verify

The provider rejects the old credential.
The application starts with the new credential and one real request succeeds.
git grep and artifact scanning find no live copy of the old value.
Collaborators have re-cloned or reset to the rewritten history.
Incident notes record what the leaked credential could access and for how long.

When not to rewrite history

If the repository is widely mirrored or the secret is already public, rewriting history adds disruption without improving security. Revoke, replace, and monitor instead. For secrets that never left a private environment, a normal rotation may be sufficient.