Deploy Pinned Artifacts by Digest, Not by Tag

Pin container images and packages by content digest, record the exact deployed artifact in metadata, and make tag drift impossible to ship.

~25 mindeploydockerci

Task

Convert deployments from mutable tags to immutable digests and verify every environment runs the exact artifact CI built.

When to use this

Two environments both run app:2.4.1 yet behave differently. Someone re-pushed the tag with different content, and the tag quietly stopped meaning anything.

Prerequisites

A registry serving content-addressable digests, such as any OCI registry
A deployment mechanism that accepts a digest reference
CI able to resolve and record digests at build time

Steps

1. Resolve the digest behind the tag at build time.

docker build -t app:ci . && docker inspect --format '{{index .RepoDigests 0}}' $(docker images -q app:ci | head -1) 2>/dev/null || docker manifest inspect app:ci | grep -m1 digest

Expected output: The immutable sha256 digest of the built artifact is captured, for example sha256:9f86d081884c7d65.

2. Deploy by digest everywhere.

image: registry.example.com/app@sha256:9f86d081884c7d659a2feaa0c55ad015a3bf4f1b2b0b822cd15d6c15b0f00a08

Expected output: The same content-addressed reference deploys to every environment, so identical means identical byte for byte.

3. Record the digest as deployment metadata.

kubectl annotate deploy/app \
  artifact.sha256=9f86d081884c7d65 \
  artifact.build-id="$BUILD_ID" \
  artifact.git-sha="$GIT_SHA"

Expected output: The running deployment carries its provenance, answering what exactly is deployed without guessing from tags.

4. Verify all environments agree.

kubectl get deploy -A -o custom-columns=NS:.metadata.namespace,IMAGE:.spec.template.spec.containers[0].image | grep app

Expected output: Every namespace shows the same digest, proving no environment drifted to different content under one tag.

5. Block tag-only references in CI.

- name: Forbid mutable tags
  run: |
    if grep -rq "registry.example.com/app:" k8s/; then
      echo "found mutable tag reference"; exit 1
    fi

Expected output: Any manifest referencing the image by tag fails the build, so drift cannot return through a routine edit.

Troubleshooting

Problem

Digest deployment breaks rollback scripts

Solution

Rollback needs the previous digest, not latest; record digests per release in the deployment metadata and have the rollback script read them.

Problem

Multi-arch images complicate pinning

Solution

A manifest list digest differs from per-arch digests; pin the manifest list digest so the reference resolves correctly on every node architecture.

Problem

Local development prefers tags

Solution

Keep tags for humans in dev; gate the transition at CI, where the tag is resolved once to a digest and everything downstream is pinned.

Limitations

Digests are opaque; tooling must translate them back to build metadata for humans, which is what the annotations provide.
Pinning prevents drift but not vulnerability inheritance; still rebuild on base-image security updates.

A tag is a rumor, a digest is a fact

Tags are mutable pointers: anyone with push rights can retag different content, and two environments claiming the same version can run different bytes. Content digests are cryptographic facts about the artifact itself. Deploying by digest makes every environment provably identical, makes rollback a precise operation on recorded digests, and turns what-exactly-is-running from archaeology into annotation lookup.

image: registry.example.com/app@sha256:9f86d081884c7d659a2feaa0c55ad015a3bf4f1b2b0b822cd15d6c15b0f00a08

# Provenance on the deployment itself
metadata:
  annotations:
    artifact.sha256: 9f86d081884c7d65
    artifact.git-sha: 4c2f9e1

Resolve once, at the boundary

The clean division of labor: developers and CI workflows may speak in tags, but the moment CI resolves a tag to a digest, everything downstream references the digest only. Humans keep readable names, environments get immutable facts, and the CI step that performs the resolution is the single place where the translation happens.

Annotations are your deployment ledger

Recording the digest, build id, and git SHA as deployment annotations creates a queryable history of what ran where. Incident response stops guessing whether the tag was retagged and starts reading. The same annotations drive automated rollback, which can only be precise if it knows exact digests to return to.