Package and Release
Build one artifact, address it by digest, promote it through environments, and make the release a recorded decision separate from the deployment.
The question, the obvious approach, and why it breaks
Every lesson starts where the work starts: an operational problem, a first attempt that is entirely reasonable, and the way production disagrees with it.
What exactly is being shipped, how do you refer to it unambiguously, and who decided it should go?
During an incident the highest-value question is "what is running and what changed". If the artifact was rebuilt per environment, or referred to by a mutable tag, or deployed without a release record, that question has no answer that can be checked.
Build the image in the deploy job for each environment. Tag it latest for convenience, or with the branch name. Deploying is releasing, so there is nothing extra to record.
Rebuilding per environment means the artifact you tested is not the artifact you shipped. Base images, transitive dependencies and build tooling all move between builds (Build Once, Deploy Many).
- Rebuilding per environment means the artifact you tested is not the artifact you shipped. Base images, transitive dependencies and build tooling all move between builds (Build Once, Deploy Many).
- A mutable tag is a pointer, not an identity. Two hosts pulling
app:latestan hour apart can legitimately run different code, and both are "on latest" (Tags Versus Digests). - With no release record there is nothing that names the artifact, its config and its migration state together — so the rollback target has to be reconstructed from deploy logs.
- Deploy-is-release means there is no point at which anyone can say "not yet". The decision is implicit, so it cannot be refused, delayed or audited (Deployment Is Not Release).
- Provenance disappears. When a vulnerability lands in a dependency, "which running artifacts contain it" becomes a rebuild-and-guess exercise (Software Bill of Materials).
What is actually happening
Underneath the tooling, which is the part that survives a change of tool.
- Packaging turns a build output into a distributable, addressable unit: an image, an archive, a bundle. The unit's identity is the content hash of what it contains — a digest — which is immutable by construction.
- A tag is a mutable name pointing at a digest. Tags are for humans; digests are for machines and for evidence. Every claim about what is running should be expressed as a digest (Tags Versus Digests).
- Promotion moves an artifact between environments by reference, not by rebuilding. The bytes that ran in staging are the bytes that run in production (Promotion).
- A release is a decision that names an artifact digest, the configuration it runs with, and any migration state it requires. It is the unit you can approve, refuse, record and roll back to (The Release Manifest).
- Deployment is the mechanical act of putting a released artifact onto infrastructure. Separating the two is what lets one release deploy to many places, and lets a deploy happen without a release taking effect (Feature Flags: Deploy Is Not Release).
- The artifact is not the whole story. Artifact plus configuration plus schema state is what actually runs, which is why a release record has to name all three (Artifact Plus Configuration).
From commit to release candidate
The chain here is short and every hop has a way to lose identity. The rule is that each step should be able to name its input exactly, and each step's output should be addressable.
- 1Build
Compiles source plus pinned dependencies into an output.
fails by Unpinned inputs, so the same commit produces different bytes on different days (Dependency Pinning).
evidence A recorded set of inputs and a build log naming the commit (Reproducible Builds).
- 2Package
Assembles the deployable form and computes its digest.
fails by Build metadata baked in that changes per build, so nothing is comparable.
evidence A digest, and a label carrying the source commit.
- 3Publish
Pushes the artifact to a registry where deploy targets can pull it.
fails by Pushed to a tag that already existed, silently replacing what it pointed to.
evidence The registry reports the digest; the digest matches what the build computed.
- 4Sign
Attests who built this artifact from what source.
fails by Verification failing open at deploy time (Signing and Verifying Artifacts).
evidence A verifiable signature over the digest, checked at admission and not only at publish.
- 5Promote
Marks the same digest as a candidate for the next environment.
fails by Rebuilding instead of promoting, which discards every test result (Promotion).
evidence Identical digest strings across environments.
- 6Release
Records the decision: this digest, this config, this migration state, go.
fails by No record, so the rollback target has to be reconstructed later (The Release Manifest).
evidence A release record that a person or a script can read afterwards.
A release names three things, not one
What runs in production is an artifact, a configuration and a schema state. A release record that names only the artifact describes a third of the system and gives false confidence about rollback: the previous digest with the current config is a combination that has never been tested.
The format below is illustrative — the point is the fields, not the syntax. Any representation that carries these facts and can be read after the fact does the job.
1{2 "release": "2026-08-26.3",3 "artifact": "registry.internal/checkout@sha256:9f3e...",4 "commit": "a1b2c3d",5 "config": "checkout-prod@v41",6 "schema": {7 "requires": "2026_08_20_add_new_name",8 "phase": "expand",9 "reversible": true10 },11 "previous": "2026-08-25.1",12 "decided_by": "release-approval#8812",13 "expected_effect": "new_name written alongside old_name; read path unchanged"14}The two fields people omit are previous and schema. Without previous, rollback starts with a search. Without schema.reversible, the roll-back-or-forward decision during an incident is made on a guess.
Rebuild versus promote
This is the single decision that determines whether your pre-production testing means anything. Rebuilding per environment is the default in many pipeline templates, because it looks tidy: each environment has its own job.
merge -> build -> deploy staging
(build again) -> deploy prod
staging ran image A
prod ran image B
A and B came from the same commit
and are not the same bytesmerge -> build -> app@sha256:9f3e...
|
+-> deploy staging (9f3e)
+-> verify
+-> release record (9f3e)
+-> deploy prod (9f3e)Between two builds of the same commit, a floating base image tag, an unpinned transitive dependency or a different builder version can all change the output. The first pipeline can pass every test in staging and ship different bytes to production, and the difference is invisible because both are "the same commit".
How to do it properly
Most important first.
- Build once, from a commit, and never rebuild for a later environment. Promote the digest instead.
- Refer to artifacts by digest in every deployment manifest. Keep tags as human labels that point at digests.
- Record a release: digest, config version, migration state, who decided, when, and what it is expected to change (The Audit Trail).
- Label the artifact with the commit it came from, so a running process can be traced back to source without a lookup table (Build Provenance).
- Keep the previous release deployable. Retention policy that expires the artifact you would roll back to is a rollback path that quietly stopped existing (Artifact Retention).
- Sign artifacts and verify signatures at deploy time if your threat model includes registry compromise (Signing and Verifying Artifacts).
How much can this affect
Every production change has a blast radius. Stated as a scale so it is comparable between changes rather than adjectival — and paired with what actually contains it, because a wide scope with a real containment mechanism is a different situation from a wide scope with none.
The release decision itself, plus whatever deployment strategy follows it. A wrong release with no progressive rollout reaches everyone as fast as the deploy runs.
What can go wrong
- Registry retention expiring the previous artifact, discovered at the moment you try to roll back to it.
- A promotion that rebuilds "because the pipeline is set up that way", silently reintroducing the drift promotion exists to prevent.
- Release records that name a tag rather than a digest, which makes them look precise and leaves them ambiguous.
- Config promoted separately from the artifact and out of step with it, so the released pair was never tested together (Configuration Drift).
- A signing step that fails open — the signature check errors and deployment proceeds, which is worse than not signing, because it is believed.
- "Immutable means the artifact cannot be replaced." It means the digest identifies exactly those bytes. You can publish a new artifact; you cannot change what an existing digest refers to.
- "Semantic versioning tells me whether an upgrade is safe." It tells you what the publisher intended to communicate. It is a claim, not a guarantee (Semantic Versioning, and Where It Stops Applying).
- "We have a release process because we have a deploy pipeline." A pipeline that deploys on merge has no release step; that may be the correct choice, but it should be a choice.
- "Signed artifacts mean the code is safe." Signing establishes who built it, not that what they built is good (Signing and Verifying Artifacts).
Operating it
Evidence is the signal, not the intention. Rollback is sometimes 'you cannot, and that is the point'.
- For any running instance, you can print the artifact digest and the commit, from the process itself.
- The staging digest and the production digest are the same string for the same release.
- The release record for what is live names a digest, a config version and a migration state.
- You have rolled back to the previous release recently enough to know the artifact is still there.
- Redeploy the previous release record by digest. This is the cheapest rollback in the entire lifecycle and the reason the record exists.
- If config changed with the release, roll back the config version named in the record too — rolling back only the artifact leaves a pair that was never tested (A Config Change Is a Production Change).
- If a migration ran, the artifact rollback is easy and the schema rollback may not exist. Expand/migrate/contract is what makes this survivable (Expand, Migrate, Contract).
- You cannot roll back a published artifact that other people have already pulled. Withdrawing a bad release means publishing a fixed one and telling consumers (Semantic Versioning, and Where It Stops Applying).
- Automate: building, digest capture, labelling with the commit, publishing, promotion between registries or paths, signature generation and verification.
- Automate the release record — a human decision should produce a machine-readable artifact, not a message in a chat channel.
- Keep human: the decision to release, particularly for changes with migrations, and the choice to promote a candidate that has known issues (Change Management).
- Digest-addressed deployment is unambiguous and unreadable. Humans need tooling to translate a digest into "the change Priya merged on Tuesday".
- Build-once forbids environment-specific build flags, which forces all environment differences into runtime configuration — usually the right outcome, occasionally awkward (Build-Time and Runtime Configuration).
- Retaining artifacts long enough for a safe rollback window costs registry storage, and every retention policy is a bet about how far back you might need to go.
- A formal release step adds latency, which is exactly what continuous deployment removes on purpose. The right amount of formality depends on how reversible your changes are (Continuous Deployment).
Where this applies
This domain is unusually tool- and organisation-dependent. These labels say what each claim is specific to, and what a different platform, provider or organisation does instead.
- GENERALBuild-once, digest identity and a recorded release apply to containers, VM images, language packages and static bundles alike. The mechanism differs — an image digest, a checksum, a lockfile entry — the property does not.
- TOOL-SPECIFICContainer registries expose
name@sha256:...digest references and OCI-style tags; language package registries expose checksums in lockfiles instead, and some allow republishing a version, which breaks the immutability assumption. Check whether your registry actually forbids overwriting a published version before relying on it.
Where the depth lives
This domain teaches delivery and operation, and hands the mechanism off to the domain that owns it.