ReleaseGENERALTOOL-SPECIFIC
The Release Manifest
One record naming the version, commit, artifact digest, config version, migration version and flag state — so "what is production right now" is a lookup rather than an investigation.
The question, the obvious approach, and why it breaks
Every lesson starts where the work starts: an operational problem, a first attempt that is entirely reasonable, and the way production disagrees with it.
The production question
What single record would let you answer "what exactly is running in production" without asking anyone?
The problem
The state of production is assembled from a deployment system, a registry, a config store, a migration table and a flag service. During an incident, reconstructing it across five systems is the first thing anyone does and the last thing they should have to.
What teams do first
We know what is deployed — the deployment system shows the current version. If we need more detail, the CI run has everything.
How it breaks
The deployment system knows what it applied, not what is running. A stalled rollout, a cached image or a failed instance replacement all produce a difference it does not report (What Can Fail Between Commit and User).
How it breaks in production
- The deployment system knows what it applied, not what is running. A stalled rollout, a cached image or a failed instance replacement all produce a difference it does not report (What Can Fail Between Commit and User).
- A version tag is not an identity. Tags move, so "v2.4.1" today and "v2.4.1" last week can be different bytes (Tags Versus Digests).
- Configuration usually changes on its own path, so knowing the artifact tells you nothing about what the process actually loaded (A Config Change Is a Production Change).
- Schema version is invisible from the application side entirely, and it is the input that most often makes a rollback unsafe (A Migration and a Deploy Are One Event).
- Flag state is production behaviour that no deploy record mentions, which is why incidents so often begin with "nothing was deployed" (Deployment Is Not Release).
- CI run history is retained on the CI system's schedule, and that schedule is shorter than the age of the oldest artifact you might roll back to.
Code→Build→Test→Artifact→Release→Deploy→Run→Observe→Operate→Incident→Recover→Learn→Improve
What is actually happening
Underneath the tooling, which is the part that survives a change of tool.
- A release is not an artifact; it is the set of inputs that were true together — code, artifact, configuration, schema and flags. The manifest is that set, written down once, at release time, as a durable record.
- It answers three questions that are otherwise expensive: what is running, what changed, and what is a safe thing to go back to.
- The rollback target is the field people miss, and it is the most valuable one. "The previous release" is ambiguous; a named prior release known to be compatible with the current schema is not (Rollback: Only Useful If It Is Actually Safe).
- The manifest is written by the pipeline as a side effect of releasing, not by a person afterwards. A record that requires discipline to maintain is a record that is wrong exactly when it matters (The Deployment Pipeline).
- It must be stored outside the systems it describes. A manifest that lives only in the deployment system is unavailable during a deployment-system outage, which is one of the times you need it most.
- The running process should be able to report its own view of the same fields — version, config version, schema version — so that "what the manifest says" and "what is actually running" can be compared rather than assumed identical (From Developer to Users).
One record, six inputs
The shape matters less than the coverage. What follows is a plausible record; the test is whether every input that can independently change production appears in it.
A release manifest
1{2 "release": "r-2026-08-26.3",3 "service": "checkout-api",4 "created_at": "2026-08-26T09:14:22Z",5 "created_by": "pipeline/run-4821",6 "source": { "commit": "a1b2c3d", "branch": "main", "pr": 5512 },7 "artifact": { "repository": "registry.internal/checkout-api",8 "tag": "2.4.1",9 "digest": "sha256:9f3e7c1b..." },10 "config": { "version": "cfg-2026-08-24.1", "checksum": "sha256:41ab..." },11 "schema": { "version": "0142", "applied_at": "2026-08-26T09:11:03Z",12 "destructive": false },13 "flags": { "checkout_v2_reads": { "state": "on", "cohort": "10%" },14 "new_pricing_rules": { "state": "off", "cohort": null } },15 "rollback": { "target": "r-2026-08-25.7",16 "safe": true,17 "reason": "schema 0142 is additive; 0141 artifact tolerates it" },18 "gates": { "tests": "pass", "scan": "recorded", "canary": "pass" }19}Three fields carry most of the value. digest makes the artifact identifiable rather than merely named. schema.destructive decides whether rollback is even available. rollback.safe is a computed claim with a stated reason, which is the difference between a rollback plan and a hope.
What each field is for
GENERALThe fields are universal; their sources are not.
schema.version may come from a migration tool's table, a platform release phase or a separate job, and flags from a flag service or a config file — the manifest is where they are brought together, wherever they come from.A field earns its place by answering a question someone actually asks during an incident. If you cannot name the question, drop the field — length is the enemy of a record that has to be read under pressure.
| Field | Question it answers | What goes wrong without it |
|---|---|---|
release | What do we call this, out loud, on a call? | People refer to changes by commit prefix or by "the one from this morning" |
source.commit | What code is this? | Diffing between releases requires reconstructing the range |
artifact.digest | Which exact bytes are running? | A moved tag means the same version name is different software (Tags Versus Digests) |
config.version | What settings did the process load? | A config-only change causes an incident with no deploy to correlate (Configuration Drift) |
schema.version | What shape is the data in? | Rollback attempted against an incompatible schema, during an incident |
schema.destructive | Is rollback available at all? | Discovered mid-rollback, which is the worst possible moment (Destructive Migrations) |
flags | What behaviour is exposed, to whom? | "Nothing was deployed" while user-visible behaviour changed (Deployment Is Not Release) |
rollback.target | Where do we go back to? | "The previous release" is ambiguous when several things changed independently |
gates | What was actually verified? | No way to tell later whether a stage was skipped (The Deployment Pipeline) |
How manifests go wrong
A manifest fails quietly. It keeps being emitted, it keeps looking right, and it stops describing production — which is worse than having none, because people trust it.
| Trigger | Symptom | Cause | Response |
|---|---|---|---|
| Emergency release bypasses the pipeline | The riskiest change in the quarter has no record | The manifest is a pipeline side effect and the pipeline was skipped | Make the emergency path emit a manifest too, even a minimal one (Break-Glass Access) |
| Config changed out of band | Manifest says one config version; the process loaded another | Config has a path to production that does not produce a record | Treat config as a deployable with its own release (A Config Change Is a Production Change) |
| Rollout stalls | Manifest describes a release only part of the fleet is running | The manifest records intent and was never reconciled with outcome | Compare manifest against the versions instances report, continuously (A Successful Deploy Is Not Evidence of a Healthy System) |
| Flags changed after release | Flag snapshot is stale; behaviour differs from the record | A snapshot is a point in time, by definition | Read the snapshot as a starting point and the audit trail for changes since (The Audit Trail) |
| Manifest stored in the deployment platform | Unavailable during a platform incident | The record depends on the system it describes | Store it independently, with its own availability (Audit Logs for Privileged Actions) |
| Unvalidated rollback target | Rollback initiated, fails, incident extends | The field was recorded but never checked against the current schema | Compute rollback safety at release time and re-check it after every migration |
How to do it properly
Most important first.
- Emit a manifest for every production change, including config-only and flag-only changes. Changes without a record are the ones that make incidents long.
- Include the artifact digest, not only the tag. The digest is the identity; the tag is a label that can move.
- Include the config version and the schema version, because those are the two inputs that most often differ from what the artifact expects.
- Include a flag snapshot at release time. Flags change independently, so the snapshot is a starting point that later audit entries build on (The Audit Trail).
- Include the rollback target and whether it is believed safe — particularly whether a migration since then makes the previous artifact incompatible.
- Expose the same fields from the running process, so drift between the record and reality is detectable (A Successful Deploy Is Not Evidence of a Healthy System).
- Store manifests durably and independently, with a retention longer than your longest plausible rollback distance (Artifact Retention).
How much can this affect
Every production change has a blast radius. Stated as a scale so it is comparable between changes rather than adjectival — and paired with what actually contains it, because a wide scope with a real containment mechanism is a different situation from a wide scope with none.
Blast radius if this is wrongEveryone
One testEveryone
What contains it
The manifest contains nothing directly — it is a record. Its absence widens the effective radius of every other change by extending time to diagnosis, which is the part of an incident most under your control.
What can go wrong
Failure modes, including of the mitigation
- A manifest written for standard releases and skipped on the emergency path, so the one release you most need a record of has none (Break-Glass Access).
- A manifest that records intent rather than outcome — written before the rollout, never reconciled against what actually became healthy.
- Config or flags changed after the release with no corresponding record, so the manifest is accurate at release time and misleading an hour later.
- Manifests stored inside the deployment platform, unavailable exactly when that platform is the problem.
- A rollback target field that is never validated, so it names a release whose schema no longer exists.
- The mitigation failing: a manifest so detailed nobody reads it during an incident. The top five fields should be legible at a glance.
Misreads this invites
- "The deployment system already has this." It has what it applied. The manifest records what the release was, including inputs the deployment system never sees (A Config Change Is a Production Change).
- "Git history is the record." Git tells you what changed in code. It says nothing about config, schema, flags, or which artifact is actually serving.
- "A version number is enough." A version number is an index into the record, not the record. Its usefulness depends entirely on something else holding the details.
- "We can reconstruct it if we need to." You can, and you will do it under time pressure with an unknown deadline. That reconstruction is the cost the manifest removes.
Operating it
Evidence is the signal, not the intention. Rollback is sometimes 'you cannot, and that is the point'.
How you know it worked
- Given a timestamp, you can produce the exact manifest that was current, without querying five systems.
- The digest in the manifest matches the digest the running process reports, checked automatically rather than by hand.
- The last three incidents did not begin with reconstructing what was deployed.
- Every production change in the last month has a manifest, including config-only and flag-only ones.
- The rollback target named in the current manifest has been checked for schema compatibility, automatically.
How you get back
- The manifest is what makes rollback a defined operation: restore this named set, not "the previous thing".
- It also makes rollback *decidable*: if the schema version has moved since the target release, the manifest tells you before you try it (Destructive Migrations).
- Rolling back the manifest itself is meaningless — it is a record, and records are append-only. A rollback produces a new manifest describing the restored state, which is the correct behaviour and worth being explicit about.
What to automate, and what stays human
- Automate generation entirely. Every field is available to the pipeline at release time; none should be typed by a person.
- Automate the comparison between manifest and reality, and alert on divergence — that check catches stalled rollouts, out-of-band config changes and manual production edits (Manual Production Changes).
- Automate the rollback-compatibility check, so the target is known-good before an incident rather than during one.
- Keep human: reading it. The manifest is an input to a decision, not a decision.
What this costs
- Assembling the fields means the pipeline must know about config, schema and flags — coupling the release process to systems it might otherwise ignore.
- Durable independent storage is another system to run, back up and secure (Audit Logs for Privileged Actions).
- A flag snapshot is a point-in-time value that goes stale immediately; it is a starting point, and treating it as current state is a real risk the format should discourage.
- Emitting a manifest for every change including trivial ones creates volume, which is the price of not having to decide which changes matter in advance.
Where this applies
This domain is unusually tool- and organisation-dependent. These labels say what each claim is specific to, and what a different platform, provider or organisation does instead.
- GENERALThe fields are the same on any platform: source, artifact, config, schema, flags, rollback target. Only where each value comes from changes.
- TOOL-SPECIFICSome platforms provide part of this natively — a revision history with the applied spec, or a release object with an immutable digest. None cover schema and flag state, because those live outside the deployment system, so the manifest is always partly yours to assemble.
Where the depth lives
This domain teaches delivery and operation, and hands the mechanism off to the domain that owns it.
Observability & Performancedeployment-markers