Continuous Deployment
Every change that passes verification reaches production automatically, with no human release step — which is a different and stronger claim than continuous delivery.
The question, the obvious approach, and why it breaks
Every lesson starts where the work starts: an operational problem, a first attempt that is entirely reasonable, and the way production disagrees with it.
What changes when the pipeline deploys to production with no human in the path, and what has to be true first?
A human release step is a gate that adds latency and, in most organisations, adds no scrutiny — the person approving cannot evaluate the change any better than the pipeline can. But removing it makes the automated verification the last line of defence, and most verification is not built to be that.
Continuous deployment is the natural end state of continuous delivery: turn off the approval step and let the pipeline deploy. The tests already pass, so the human was only clicking a button.
The human was often doing something the pipeline does not: noticing that this particular change is unusual, or that a migration is involved, or that it is Friday afternoon before a holiday. That judgement disappears with the button.
- The human was often doing something the pipeline does not: noticing that this particular change is unusual, or that a migration is involved, or that it is Friday afternoon before a holiday. That judgement disappears with the button.
- Verification that was good enough as one signal among several is now the only signal. Every gap in it becomes a direct path to production.
- Without progressive rollout and automated rollback, continuous deployment means a bad change reaches everyone at machine speed (Progressive Delivery: Exposure as a Dial).
- Schema changes do not fit the model without extra rules, because a migration is not reversible in the way a deploy is (A Migration and a Deploy Are One Event).
- Deploy frequency rises sharply, so anything that was tolerable once a week — a brief connection drop, a cache flush, a warm-up period — now happens many times a day (Draining: Stopping Without Dropping).
What is actually happening
Underneath the tooling, which is the part that survives a change of tool.
- Continuous delivery is a property: any commit could be released. Continuous deployment is a policy: every commit that passes verification is released, automatically. The first is a capability, the second is a decision to use it without asking.
- The policy is only safe when the pipeline's verification is trusted to be the final gate, which is a much higher bar than "the tests pass". It requires that the failure modes you care about are detectable automatically, before enough users are affected to matter.
- The mechanism that makes it work is not better testing. It is moving the last verification into production, on a small population, with automatic reversal: deploy to a canary, compare against a baseline, promote or roll back without a human (Canary Analysis: Compared Against What?).
- That turns the safety question from "can we predict this change is safe" — which is not answerable — into "can we detect that it is not, fast enough, and stop it automatically".
- It also depends absolutely on Deployment Is Not Release. Deploying every change automatically is only tolerable if deploying is not releasing, so unfinished work is inert and product exposure stays a separate, human decision.
- The prerequisites are cumulative, not optional. Removing the human gate from a pipeline that lacks them does not produce continuous deployment; it produces an unattended path to production.
The distinction, stated precisely
These two are conflated so routinely that it is worth putting them side by side and being pedantic about it. One is a capability; the other is a policy that uses the capability.
| Continuous delivery | Continuous deployment | |
|---|---|---|
| Claim | Any commit could be released | Every verified commit is deployed |
| Type of claim | A property of the main line | A policy about the pipeline |
| Human in the path | Yes — someone decides to release | No — the pipeline decides |
| Last line of defence | The person deciding, plus the pipeline | The pipeline's verification and its automatic rollback |
| Requires | Automated checks, promotable artifact, rehearsed deploy | All of that, plus canary analysis, baseline comparison and automated rollback |
| Failure of the gate looks like | A delayed release | A bad change in production, unattended |
| Relationship | The prerequisite | One thing you can do with it |
| Common confusion | Called "CD" and assumed to mean the other | Claimed by teams who deploy manually but often |
What has to be true before you remove the button
Each prerequisite exists because a specific thing the human was doing has to be replaced. Removing the gate without replacing the function is the failure this list prevents.
- 1Releasable main line
Ensures the thing being deployed automatically is deployable at all.
fails by Automating a pipeline whose green state is not trustworthy (Continuous Delivery).
evidence Arbitrary commits have been released successfully as normal practice.
- 2Deployment separated from release
Makes deploying unfinished work harmless.
fails by Half-built features becoming visible because the flag only guarded part of the path (Deployment Is Not Release).
evidence Flags default off; incomplete work is in production and inert.
- 3Progressive rollout
Limits how many users meet a bad change.
fails by Deploying to the whole fleet at once, so detection and impact are simultaneous (Canary: One Percent, Then Five, Then Watch).
evidence A canary stage exists and has stopped a rollout.
- 4Baseline comparison
Replaces the human noticing something looks wrong.
fails by Fixed thresholds instead of a comparison, so a general degradation reads as healthy (Canary Analysis: Compared Against What?).
evidence Canary verdicts are computed against the concurrently running previous version.
- 5Automatic rollback
Replaces the human deciding to stop.
fails by Detection without action, so the alert fires and the rollout continues (Rollback: Only Useful If It Is Actually Safe).
evidence It has fired on a real regression and the fleet returned to the previous version unattended.
- 6Migration rules
Prevents the pipeline from automating an irreversible change.
fails by A destructive statement deploying itself at 2am (Destructive Migrations).
evidence The pipeline detects destructive migrations and requires an explicit approval.
- 7Observability of the pipeline
Makes the now-unattended path visible.
fails by A pipeline nobody watches deploying to production continuously (CI/CD Anti-Patterns).
evidence Deploy events are annotated on the same dashboards as service metrics (Deploys on the Same Timeline as the Symptom).
Read the middle column as the point: every prerequisite replaces something the removed human was doing. A team that can name which function each one replaces is ready; a team that removed the button to go faster is not.
What actually breaks
These failures are specific to the unattended path. Each is survivable in a manual pipeline because a person notices; here, nobody is looking.
| Trigger | Symptom | Cause | Response |
|---|---|---|---|
| Canary compared to a fixed threshold | A degraded release promotes cleanly; everything is slow and nothing failed | No baseline, so a general regression is invisible to the comparison | Compare against the previous version running concurrently, on the same traffic (Canary Analysis: Compared Against What?) |
| Noisy rollback metric | Versions flap; the fleet holds two versions indefinitely | Threshold tuned below the metric's natural variance | Require a sustained breach and a minimum sample; hold the rollout rather than oscillating |
| Deploy rate exceeds signal latency | Metrics move but cannot be attributed to a version | The next deploy starts before the previous one is conclusive | Enforce a minimum soak per deploy, or batch deliberately at a rate the signal supports |
| Destructive migration in an automated release | Schema change applied unattended; rollback impossible | The migration path was not given its own rules | Detect destructive forms in CI and require approval (Destructive Migrations) |
| Pipeline change deploys itself | The pipeline is broken and is also the tool needed to fix it | The pipeline treats its own definition as ordinary code with no extra care | Keep a manual path to production that does not depend on the pipeline, and test it (Break-Glass Access) |
| Correct change, wrong judgement | All checks green; the change should not have shipped for a non-technical reason | The removed human was applying judgement no check encodes | Route the classes where judgement matters to an explicit approval; automate everything else (Change Management) |
How to do it properly
Most important first.
- Establish continuous delivery first, and confirm it honestly. Continuous deployment on top of a main line that is not reliably releasable automates the wrong state (Continuous Delivery).
- Make production the last test environment, with the blast radius controlled instead: canary, small cohort, automatic comparison against a baseline (Canary: One Percent, Then Five, Then Watch).
- Automate rollback on a metric breach, and make the decision threshold explicit and reviewed rather than implicit in a dashboard.
- Separate release from deployment absolutely. Every incomplete change ships behind a default-off flag (Feature Flags: Deploy Is Not Release).
- Give migrations their own rules: never automatic for a destructive step, and always sequenced so the previous artifact still works (Destructive Migrations).
- Keep an explicit stop control — a way to halt the pipeline that anyone can use, that does not require a code change (The Agent Kill Switch is the same idea for a different subject).
- Instrument the pipeline itself, because it is now a production system with a production blast radius (CI/CD Anti-Patterns).
How much can this affect
Every production change has a blast radius. Stated as a scale so it is comparable between changes rather than adjectival — and paired with what actually contains it, because a wide scope with a real containment mechanism is a different situation from a wide scope with none.
The canary population and the automatic rollback, together — and only together. Without the automated comparison the scope is everyone, at machine speed, with nobody watching.
What can go wrong
- Automatic rollback triggered by a noisy metric, producing a flap between two versions and a fleet in an indeterminate state.
- Canary analysis with no valid baseline — comparing the canary against a fixed threshold rather than against the current version, so a general degradation looks like a healthy canary (Canary Analysis: Compared Against What?).
- A change that passes every automated check and is wrong in a way no check covers: a copy change with a legal implication, a pricing rule, a data-retention default. These are exactly the ones a human gate was catching.
- The pipeline itself becoming the outage: a bad pipeline change deploys itself, and the mechanism you would use to fix it is the broken one (CI Security).
- Deploy frequency exceeding the rate at which a signal becomes conclusive, so each deploy is still unverified when the next one starts and attribution collapses (Change Correlation).
- The mitigation failing: an automatic rollback that rolls back the artifact and not the schema, or not the flag, leaving a half-reverted system (A Migration and a Deploy Are One Event).
- "Continuous delivery and continuous deployment are the same thing." They are not, and the conflation is the most common error in this whole area: delivery is being able to release any commit; deployment is doing it automatically. You can have the first without the second, and the second is meaningless without the first.
- "Continuous deployment means no human ever looks at production." It means no human is in the *release path*. Someone still owns the service, watches the signals and gets paged (Shared Ownership).
- "It is riskier because changes go out unreviewed." Changes are reviewed — at merge. What is removed is a second approval after the fact, which in most organisations was not adding scrutiny (Review as a Gate).
- "We deploy to production several times a day, so we do continuous deployment." Not if a person triggers each one. That is continuous delivery exercised frequently, which is a perfectly good place to be.
- "If the tests pass it is safe." The practice does not rest on tests being sufficient. It rests on production verification with a small blast radius and an automatic reversal (A Successful Deploy Is Not Evidence of a Healthy System).
Operating it
Evidence is the signal, not the intention. Rollback is sometimes 'you cannot, and that is the point'.
- Changes reach production without a human action, and you can show the median time from merge to serving traffic.
- The automated rollback has fired on a real regression, correctly, and you can point at the incident where it did.
- Canary decisions are made against a live baseline, and you can show a case where the canary was stopped.
- Change failure rate is measured over time and is not rising as frequency rises — read as a signal about the pipeline, never as a number to optimise (Learn and Improve).
- Every deploy is annotated in the observability system, so a metric change can be correlated with a version automatically (Deploys on the Same Timeline as the Symptom).
- Rollback must be automatic and fast, because there is no human watching the deploy that would otherwise notice. That is the trade: you removed the human from the release and must therefore add them back as an automated detector.
- The rollback path must cover everything the deploy changed — artifact, config and flags together — or it produces a state that was never tested (A Config Change Is a Production Change).
- Rolling back the *policy* is straightforward and worth knowing: reinstate a manual approval on the production stage. That is a config change to the pipeline, not an architectural retreat.
- For anything with no rollback — destructive migrations, irreversible external side effects — the policy must have an explicit exception, enforced by the pipeline rather than by convention (Policy as Code).
- Automate the deployment decision, the canary comparison and the rollback. That is the substance of the practice.
- Automate the exceptions too: a pipeline that detects a destructive migration or a flagged high-risk path and requires an approval is far more reliable than a rule people remember (Guardrails, Not Gates).
- Keep human: the release decision (exposure to users), the risk acceptance for changes the pipeline flags, and the choice of rollback thresholds (The Automation Trap).
- It demands verification and observability that most teams do not have, and building them is a larger investment than removing the approval step.
- Higher deploy frequency multiplies every per-deploy cost: connection churn, cache warm-up, pipeline compute, observability cardinality (Cardinality: The Label That Took Down Monitoring).
- It concentrates trust in the pipeline, which becomes a high-value target and a single point of failure (The Delivery Chain as Attack Surface).
- For some products the automated gate genuinely cannot cover the risk — safety-critical, financial settlement, regulated content — and the honest answer is continuous delivery with a real approval, not a ceremonial one.
Where this applies
This domain is unusually tool- and organisation-dependent. These labels say what each claim is specific to, and what a different platform, provider or organisation does instead.
- ORG-SPECIFICWhether an automated path to production is permissible is a governance question before it is a technical one. Regulated environments often mandate a separation between the person who authors a change and the person who authorises its release; that requirement is satisfiable with automated evidence and a second approver at merge, but it is not satisfiable by removing approval entirely.
- GENERALThe prerequisite list — releasable main line, flags, progressive rollout, baseline comparison, automatic rollback, migration rules — is stack-independent. Only the tools change.
- SCALE-SPECIFICBelow a certain traffic level canary analysis stops working, because a 1% cohort does not generate enough events for a difference to be distinguishable from noise. Small services often need a longer soak, a larger cohort, or a human looking — which is continuous delivery again.
Where the depth lives
This domain teaches delivery and operation, and hands the mechanism off to the domain that owns it.