The question this answers
What states does a compute instance pass through, and at which of them can it silently lose the requests it was supposed to serve?
Instances must be added and removed continuously — for deploys, for scaling, for hardware failure, for spot reclamation — without any client observing a failed request.
A defined progression with explicit gates: traffic arrives only after the workload declares itself ready, and stops arriving before the workload is terminated, with a window in which in-flight work can finish.
Six states, and the two gates that matter
The lifecycle is the same whether the unit is a virtual machine, a container task or a pod, and whether the trigger is a deploy, an autoscaling event or a hardware failure. What changes is the duration of each state, and duration is what decides whether the design works: an instance that takes four minutes to become ready cannot respond to a burst that saturates in twenty seconds.
Two transitions carry almost all the risk. The first is running → in service: the moment the load balancer starts sending real traffic. If the readiness gate is a check that returns 200 from a static handler, traffic arrives before the connection pool is warm, before caches are populated, and sometimes before the process can reach its database — and the first users of every new instance get errors. This is the Liveness vs Readiness confusion, and it turns a deploy into a brief outage on every release.
The second is in service → draining: the moment the instance is told it is going away. If it stops accepting connections immediately, every in-flight request dies. If the load balancer keeps sending traffic after the process has begun shutting down — which happens whenever deregistration is slower than the shutdown signal — new requests arrive at a process that is closing. Both directions of that race are real, and the fix is the same: deregister first, keep serving for a grace period, then exit. See Graceful Shutdown: The 502 Spike Nobody Investigates.
- 1Requestedseconds
Capacity is requested from the provider: a size, an image, a zone, a network placement, an identity.
Insufficient capacity in the chosen family and zone, or an account quota — both fail exactly when demand is highest and everyone else is also scaling.
- 2Booting5 s (container) to 60 s+ (VM)
The host is allocated, the image is fetched if not cached, the guest or container starts.
Image pull failure, a bad image reference, or a large image making this state minutes instead of seconds — see Why Image Size Is an Infrastructure Problem.
- 3Initializing2 s to several minutes
Configuration and secrets are fetched, the identity is assumed, connections are opened, caches are warmed, migrations are checked.
The longest and most variable state, and the one nobody measures. A slow dependency here is what makes scaling arrive late.
- 4Ready · GATE 1one check interval
The readiness check passes and the load balancer registers the instance as a target. Real traffic begins.
A check that only proves the process is alive lets traffic in before dependencies are reachable. Every new instance then serves errors to its first users.
- 5In servicehours to months
Serving requests. Health checks continue; failures remove the instance from rotation without terminating it.
A health check that passes while the workload is degraded — a pool exhausted, a disk full — keeps traffic flowing into a broken instance.
- 6Draining · GATE 210–60 s grace
Deregistered from the load balancer first, then sent a termination signal. In-flight requests are allowed to complete within a grace period.
Exiting immediately on the signal kills in-flight requests. Deregistering after the signal lets new requests arrive at a closing process. Both are common.
- 7Terminatedimmediate
The process exits, the instance is destroyed, local disk is gone, billing stops.
Anything on local disk that was not shipped elsewhere is lost — logs, uploads, queue state. Interruptible capacity reaches this state with as little as ~2 minutes' warning.
Where the requests actually go during a replacement
A rolling replacement is the lifecycle run twice, overlapping. The old instance must remain in service until the new one has passed gate 1, and the new one must not receive traffic until it can serve it. When a deploy drops requests, one of those two conditions was violated, and which one is usually visible from *whose* requests failed.
If errors cluster at the start of each instance's life, gate 1 is wrong: readiness is passing too early. If errors cluster at the end, gate 2 is wrong: draining is too short or the ordering is inverted. If errors span the whole rollout, the surge and unavailability settings allowed both instances to leave at once — the "zero-downtime" rollout that briefly has zero instances.
One more case is worth naming because it is subtle: a client holding a keep-alive connection to an instance that is draining. Deregistration stops *new* connections being routed there; it does nothing about an established one. Unless the server closes idle connections during the grace period — or the load balancer terminates them — the client will happily send its next request down a socket to a process that is exiting. That is the connection-reuse failure that makes rollouts look flaky at low rates that never quite reproduce in testing.
- Errors at the start of an instance's life mean readiness passes too early; errors at the end mean draining is too short or inverted.
- Deregistration does not close established keep-alive connections. The server must close idle ones during the grace period.
- Replacing a large fleet opens a large number of new database connections at once — a connection spike that is a lifecycle artifact, not a traffic one.
- Interruptible capacity compresses gate 2 to roughly a two-minute warning, which is a design constraint, not a detail.
- Anything on local disk at termination is gone. Logs must be shipped continuously, not at shutdown.
The lifecycle is a cost decision as much as an availability one
Billing starts at boot and stops at termination, which means every state before "in service" is paid time during which the instance serves nothing. For a container starting in five seconds this is irrelevant. For a virtual machine that takes ninety seconds to boot and two minutes to warm caches, an autoscaling policy that adds and removes instances aggressively can spend a meaningful share of its compute budget on instances that never served a request.
That is also the argument for keeping warm headroom rather than scaling reactively: paying for idle capacity that can serve immediately is frequently cheaper than paying for capacity that arrives after the burst — and always better for the users during it. See Startup Time & Cold Start and Autoscaling.
The much larger lever in the other direction is the lifecycle *guarantee*. Interruptible capacity is dramatically cheaper and can be reclaimed with a short warning. For CI runners, batch jobs, media processing and anything queue-driven with idempotent work, that is close to free money — the only requirement is that losing an instance mid-task loses no work. For a stateful primary or a long-lived session-holding service it is unusable. Matching the guarantee to the workload is one of the highest-return decisions in the domain, and it is a lifecycle decision.
Bars are relative weights, not currency. Real rates depend on provider, region, commitment and volume.
Key points
- Six states: requested, booting, initializing, ready, in service, draining, terminated — with two gates that decide whether requests are lost.
- Gate 1 (readiness) must reflect the dependencies the request path needs, or every new instance serves errors to its first users.
- Gate 2 (draining) must deregister before signalling shutdown and allow in-flight work a grace period, or every replacement drops requests.
- Deregistration does not close established keep-alive connections; the server must close idle ones itself during the grace period.
- Time-to-ready is the number that decides whether autoscaling can respond to your bursts, and it is paid for while serving nothing.
The loop, answered
Every field is required, which is why no lesson here can recommend something without saying what it costs and what simpler thing to consider first.
- • Capacity is requested from the provider, subject to available capacity in the family and zone and to account quotas.
- • The image is fetched and started; the workload fetches configuration and secrets and assumes its identity.
- • A readiness probe gates registration with the load balancer, so traffic only arrives once the workload declares it can serve.
- • On termination the orchestrator or scaling group deregisters the target, waits for the deregistration delay, then sends a termination signal and finally a kill after the grace period.
- • Interruptible capacity injects a reclamation notice into this sequence with a short fixed warning, which the workload must handle as an ordinary drain.
- • Readiness checks that exercise the real dependencies, and liveness checks that do not — conflating them causes restart loops on a slow dependency.
- • A shutdown handler that stops accepting new work, closes idle keep-alive connections, finishes in-flight requests and then exits.
- • A grace period longer than your slowest normal request, and a deregistration delay long enough for the load balancer to notice.
- • Continuous log shipping, because local disk vanishes at termination and shutdown is the worst possible time to try to flush.
- • Handling of the interruption notice for any workload on reclaimable capacity — the notice is useless if nothing listens for it.
- • Traffic routed to an instance whose readiness check returns a static 200 while its database connection pool is still empty.
- • Requests dropped at every deploy because the process exits on the termination signal without a grace period.
- • A client keep-alive connection sending a request into a draining process, producing a low but persistent error rate that never reproduces in testing.
- • A scale-out that fails because the instance family is exhausted in that zone, during the exact traffic event that triggered it.
- • A large fleet replacement exhausting database connections, because every new instance opens a fresh pool simultaneously.
- • An interruptible instance reclaimed mid-task on a workload that assumed it would not be, losing the work silently.
- • Time-to-ready is the hard limit on how fast capacity can respond; below it, scaling cannot help and only warm headroom can.
- • Replacement rate matters at fleet scale: replacing 200 instances is 200 initializations, and whatever they all do at startup happens at once.
- • Capacity availability, not policy, is the real ceiling during a large scale-out — a zone can be out of a family exactly when everyone wants it.
- • Identity is assumed during initialization; a workload that starts serving before its identity is available fails in ways that look like permission bugs.
- • Secrets are fetched at startup and live in memory for the instance's life, so instance lifetime is effectively secret lifetime — short-lived instances are a security benefit.
- • Termination should destroy local state, and anything sensitive written to local disk needs encryption because instance storage is reclaimed rather than erased.
- • Long-lived instances accumulate drift, unpatched packages and stale credentials — the argument for Mutable Servers and Immutable Images and short lifetimes.
- • Billing runs from boot to termination, so every second before readiness is paid time serving nothing.
- • Warm headroom is deliberate paid idleness bought to remove scaling lag, and it is often the correct trade.
- • Interruptible capacity is the largest available compute discount and is purely a lifecycle decision.
- • Orphaned disks, addresses and snapshots survive their instances and accumulate quietly for years.
- • Time-to-ready as a tracked metric, broken into boot and initialization — you cannot design autoscaling without it.
- • Per-target error rate during rollouts, which localizes a failure to gate 1 or gate 2 by where in the instance's life errors cluster.
- • Termination reasons: scaled-in, health-check failure, hardware event, interruption. They demand different responses and look identical in aggregate.
- • The signal that lies: overall request success rate during a rollout. A 99.6% aggregate can be one instance failing every request it receives for ninety seconds.
- • A platform that manages the whole lifecycle for you — a managed container service or function platform handles registration, draining and replacement, and removes most of these failure modes by construction.
- • Long-lived instances that are patched in place, if replacement is genuinely rare. This trades drift for stability and is defensible for a small stable fleet, though it forfeits the security benefit of short lifetimes.
- • Blue-green instead of rolling replacement, when in-place draining is hard to get right: two complete environments and one traffic switch has a simpler failure model — see Blue/Green: Two Environments, One Switch.
- • For a workload with a single instance and an acceptable maintenance window, a stop-start deploy is honest and simpler than an unreliable "zero-downtime" one.
- • Longer grace periods buy fewer dropped requests and charge slower rollouts and slower incident response, since draining blocks replacement.
- • Warm headroom buys immediate capacity and charges continuous idle spend.
- • Interruptible capacity buys a large discount and charges the engineering work to make every task resumable — work that also improves reliability generally.
What people believe, and what is true
A rolling deploy is zero-downtime by definition.
It is zero-downtime only if readiness reflects real dependencies, draining deregisters before shutting down, and the rollout never removes all healthy targets at once. All three are configuration.
Deregistering from the load balancer stops all traffic to the instance.
It stops new routing decisions. Established keep-alive connections keep delivering requests until someone closes them.
Spot and preemptible capacity is too risky for production.
It is unusable for stateful primaries and excellent for CI, batch, media processing and queue workers. The requirement is idempotent, resumable work — which is worth having regardless.