The question this answers
What does a load balancer own in a deployment, and which of its settings turns a rolling deploy into an outage?
The API runs on instances that are replaced on every deploy and scale in and out during the day. Users must reach a single stable address, must never be sent to an instance that is not ready, and must not notice a deploy.
A stable public entry point in front of an ephemeral fleet, with automatic removal of unhealthy targets, TLS terminated in one place, and registration that follows the fleet as it changes.
External and internal are two different jobs
The one everyone pictures is the external load balancer: a public address in a public subnet, terminating TLS, distributing to private targets. It is the reason the application tier needs no public address at all, and it is the single point where certificates, WAF rules and access logs for inbound traffic naturally belong.
The one that gets forgotten is the internal load balancer, which has no public address and lives entirely inside the private tier. It exists for the same reason as the external one — a stable address in front of things that are replaced — but for service-to-service traffic. Without it, one service holds the addresses of another service's instances, and every replacement becomes a coordination problem. In a container platform this role is filled by the platform's own service abstraction rather than by a separate component. See Service: A Stable Name in Front of Moving Pods.
Both are provider-managed, both scale horizontally without intervention, and both are deployed into subnets in multiple zones. That last point is not a detail: a load balancer with a node in only one zone is a single point of failure in front of a multi-zone fleet, which is a genuinely common way to build a highly available system that is not.
L4 or L7 — a decision about what it is allowed to understand
A layer-4 balancer forwards connections. It sees addresses and ports, picks a target, and passes bytes; it cannot read a path or a header because it never parses the protocol. That makes it fast, protocol-agnostic and capable of preserving the client address naturally — and it makes routing by URL, host or header impossible.
A layer-7 balancer terminates the connection, parses the request, and makes a decision with the whole request in hand. That is what buys path-based routing, header-based routing, cookie stickiness, request-level access logs, retries on idempotent requests and per-request WAF inspection. The cost is that it is now an HTTP participant: it has its own timeouts, its own header handling, its own idea of what a valid request is, and it hides the original client address unless a forwarded-for header is threaded through and trusted correctly.
The practical rule is to use L7 for HTTP APIs and websites, because almost every operational feature you will eventually want is an L7 feature, and to use L4 for non-HTTP protocols, for extreme throughput, or where TLS must pass through untouched to the backend. Mixing them — L4 in front of L7 — is a real pattern for static-address requirements, and it adds a hop that must be accounted for in every timeout budget.
| Capability | Layer 4 | Layer 7 |
|---|---|---|
| Routes on | Address and port only. | Host, path, header, method, query. |
| TLS | Passes through, or terminates without inspecting. | Terminates and can re-encrypt to the backend. |
| Client address | Preserved naturally. | Replaced; needs a forwarded-for header the app must trust correctly. |
| Per-request retries | No — it forwards a connection, not requests. | Yes, for idempotent requests, which hides a single target failure. |
| Access logs | Connection-level. | Request-level: path, status, latency, user agent. |
| Protocols | Anything over TCP or UDP. | HTTP-family, gRPC, WebSocket. |
| Typical use | Databases, message brokers, custom protocols, extreme throughput. | Public APIs and websites — the default choice. |
Registration, health and the deploy that goes wrong
The lifecycle below is where load balancers actually cause incidents, and the two ends are the dangerous ones. At registration, a target that passes its health check before the application can serve real traffic receives requests it cannot answer — which is why the check must exercise a path that fails when dependencies are missing, not a static file that returns 200 as soon as the web server binds. At deregistration, a target removed without draining drops in-flight requests, which appear to users as errors during a deploy that the deploy tooling reported as successful.
The health-check settings are a genuine trade-off rather than a best practice. A short interval with a low unhealthy threshold detects failures fast and also ejects a target that had one slow moment; a long interval with a high threshold is stable and keeps sending traffic to a broken instance for a minute. The setting that matters most is the *healthy* threshold on the way back in, because a target that flaps in and out under load creates a feedback loop where the remaining targets absorb the load, slow down, and fail their own checks.
And the classic outage: a health check that hits an endpoint which itself calls the database. The database has a bad minute, every target fails its check simultaneously, all targets are removed, and the load balancer returns an error for every request — turning a degraded dependency into a total outage. The check should verify that *this instance* can serve, not that the entire system is well. See Liveness vs Readiness and Health Checks.
- 1Registeredseconds
The target is added to the pool by an autoscaling group, a deploy, or the container platform's controller.
Registration lag: new capacity exists and receives nothing while the existing fleet is still saturated.
- 2Initial health checksinterval × healthy threshold
Consecutive successful checks are required before traffic is sent.
A check that passes before the application is ready sends real requests to an instance that cannot answer them.
- 3In servicehours to weeks
Requests are distributed to the target by the configured algorithm.
A slow target keeps receiving its share under round-robin; least-outstanding-requests degrades more gracefully.
- 4Failing checksinterval × unhealthy threshold
Consecutive failures cross the unhealthy threshold and the target stops receiving traffic.
If the check depends on a shared dependency, every target fails at once and the pool empties.
- 5Drainingthe configured drain window
On deregistration, existing connections are allowed to finish while no new ones are sent.
A drain timeout shorter than the longest request drops in-flight work — errors during a "successful" deploy.
- 6Deregisteredimmediate
The target is removed from the pool and can be terminated.
Terminating before draining completes turns a routine replacement into user-visible 5xx responses. See Graceful Shutdown: The 502 Spike Nobody Investigates.
Key points
- External load balancers are the reason application instances need no public address; internal ones do the same job for service-to-service traffic.
- A load balancer with nodes in one zone is a single point of failure in front of a multi-zone fleet.
- L7 buys path routing, request retries, request-level logs and WAF inspection, and costs you the client address unless forwarded-for is handled correctly.
- A health check that calls a shared dependency turns a degraded dependency into a total outage by emptying the pool at once.
- Draining is what makes a deploy invisible; a drain window shorter than the longest request produces errors the deploy tooling never sees.
The loop, answered
Every field is required, which is why no lesson here can recommend something without saying what it costs and what simpler thing to consider first.
- • The balancer is deployed with nodes in several zones and presents one stable address or hostname.
- • Targets are registered by an autoscaling group, a deploy pipeline or a container platform controller as the fleet changes.
- • Health checks probe each target at an interval; consecutive passes admit it and consecutive failures eject it.
- • For each incoming connection or request the balancer selects a healthy target by the configured algorithm.
- • TLS is terminated at the balancer using a managed certificate, and traffic to the target continues over plain HTTP or a re-encrypted connection.
- • Own the health-check path and make it verify that this instance can serve, not that the whole system is healthy.
- • Own the drain window, and set it longer than your longest legitimate request.
- • Own certificate lifecycle — expiry is a total outage at the entry point, and automated renewal must be verified, not assumed.
- • Own the timeout hierarchy: the balancer's idle timeout, the application's timeout and the client's must be ordered deliberately or connections are reset mid-response.
- • Own zone coverage, and confirm the balancer has nodes in every zone its targets live in.
- • All targets unhealthy: the balancer answers every request with an error, which reads as a total outage while every instance is running fine.
- • A health check passing too early, so a fraction of requests hit an instance whose dependencies are not connected yet.
- • No draining on deregistration, producing a burst of errors on every deploy that the pipeline reports as green.
- • An expired certificate — the entry point stops working entirely, and no amount of backend health helps. Certificate expiry is the one entry-point failure that is fully predictable and still routinely missed.
- • Idle-timeout mismatch: the balancer closes a connection the application still considers open, surfacing as intermittent resets under low traffic.
- • Managed balancers scale their own capacity, but the ramp is not instantaneous — a step-function traffic spike can outpace it.
- • Connection and request rate limits exist and are usually generous; the real ceiling is the target fleet behind it.
- • Cross-zone distribution evens out load across unevenly sized zones, at the cost of cross-zone data transfer charges.
- • The algorithm matters under saturation: least-outstanding-requests degrades far more gracefully than round-robin when one target is slow.
- • A public load balancer on 443 is the design, not a finding — this is the canonical example of exposure judged in context.
- • It is the natural TLS boundary: one place for certificates, cipher policy and protocol versions. See TLS as a Security Boundary.
- • Its security group should permit only 443 from the internet, and the target group should permit only the balancer's group — this pairing is what keeps the fleet unreachable.
- • Access logs at the balancer are the request-level record of everything that reached the system, and they belong in an immutable store.
- • Two meters: an hourly charge for the balancer and a usage charge based on connections, new connections, bandwidth and rule evaluations.
- • Cross-zone distribution can add data-transfer charges depending on provider and configuration.
- • The usage meter is unusual in that it counts several dimensions at once, so a workload with many short connections costs differently from one with few long ones.
- • An internal balancer per service adds up: in a container platform, the platform's own service abstraction is usually the cheaper equivalent.
- • Healthy target count — the first number to look at in any entry-point incident, and the one that reveals a health-check-induced outage instantly.
- • Balancer-generated error codes separated from target-generated ones; the distinction says whether the problem is in front of or behind the balancer.
- • Request latency measured at the balancer, which includes queueing the application never sees.
- • The signal that lies: the balancer's own state, which stays healthy and continues answering while returning an error for every single request.
- • A single instance with a static address, for an internal tool with one user and no availability requirement — a balancer in front of one target buys nothing.
- • The container platform's built-in service and ingress, when you already run one; a second load-balancing layer is duplicated machinery. See Ingress and Gateway: Getting Traffic In.
- • DNS multi-value answers, for cheap spreading across a couple of stable endpoints with no health-based ejection.
- • A CDN as the public entry point, when the traffic is mostly cacheable — it fronts the origin and reduces what reaches the balancer at all.
- • Buys a stable entry point and automatic ejection of bad targets; costs a component that can fail closed and take everything with it.
- • L7 buys routing and observability; costs latency, a new HTTP participant in the path, and client-address handling you must get right.
- • Aggressive health checks buy fast failure detection; cost stability, because they eject targets that were merely slow.
What people believe, and what is true
A load balancer makes the system highly available.
It removes one single point of failure and adds itself. Availability comes from targets in several zones, a balancer with nodes in those zones, and health checks that eject only what is actually broken.
A deeper health check is a better health check.
A check that calls a shared dependency fails on every target simultaneously when that dependency degrades, converting partial degradation into total outage.
Deploys are safe because the balancer handles it.
Only with a drain window longer than the longest request and a readiness check that does not pass before the application can serve.