Production Networking
The operational half of the network: service discovery, DNS behaviour under change, certificate lifecycles, load balancer health and draining connections without dropping work.
Instances appear and disappear continuously, so callers cannot hold addresses. Discovery is a registry plus a health signal plus a propagation delay — and the delay is where the incidents are.
A DNS change is not an action, it is an expiry schedule. TTL decides how long the old answer keeps being used, and several caches between you and the user do not necessarily obey it.
Issue, deploy, renew, rotate, revoke. A certificate is the only production component with a hard expiry date, which is why expiry remains one of the most common outages in the industry.
Certificate renewal is the textbook case for automation — predictable, recurring, error-prone by hand. It is also the textbook case for monitoring the automation, because silent renewal failure is how certificates expire anyway.
The algorithm matters less than the health check, the connection lifetime and the capacity that remains when a backend leaves. Most balancing incidents are about membership, not distribution.
Stop new connections, let active work finish, then exit. The whole difficulty is ordering — the instance must leave the routing layer before it stops serving, and those two events are not naturally sequenced.
The ingress is where everyone's traffic meets one shared configuration. It is the component with the widest blast radius per line of config, and the one most often changed by people who own only one route.
A catalogue: DNS, certificates, blocked ports, security group mistakes, connection exhaustion, NAT port exhaustion, packet loss and latency spikes — each with a symptom that identifies it.