CDNs: The Networking View

Light in fibre travels at roughly 200,000 km/s — about two thirds of its speed in vacuum, because of the glass’s refractive index. Frankfurt to Sydney is ~16,500 km on the map and more along real cable routes, so one way is ≥ 80 ms and a round trip ≥ 165 ms before a single router has queued anything. New York–London is ~5,600 km: a physical floor of ~56 ms RTT, ~70–75 ms in practice. These numbers are lower bounds set by physics; faster CPUs, more bandwidth and better code do not move them.

A cold HTTPS request from a browser costs: DNS (often cached), one RTT for the TCP handshake (The Three-Way Handshake), one RTT for TLS 1.3 (The TLS Handshake), one RTT for the request and first byte, then further RTTs as TCP slow start ramps up. At 165 ms that is half a second before any content, and a 1 MB page takes several more round trips as cwnd doubles (Congestion Control: Protecting the Network). Halving the distance halves every one of those.

The only way to reduce RTT is to reduce distance. A CDN is a fleet of points of presence (PoPs, edge locations) — hundreds of small data centres inside ISPs and internet exchanges — so that the machine the client connects to is 5–20 ms away instead of 165. Software Architecture teaches when to use one; here we follow the packets.

The client resolves www.example.com. The record is a CNAME to the CDN (example.cdn-provider.net) whose authoritative servers answer differently depending on where the query comes from. Two mechanisms exist. DNS steering returns the IP of a PoP near the resolver that asked — which is near the user if they use their ISP’s resolver, and can be badly wrong if they use a distant public resolver; the EDNS Client Subnet extension lets a resolver pass part of the client’s address to fix that (Following One Lookup Through Every Cache).

Anycast announces the same IP prefix from every PoP via BGP. Each router on the internet forwards to whichever announcement is nearest in its own routing table, so the client’s packets land at the topologically closest PoP without DNS having to guess (Internet Routing: Autonomous Systems and BGP). The subtlety: "nearest" in BGP terms is fewest AS hops, not fewest milliseconds, and a route change mid-connection can move a TCP connection to a different PoP that has no state for it — which is one reason CDN TCP sessions are kept short or QUIC connection IDs are used.

The edge holds the site’s certificate and private key (or a CDN-managed one), so the client’s TLS handshake completes against the edge at short-hop RTT. From the edge to the origin a separate TLS session runs over a persistent, pre-warmed connection pool — the origin handshake was paid once, long ago, not per user. The origin sees the CDN’s IPs and must be told the client’s via X-Forwarded-For or a CDN-specific header, with the same trust rules as any reverse proxy (Forward and Reverse Proxies).

A cache entry is looked up by a cache key: by default host + path + query string, extended by whatever the response’s Vary header names (Accept-Encoding, sometimes Accept-Language). A query string that carries a per-user token, or a Vary: Cookie, makes every request unique and the cache useless. Cache-Control from the origin decides the rest: public, max-age=60 (client and CDN may cache 60 s), s-maxage=3600 (CDN may cache an hour even if browsers may not), private (only the browser), no-store (nobody), stale-while-revalidate=30 (serve the old copy while fetching a new one), immutable (never revalidate — pair it with content-hashed filenames so a change is a new URL).

A cache miss at a busy edge is the dangerous moment: a thousand clients requesting a just-expired object simultaneously would produce a thousand origin fetches. Edges coalesce identical in-flight requests into one, and an origin shield — a designated mid-tier PoP that all edges fetch through — turns N edges’ misses into one origin request. Without shielding, a global site has one origin fetch per PoP per TTL expiry.

A long TTL means more hits and a smaller origin, and a stale object served for up to that long after a change. A short TTL means fresh content and an origin that sees a request per PoP per TTL. Purge (invalidate by URL, tag or everything) breaks the trade: cache long, purge on deploy. Purges propagate to hundreds of PoPs in seconds, not instantly, and a purge storm during a rollback can briefly send every edge to the origin at once — exactly the stampede shielding exists to prevent. Compare cache invalidation in the Database domain: the same problem one layer down.

A CDN cannot shorten a trip it has to make. An uncacheable response — Cache-Control: private on a personalised page, a POST, an API call with a per-user token — still travels edge → origin → edge. What the user gains is smaller than it looks and larger than nothing: the TCP and TLS handshakes are against the near edge (three short RTTs instead of three long ones), the edge→origin connection is already open and its congestion window already large, and the response comes back over a warm path. The one long RTT for the request itself remains. A page that needs five sequential uncacheable API calls from Sydney to Frankfurt pays five long RTTs no matter who is in front of the origin; the fix is fewer round trips or an origin nearer the user (Latency: Same Machine to Cross-Continent).