Latency: Same Machine to Cross-Continent

The interactive uses illustrative values chosen to be typical; real numbers depend on hardware, provider, path and load, and the only way to know yours is to measure. With that said, the ladder is remarkably stable across environments because each rung is set by a different physical fact: loopback by kernel stack cost, in-DC by switch hops and cable length, in-region by the distance between availability zones, cross-region and cross-continent by the great-circle distance in fibre.

What matters is not the values but the ratios: each rung is roughly 5–50× the previous. A design that tolerates a thousand round trips at 50 µs (50 ms total) tolerates forty at 1 ms (40 ms) and two at 150 ms. Distance does not make individual requests slower in a way you can optimise; it multiplies the number of round trips by a constant you cannot change.

Light in fibre travels at about 200,000 km/s, two thirds of c, because of the glass’s refractive index. A round trip over d kilometres of fibre therefore takes at least 2d/200,000 seconds: 10 µs per kilometre, round trip. New York to London is ~5,600 km on the map, giving a floor of ~56 ms; real cables are longer and add amplifiers and routers, so ~70–75 ms is what you see. Frankfurt to Sydney: ~16,500 km, a floor of 165 ms, ~250 ms in practice through Singapore.

Each rung also adds equipment delay — a switch forwards in a few microseconds, a router in tens, and a queue in front of a congested link in milliseconds (Where the Time Goes: The Request Timeline) — but at continental distance propagation dominates everything. This is a lower bound: no faster server, no better protocol, no more bandwidth reduces the time a photon needs to cross the Atlantic. Faster is only possible by being closer, which is the entire reason a CDN exists (CDNs: The Networking View).

No request costs less than one RTT: the bytes go there and an answer comes back. A cold HTTPS request costs more: a DNS lookup (often cached, otherwise ≥ 1 RTT to the resolver), one RTT for the TCP handshake (The Three-Way Handshake), one for TLS 1.3 (The TLS Handshake; two for TLS 1.2), then one for the request and response — three to four RTTs before the first byte. A warm request over a pooled keep-alive connection (Keep-Alive and Connection Reuse, Connection Pooling) costs one. Across an ocean that is the difference between ~180 ms and ~700 ms for the same call, which is why connection reuse matters more the further away the peer is.

Larger responses add round trips of their own: TCP slow start begins at ~10 segments (~14 kB) and doubles every RTT (Congestion Control: Protecting the Network), so a 1 MB response over a fresh connection needs about seven RTTs of ramp-up regardless of bandwidth — 700 ms at 100 ms RTT, 7 ms at 1 ms. Bandwidth vs Latency works the arithmetic.

The N+1 pattern — one query for a list, then one query per item — costs N+1 round trips. Against a database in the same rack at 0.2 ms, 40 items is 8 ms and nobody notices. Move the database to another region at 100 ms and the identical code takes 4.1 s. The ORM did not change; the SQL did not change; the round-trip count was always 41, and each one is now 500× more expensive. The Database domain treats this as a query-shape bug; from here it is a latency-multiplier bug, and the fix is the same: fewer round trips — one query with a join or an IN list, a batch API, a pipeline that sends several commands without waiting.

The same shape hides in microservice chains (service A calls B calls C, each in turn), in sequential API calls from a browser, in a Redis client that does not pipeline, and in any loop that awaits a remote call per iteration. Concurrency helps only when the calls are independent: 40 parallel calls cost ~1 RTT of wall-clock time plus queueing, 40 sequential ones cost 40. The rule that survives every environment is: count the sequential round trips in the critical path, multiply by the RTT of the farthest hop. That number is your floor, and it is usually the bill.