Distributed Caching
5 lessons. Every one names the guarantee it claims, what a node can know, and how it fails.
A local cache is a variable. The moment the cache is shared between machines, or duplicated on each of them, you have built a replicated store — with copies, staleness, a new network hop, and a new dependency — and you have built it without any of the machinery a replicated store normally comes with.
Q · Where should the cached copy live, and what did putting it there cost me?
The source changed. Which caches are now wrong, and how would they find out? Every answer is a message with a delivery semantics — and the ordering between that message and an in-flight cache fill is where the permanent-staleness bugs live.
Q · A value changed at the source. Which copies are stale, and what actually tells them?
In one process, a stampede is a few threads racing to recompute the same value and a mutex solves it. Across five hundred instances there is no mutex to take, so the database receives every one of those misses at once — for a single key.
Q · A hot key just expired and every instance missed simultaneously. What stops all of them hitting the database?
One product changes and ten thousand derived keys are wrong. A bulk import changes five million rows. And some of the copies are in browsers and CDN nodes you will never reach. At scale, invalidation stops being a correctness mechanism and becomes a best-effort speed-up over a TTL you must be willing to live with.
Q · One source record changed. How many cached things are now wrong, where are they, and can I actually reach them all?
A key lives on one shard. Add a hundred nodes and it still lives on one. When one celebrity, one flash-sale product or one feature flag receives more traffic than a single node can serve, the only options are to replicate the key, move it closer, or stop asking for it.
Q · One key is taking more traffic than any single node can serve. What can I actually do about it?