The Price That Kept Coming Back

Read what each party saw, commit to a cause, and only then find out which of them was right. The root cause, why the obvious reading was wrong, and the fixes are all held back until you have answered.

The symptom

What was reported, before anyone knew what was happening.

A price change from £40 to £30 was applied, the cache was invalidated, and within seconds roughly 15% of page views showed £40 again. Re-invalidating fixed it for a few seconds each time. It took eleven manual invalidations before it stuck.

What each party saw, and what each concluded

The evidence, in the form it actually arrived in — several parties, several partial views, several confident conclusions.

WhoWhat they could actually seeWhat they concludedVerdict
Admin serviceRow updated to £30, then DEL price:sku-991 acknowledged by the cache.The cache no longer holds a stale value.✓ right
Read path A (a request that started before the update)A cache miss at T0, read £40 from a replica, then wrote £40 into the cache at T2 — after the delete at T1.Populating the cache with the value I read.✕ wrong
CacheA delete, then a set with £40.Nothing — both operations were valid and applied in order.✓ right
EngineerInvalidation working and then not working.The cache cluster is dropping our deletes; some nodes are not receiving them.✕ wrong

2 of 4 parties reasoned correctly from what they could see and still reached the wrong conclusion. Nobody in this table is careless. Each one acted on complete-looking local information, and the information was local. That gap — between what a node can observe and what is true — is the whole domain, and one of these readings will usually be yours.

Commit before you read on

Every delete was acknowledged and every one of them worked. Commit before reading on: describe the interleaving that lets a correctly-executed delete be undone, and say why re-running the delete is not a fix.

Write it down even if you are unsure. An unwritten guess quietly becomes “that is what I thought” the moment you read the answer.

Everything below stays hidden until you commit. Being wrong here is the exercise — the answer looks obvious the moment you read it, which is exactly why reading it first teaches nothing.