Distributed DatadebuggingBeginner
I clicked Cancel and the order still says Confirmed
Symptoms
- After cancelling an order the order page still shows
CONFIRMEDfor one to three seconds; a refresh later showsCANCELLED. - Users click Cancel again and get "order already cancelled" — support calls it "the cancel bug".
- Worse under load; never loses data — the order is always cancelled eventually.
- The command endpoint returns 202 in under 20 ms; the "bug" is entirely in what the user sees next.
# one user action, traced
12:40:01.000 POST /orders/9921/cancel → order-svc (write model) 18 ms → 202 {"status":"accepted"}
12:40:01.020 order-svc INSERT order_events (OrderCancelled 9921) + outbox row commit
12:40:01.060 GET /orders/9921 → order-query-svc (read model: orders_view) → {"status":"CONFIRMED"} ← UI navigates here right away
12:40:01.510 outbox-relay publish OrderCancelled 9921 (relay polls every 500 ms)
12:40:02.140 projector consume OrderCancelled → UPDATE orders_view SET status='CANCELLED', version=7
12:40:03.100 GET /orders/9921 (user pressed refresh) → {"status":"CANCELLED"}
# projector lag (write commit → read model updated): p50 600 ms · p99 2.8 s
# frontend: on 202 → router.push('/orders/9921') → fetch read modelInvestigate
Inspect areas in any order (0/6 inspected). When you think you know the root cause, commit to it.
Projector lag
The frontend flow
Outbox relay interval
Database replication
HTTP caching on the query service
The API contract