The LLM is unavailable.Investigate with fundamentals.
DatabaseIntermediate
Requests occasionally return stale data
Ticket: since Tuesday, about 2% of profile saves "don’t stick" — the user changes their display name, the page reloads and shows the old name, and a second reload shows the new one. PATCH /me returns 200 every time; nothing is in the error logs.
What you know
profile-svcreads and writes auserstable in Postgres: one primary, two asynchronous streaming replicas. The service has had adb.reader()/db.writer()split since Q1.- Tuesday’s release included PR #4127, "route all GET traffic through the reader pool", to take load off the primary ahead of a marketing push.
- The web client sends
PATCH /me, waits for the 200, then immediately sendsGET /meto re-render — so the read follows the write by roughly 150–200 ms. - A nightly-style batch job runs on the primary from 11:00 to 11:10 and again at 16:00; support tickets cluster in those windows.
- Available: replica lag per replica, Redis hit rates, CDN logs, per-request logs that record which pool served each query.
# request log, one affected user (uid 7731), Wednesday 11:02
11:02:14.201 PATCH /me uid=7731 db=primary UPDATE users SET display_name='Ada L.' WHERE id=7731 200 (14 ms)
11:02:14.377 GET /me uid=7731 db=replica-b SELECT … FROM users WHERE id=7731 200 (3 ms)
→ display_name='Ada' (old value, 176 ms after commit)
11:02:16.900 GET /me uid=7731 db=replica-a SELECT … FROM users WHERE id=7731 200 (2 ms)
→ display_name='Ada L.'
# replica lag, 5-minute p50 / p99, Wednesday 11:00–11:10
replica-a 120 ms / 310 ms
replica-b 180 ms / 1 400 ms (p99 spikes coincide with the 11:00 batch job)
# same graph, last Monday (before the release) — same shape
replica-a 110 ms / 290 ms
replica-b 170 ms / 1 300 ms
# response headers, GET /me
cache-control: private, no-store
cdn-cache: BYPASS
# profile-svc/src/db/route.ts PR #4127, merged Tuesday
- const conn = req.method === 'GET' && !session.recentWrite(req.user, 2_000) ? db.reader() : db.writer()
+ const conn = req.method === 'GET' ? db.reader() : db.writer()Investigate
For each area: first say why you would check it (reveal the reasoning), then look. Commit to a root cause when you are confident.
Cache TTL
Replica lag
The read-routing change
Browser caching headers
The database itself
CDN
Client request ordering