DNSBeginner
Half the users still hit the old server, two hours after the migration
Symptoms
- The API was moved to a new host at 14:00; the old host was shut down at 14:30.
- Since 14:30 roughly half of customers report "connection timed out"; the other half are fine.
- From the office,
curl https://api.example.com/healthreturns 200 every time. - Nothing in the new server’s logs looks wrong — it simply never sees the failing customers.
$ dig +short api.example.com 203.0.113.20 # office resolver: the NEW address $ dig +short api.example.com @8.8.8.8 203.0.113.10 # public resolver: the OLD address $ dig api.example.com @ns1.example.com ;; ANSWER SECTION: api.example.com. 86400 IN A 203.0.113.20 $ dig api.example.com @8.8.8.8 | grep -A1 ANSWER ;; ANSWER SECTION: api.example.com. 79212 IN A 203.0.113.10 # 79,212 s of TTL still to go
Investigate
Inspect areas in any order (0/6 inspected). When you think you know the root cause, commit to it.
The authoritative record
The TTL on the record
Public resolver answers
The new server’s firewall
Old host shutdown
Office resolver