Scaling & CachingdebuggingBeginner

Users are randomly logged out since we turned on autoscaling

Symptoms

  • Users report being logged out at random, often mid-checkout; shopping carts vanish and reappear.
  • It started the afternoon autoscaling was enabled (fixed 2 instances → 2–8 instances) and gets worse at peak, when more instances are running.
  • Reproducible by refreshing the account page a few times: roughly every other refresh is a 401.
  • No errors in the logs — the 401s are "session not found", which is a normal response.
                     ┌── api-1  (up 14 d)     sessions in process memory: 4,120
client ──► LB ───────┼── api-2  (up 14 d)     sessions in process memory: 3,980
   (round robin,     ├── api-3  (up 6 min)    sessions in process memory: 212
    no affinity)     └── api-4  (up 6 min)    sessions in process memory: 198

# access log, one user (cookie sid=ab12 present on every request)
14:07:31  api-1  GET /cart   sid=ab12  → 200  (3 items)
14:07:34  api-3  GET /cart   sid=ab12  → 401  session not found
14:07:36  api-2  GET /cart   sid=ab12  → 401  session not found
14:07:40  api-1  GET /cart   sid=ab12  → 200  (3 items)

# before today: api-1 active, api-2 hot standby (LB weight 0); autoscaling policy switched both to weight 1

Investigate

Inspect areas in any order (0/6 inspected). When you think you know the root cause, commit to it.

Cookie configuration
Recent code deploys
Where sessions are stored
Load balancer routing
Clock skew between instances
Redis