ScalingBeginner

Why must services be stateless to scale?

“Why does horizontal scaling require stateless services? What state is unavoidable, and how do you handle it?”

What this tests

  • The mechanism: any instance can serve any request, instances are disposable
  • Concrete examples of state that breaks this (sessions, local files, in-process caches)
  • Externalising state and its cost (extra hop, new dependency)
  • Handling genuinely stateful cases: WebSockets, uploads, caches

Answers by level

Read the beginner answer first and notice what is missing.

Horizontal scaling works when any instance can serve any request and instances can be killed and replaced without anyone noticing. If an instance holds state — a user's session in memory, an uploaded file on local disk, a cache that other instances do not have — then requests must be routed to that specific instance, it cannot be replaced without losing something, and rolling deploys log users out. The classic incident is autoscaling plus round-robin plus in-memory sessions: random logouts.

State is externalised: sessions in Redis with a TTL or in a signed token the client carries; uploads to object storage; shared caches in Redis rather than per-process. The cost is real: an extra network hop per request (Redis at ~1 ms versus a memory read), and Redis becomes a dependency whose outage logs everyone out at once, so it needs its own availability story.

Sticky sessions are a workaround, not a solution: they fight the load balancer (uneven load), break when an instance dies, and complicate deploys. They are acceptable temporarily while sessions are being moved out.

Green flags · Red flags

Strong green flag · Identifies per-instance rate-limit counters or dedup sets as silent correctness bugs the moment a second instance exists.
Green flags
  • Explains the mechanism: any instance, any request, disposable instances
  • Names the concrete failures: random logouts, lost uploads, inconsistent caches
  • Externalises to Redis/object storage and names the cost (hop, new dependency)
  • Handles unavoidable state with consistent-hash routing and graceful drain
  • Distinguishes correctness state from performance caches
Red flags
  • "Just use sticky sessions."
  • Believes stateless means no database
  • Ignores that Redis is now a dependency on the login path
  • Cannot say what happens to a WebSocket on deploy

Follow-up questions

F1
Users are randomly logged out after enabling autoscaling. Diagnose.
F2
What is the cost of moving sessions to Redis?
F3
How do you deploy a service holding 50,000 WebSockets?

Scenario

A service stores sessions in process memory, rate-limit counters in a local map, and resized images in /tmp before upload. It runs as one instance. The team wants to run five instances behind a load balancer. List everything that breaks, in the order a user would notice, and the fix for each.

Learn this topic