Scalingstatelessstatefulsessionredishorizontal scaling

Stateless vs Stateful Services

A stateless service keeps nothing between requests that another instance would need, so any instance can serve any request and instances become disposable; the state does not vanish — it moves to Redis or the database, and that extra hop and new dependency are the price of horizontal scaling.

▶ InteractiveInterview questionDebug it
Progress
What problem does this solve?

A server that remembers users in its own memory cannot be duplicated, replaced or drained without losing what it remembers. Moving that state into a shared store makes every instance interchangeable, which is the precondition for load balancing, autoscaling and rolling deploys.

What "state" means here

Every service has state in the sense that it reads and writes a database. The question is narrower: does the instance that handled request 1 hold anything in memory that request 2 from the same client needs? A login session in a Map<sessionId, User>, a shopping cart in process memory, an upload buffered on local disk, a WebSocket connection — these make the instance special. If the balancer sends request 2 elsewhere, the user is logged out, the cart is empty, the upload fails.

A stateless instance holds none of that. It reads the session from Redis, the cart from the database, streams the upload straight to object storage, and could be killed between any two requests without a client noticing. The state still exists; it lives in a component built to be shared and durable.

Session in memory vs session in a shared store
request 2request 1request 2SET sess:… (stateless)GET sess:… → foundClientLoad balancerInstance 1 (had the session)Instance 2 (never saw it)Redis sessionsDatabase
ClientGateway / LBServiceWorkerDatabaseCacheQueue / LogObject storageCDNExternal system

Why stateless scales horizontally

Three operational properties fall out of interchangeability. Any instance can serve any request, so the balancer can use least-connections or round robin and load stays even. Instances are disposable: autoscaling can add ten at 09:00 and remove them at 18:00; a crashed instance is replaced, not repaired. Rolling deploys work: drain one instance, replace it, move on — no one is logged out because no instance owned anyone.

The failure it prevents is the one in the challenge logged-out-after-scaling: sessions in process memory, then autoscaling doubles the fleet and round robin sends the next request to an instance that has never seen the user. Stickiness patches this and breaks again on every scale-in and deploy; externalising the session fixes it.

Session lookup that any instance can perform
1// Before: state on the instance — only this process can serve this user
2const sessions = new Map<string, Session>()
3
4// After: state in Redis — any instance resolves the same session
5async function loadSession(sessionId: string): Promise<Session | null> {
6 const raw = await redis.get(`sess:${sessionId}`)
7 if (!raw) return null
8 await redis.expire(`sess:${sessionId}`, 60 * 60 * 24) // sliding 24 h TTL
9 return JSON.parse(raw) as Session
10}

State you cannot remove, and what to do with it

Some state is bound to a connection or a machine. A WebSocket is a TCP connection to one process; a chunked upload in flight lives where the first chunk landed; an in-process cache is by definition local. You do not eliminate these; you contain them.

Externalise what can be externalised: upload chunks go to object storage with a resumable upload id, not to local disk; presence and pub/sub go through Redis so a message for a user connected to instance 3 can be published by instance 7. Route deliberately what must stay local: hash the connection or room id with Consistent Hashing so reconnects land on the same node, and give every node a graceful drain — on shutdown, stop accepting new connections, tell clients to reconnect, finish in-flight work, then exit. Accept and bound the rest: an in-process cache is fine as long as a miss is merely slower, not wrong.

Where the state can live
StateNaive (stateful)Stateless formCost of the move
Login sessionIn-memory mapRedis hash with TTL, or signed tokenOne Redis round trip per request
Shopping cartProcess memoryDatabase row or Redis hashWrite per change
Upload in progressLocal diskObject storage multipart uploadUpload id must be returned to the client
WebSocketPinned processPinned process + Redis pub/sub + drainFan-out through Redis
Hot lookup cacheProcess memoryKeep local; shared Redis as second levelCold start after deploy

The price: an extra hop and a new dependency

Every request now pays a Redis round trip — 0.3–1 ms in the same zone, more across zones — before it can do anything. At 5,000 req/s that is 5,000 Redis reads per second, which is trivial for Redis but a real line in the latency budget. The session store is a new dependency: if Redis is down, every user is logged out at once, so it needs replication and a failover plan, and the application needs a decision about what to do when it is unreachable (fail closed for auth; fail open for a preferences cache).

Signed tokens (JWT) remove the hop by carrying the session in the request. The tradeoff is that you cannot revoke one without a server-side list — which is a session store again — and a token is only as fresh as when it was issued. Use them where sessions are short and revocation is not a requirement; use Redis where "log out everywhere" must work. The Redis data types that fit sessions, presence and rate counters are in Redis: Data Structures, Not a Cache.

Key points

  • Stateless means: nothing in this instance’s memory is needed by the next request from the same client.
  • Interchangeable instances give you even load, autoscaling, disposability and zero-logout rolling deploys.
  • The state moves, it does not disappear: sessions to Redis, carts to the DB, uploads to object storage.
  • Connections and in-flight uploads are genuinely local: route them deliberately and drain gracefully on shutdown.
  • The cost is one more hop per request and one more dependency that must be highly available.

Sessions: local vs shared

Sessions: local vs shared
Three servers behind round robin. Where does the login session live, and what happens when the fleet changes?
BrowserLoad balancerServer 1Server 2Server 3
Where the session lives
t+1POST /login → Server 1 · 200 · session sess-91 created in Server 1 memory
t+2GET /account → Server 2 (round robin) · 401 "please log in again"
t+3GET /orders → Server 3 · 401
t+4autoscaler adds Server 4 and 5 · GET /cart → Server 4 · 401
t+5Server 1 crashes (OOM) · GET /checkout → Server 2 · 401 "please log in again" — session sess-91 gone
Failed requests
0
Sessions lost on death
Servers
3
Extra hop per request
none
Server 1 now holds the only copy of this session. The cookie the browser got back is only meaningful to Server 1.
Stateless means any instance can serve any request, so instances become disposable: kill one to deploy, add five when traffic spikes, lose one and nobody logs out. The cost is one extra hop to Redis on every request and a new dependency that itself must be replicated. Sticky sessions are a stopgap that keeps the state problem and adds a routing one.
1/5 · login

How data moves through it

One request or event, hop by hop.

  1. 1Client → LB: request with a session cookie or bearer token.
  2. 2LB → any instance: chosen by load, not by who saw this client before.
  3. 3Instance → Redis: GET sess:<id>; missing means unauthenticated.
  4. 4Instance → DB: the actual work, keyed by the user id from the session.
  5. 5Instance → client: response; nothing about this client is retained in process memory.

When to use — and when not

Use it when
  • Any HTTP service that will run as more than one instance — which is every service you intend to load-balance or autoscale.
  • Before the first horizontal scale-out, not after the first incident.
  • Rolling or blue-green deploys where no request may be lost and no user logged out.
Avoid it when
  • A single-instance internal tool with a handful of users; the Redis dependency costs more than the memory map.
  • Connection-bound protocols where per-connection state is the whole point (game servers, live collaboration) — there you design routing and drain instead of pretending the node is stateless.
  • Latency-critical paths where the session round trip is measurable and a short-lived signed token is acceptable.

Tradeoffs

Complexity
low → high
Ops cost
low → high
Latency
low → high
Consistency
weak → strong
Scalability
poor → strong

Almost always worth it: one more hop and one more highly available store buy linear horizontal scaling of the whole application tier.

How it fails

  • Sessions in process memory plus autoscaling or a deploy: users randomly logged out, carts emptied, uploads lost.
  • Redis session store down with no fallback: 100% of users logged out simultaneously; treat it as tier-one infrastructure.
  • Stickiness by source IP behind a corporate NAT: thousands of users on one instance while others idle.
  • Shutdown without drain: SIGTERM kills 2,000 WebSocket connections mid-message on every deploy.
  • A "stateless" service that writes temp files to local disk and reads them in a later request — works with one instance, breaks with two.

How it scales

  • The application tier scales linearly by adding instances; load balancer and autoscaler handle the rest.
  • The session store becomes the shared component: Redis handles ~100k ops/s per node; cluster or shard by session id beyond that.
  • Connection-bound state scales by hashing connections across nodes and keeping the node count change rare and drained.

How it interacts with databases, queues, caches, APIs and external systems

  • Cache (Redis): sessions, presence, pub/sub fan-out for WebSockets; the store that makes instances interchangeable.
  • Database: durable state such as carts and orders; the session should hold ids, not copies of rows.
  • Object storage: in-flight uploads via multipart or resumable upload ids instead of local disk.
  • Load balancer: no stickiness required; connection-bound services use hashing plus drain instead.
  • Orchestrator: readiness probes and termination grace periods implement the drain.