Stateless vs Stateful Services
A stateless service keeps nothing between requests that another instance would need, so any instance can serve any request and instances become disposable; the state does not vanish — it moves to Redis or the database, and that extra hop and new dependency are the price of horizontal scaling.
A server that remembers users in its own memory cannot be duplicated, replaced or drained without losing what it remembers. Moving that state into a shared store makes every instance interchangeable, which is the precondition for load balancing, autoscaling and rolling deploys.
What "state" means here
Every service has state in the sense that it reads and writes a database. The question is narrower: does the instance that handled request 1 hold anything in memory that request 2 from the same client needs? A login session in a Map<sessionId, User>, a shopping cart in process memory, an upload buffered on local disk, a WebSocket connection — these make the instance special. If the balancer sends request 2 elsewhere, the user is logged out, the cart is empty, the upload fails.
A stateless instance holds none of that. It reads the session from Redis, the cart from the database, streams the upload straight to object storage, and could be killed between any two requests without a client noticing. The state still exists; it lives in a component built to be shared and durable.
Why stateless scales horizontally
Three operational properties fall out of interchangeability. Any instance can serve any request, so the balancer can use least-connections or round robin and load stays even. Instances are disposable: autoscaling can add ten at 09:00 and remove them at 18:00; a crashed instance is replaced, not repaired. Rolling deploys work: drain one instance, replace it, move on — no one is logged out because no instance owned anyone.
The failure it prevents is the one in the challenge logged-out-after-scaling: sessions in process memory, then autoscaling doubles the fleet and round robin sends the next request to an instance that has never seen the user. Stickiness patches this and breaks again on every scale-in and deploy; externalising the session fixes it.
1// Before: state on the instance — only this process can serve this user2const sessions = new Map<string, Session>()3 4// After: state in Redis — any instance resolves the same session5async function loadSession(sessionId: string): Promise<Session | null> {6 const raw = await redis.get(`sess:${sessionId}`)7 if (!raw) return null8 await redis.expire(`sess:${sessionId}`, 60 * 60 * 24) // sliding 24 h TTL9 return JSON.parse(raw) as Session10}State you cannot remove, and what to do with it
Some state is bound to a connection or a machine. A WebSocket is a TCP connection to one process; a chunked upload in flight lives where the first chunk landed; an in-process cache is by definition local. You do not eliminate these; you contain them.
Externalise what can be externalised: upload chunks go to object storage with a resumable upload id, not to local disk; presence and pub/sub go through Redis so a message for a user connected to instance 3 can be published by instance 7. Route deliberately what must stay local: hash the connection or room id with Consistent Hashing so reconnects land on the same node, and give every node a graceful drain — on shutdown, stop accepting new connections, tell clients to reconnect, finish in-flight work, then exit. Accept and bound the rest: an in-process cache is fine as long as a miss is merely slower, not wrong.
| State | Naive (stateful) | Stateless form | Cost of the move |
|---|---|---|---|
| Login session | In-memory map | Redis hash with TTL, or signed token | One Redis round trip per request |
| Shopping cart | Process memory | Database row or Redis hash | Write per change |
| Upload in progress | Local disk | Object storage multipart upload | Upload id must be returned to the client |
| WebSocket | Pinned process | Pinned process + Redis pub/sub + drain | Fan-out through Redis |
| Hot lookup cache | Process memory | Keep local; shared Redis as second level | Cold start after deploy |
The price: an extra hop and a new dependency
Every request now pays a Redis round trip — 0.3–1 ms in the same zone, more across zones — before it can do anything. At 5,000 req/s that is 5,000 Redis reads per second, which is trivial for Redis but a real line in the latency budget. The session store is a new dependency: if Redis is down, every user is logged out at once, so it needs replication and a failover plan, and the application needs a decision about what to do when it is unreachable (fail closed for auth; fail open for a preferences cache).
Signed tokens (JWT) remove the hop by carrying the session in the request. The tradeoff is that you cannot revoke one without a server-side list — which is a session store again — and a token is only as fresh as when it was issued. Use them where sessions are short and revocation is not a requirement; use Redis where "log out everywhere" must work. The Redis data types that fit sessions, presence and rate counters are in Redis: Data Structures, Not a Cache.
Key points
- Stateless means: nothing in this instance’s memory is needed by the next request from the same client.
- Interchangeable instances give you even load, autoscaling, disposability and zero-logout rolling deploys.
- The state moves, it does not disappear: sessions to Redis, carts to the DB, uploads to object storage.
- Connections and in-flight uploads are genuinely local: route them deliberately and drain gracefully on shutdown.
- The cost is one more hop per request and one more dependency that must be highly available.
Sessions: local vs shared
How data moves through it
One request or event, hop by hop.
- 1Client → LB: request with a session cookie or bearer token.
- 2LB → any instance: chosen by load, not by who saw this client before.
- 3Instance → Redis:
GET sess:<id>; missing means unauthenticated. - 4Instance → DB: the actual work, keyed by the user id from the session.
- 5Instance → client: response; nothing about this client is retained in process memory.
When to use — and when not
- Any HTTP service that will run as more than one instance — which is every service you intend to load-balance or autoscale.
- Before the first horizontal scale-out, not after the first incident.
- Rolling or blue-green deploys where no request may be lost and no user logged out.
- A single-instance internal tool with a handful of users; the Redis dependency costs more than the memory map.
- Connection-bound protocols where per-connection state is the whole point (game servers, live collaboration) — there you design routing and drain instead of pretending the node is stateless.
- Latency-critical paths where the session round trip is measurable and a short-lived signed token is acceptable.
Tradeoffs
Almost always worth it: one more hop and one more highly available store buy linear horizontal scaling of the whole application tier.
How it fails
- Sessions in process memory plus autoscaling or a deploy: users randomly logged out, carts emptied, uploads lost.
- Redis session store down with no fallback: 100% of users logged out simultaneously; treat it as tier-one infrastructure.
- Stickiness by source IP behind a corporate NAT: thousands of users on one instance while others idle.
- Shutdown without drain: SIGTERM kills 2,000 WebSocket connections mid-message on every deploy.
- A "stateless" service that writes temp files to local disk and reads them in a later request — works with one instance, breaks with two.
How it scales
- The application tier scales linearly by adding instances; load balancer and autoscaler handle the rest.
- The session store becomes the shared component: Redis handles ~100k ops/s per node; cluster or shard by session id beyond that.
- Connection-bound state scales by hashing connections across nodes and keeping the node count change rare and drained.
How it interacts with databases, queues, caches, APIs and external systems
- Cache (Redis): sessions, presence, pub/sub fan-out for WebSockets; the store that makes instances interchangeable.
- Database: durable state such as carts and orders; the session should hold ids, not copies of rows.
- Object storage: in-flight uploads via multipart or resumable upload ids instead of local disk.
- Load balancer: no stickiness required; connection-bound services use hashing plus drain instead.
- Orchestrator: readiness probes and termination grace periods implement the drain.