Scaling & Caching

A load balancer turns N servers into one address so capacity and availability stop depending on a single machine — and the algorithm, the health check and the balancer’s own redundancy each decide whether it helps or hurts under load.

A stateless service keeps nothing between requests that another instance would need, so any instance can serve any request and instances become disposable; the state does not vanish — it moves to Redis or the database, and that extra hop and new dependency are the price of horizontal scaling.

Caches sit at five distances from the user — browser, CDN, in-process, distributed, database buffer — each one trading freshness for latency and load, and the architecture questions are which layer answers which read, how a key expires, and what happens when thousands of requests miss at once.

A CDN puts copies of responses in hundreds of edge locations so a user in Frankfurt gets bytes from Frankfurt instead of Virginia — beating the speed of light by not crossing the ocean — and the design questions are what the cache key is, how the origin is protected from misses, and how a change reaches every edge.

Vertical scaling makes one machine bigger and adds no new failure modes until it hits the largest instance; horizontal scaling adds machines without limit and adds coordination — and stateless tiers scale out cheaply while databases pay for every rung.

Start with one server and one database and add exactly one component per measured problem — load balancer, cache and replicas, sharding decisions, CDN, queue and workers — noting at every step the symptom that forced it and the new problem it introduced, until the diagram every system-design answer draws has been earned box by box.