Horizontal vs Vertical Scaling
Vertical scaling makes one machine bigger and adds no new failure modes until it hits the largest instance; horizontal scaling adds machines without limit and adds coordination — and stateless tiers scale out cheaply while databases pay for every rung.
A component is at its capacity limit — CPU, memory, connections, disk I/O — and the question is whether to buy a bigger one or add more of them. Each answer buys capacity by paying a different price, and picking the wrong one at the wrong time is the most common scaling mistake.
Two axes
Vertical scaling replaces the 4-CPU, 16 GB instance with a 32-CPU, 256 GB one. Nothing about the architecture changes: same code, same single process, same single failure domain. Horizontal scaling keeps the 4-CPU instance and runs eight of them behind a load balancer. The code must tolerate copies of itself — which is the whole of Stateless vs Stateful Services — and the system now has coordination: a balancer, health checks, N connection pools, N caches to keep consistent.
The two are not alternatives; they are an order. The question at each capacity wall is which one is cheaper to climb right now, measured in money, engineering time and new failure modes.
Cost curves and hard limits
Cloud pricing is roughly linear in cores up to a point — 8 → 16 → 32 CPUs each about double — and then steepens: the largest instances (hundreds of cores, terabytes of RAM) carry a premium and must be reserved. Beyond the largest instance there is no vertical step at all, and the resize itself is a restart: a minute of downtime for a stateless service, potentially longer for a database that must warm its buffer pool. Horizontal cost is linear forever, and small instances are the cheapest per core — but you pay for the balancer, the extra network hops, and the engineering to make copies safe.
Utilisation is the hidden term. One 32-CPU box at 30% utilisation wastes 22 cores; eight 4-CPU boxes with autoscaling can drop to two at night. Horizontal scaling is what makes elastic cost possible; vertical scaling is a fixed bet.
| Dimension | Vertical (bigger box) | Horizontal (more boxes) |
|---|---|---|
| Hard limit | Largest available instance | None in practice; the shared tier behind it |
| Code changes | None | Must be stateless or partition-aware |
| New components | None | Balancer, discovery, shared session store |
| Failure domain | One machine: lose it, lose everything | Lose one of N; capacity drops by 1/N |
| Elasticity | Restart to resize | Add/remove instances in minutes |
| Per-core cost | Rises at the top end | Flat; small instances cheapest |
| Operational complexity | Low | Rolling deploys, N logs, N caches, coordination |
Vertical is underrated
Teams reach for horizontal scaling because it feels like the "real" solution, and pay for it with a distributed system before they have the traffic to need one. A single modern server has 64–192 cores and terabytes of RAM; a working set that fits in memory runs from memory. Doubling the instance is a config change, no new failure modes, no consistency questions, no application change. It runs out eventually, but "eventually" is far past where most products get, and every rung you postpone is a class of bug you do not yet own.
The honest test is measurement: if CPU is at 85% on a 4-core instance, the cheapest next step is 8 cores, not a load balancer and a session store. Horizontal scaling becomes the right answer when one of three things is true — the largest instance is in sight, a single machine failure is an unacceptable outage, or utilisation swings enough that elasticity pays for the coordination.
Failure domain vs coordination
The vertical box is one failure domain: a kernel panic, a bad deploy, a disk fault — all of it takes 100% of capacity. Eight horizontal instances lose 12.5% per failure, and the balancer routes around it in seconds. That is the strongest argument for horizontal scaling, and it is about availability, not throughput. For availability you need at least two instances regardless of load — see Availability, SLOs and Error Budgets for how redundancy in parallel multiplies uptime.
What you pay is coordination. Eight instances have eight in-process caches that disagree, eight connection pools that together exceed the database limit, eight log streams to correlate, and a deploy that must roll without a moment where old and new versions disagree about a schema. None of these are hard individually; all of them are new.
Databases scale differently
A stateless service scales horizontally by copying; a database cannot be copied without deciding who owns the truth. Adding a read replica (Replication and Read Scaling) scales reads, not writes, and introduces replication lag. Scaling writes horizontally means Partitioning and Sharding — each shard owns a slice of the data, and cross-shard joins, transactions and uniqueness become application problems. So the order for a database is: indexes, then vertical (more RAM so the working set fits), then replicas for reads, then a cache, then sharding as the last resort. The full ladder is Scaling from One User to Millions; the architecture around it is Scale This System.
Key points
- Vertical: same architecture, bigger box, restart to resize, hard ceiling at the largest instance.
- Horizontal: more boxes, linear cost, elastic — and every copy-related problem (state, pools, caches, deploys) at once.
- Measure first; a CPU at 85% on 4 cores wants 8 cores, not a distributed system.
- Horizontal wins on availability: losing one of eight is a 12.5% capacity dip, losing one of one is an outage.
- Databases scale by replicas for reads and shards for writes, each paying with lag or cross-shard complexity; stateless tiers scale by copying for free.
Vertical vs horizontal: cost and limits
How data moves through it
One request or event, hop by hop.
- 1Client → LB: identical for both axes from the client’s view; the balancer exists only on the horizontal path.
- 2LB → instance i of N: any instance, because the tier is stateless.
- 3Instance → shared state: Redis for sessions, database for truth — the components that did not multiply.
- 4Instance → DB: N pools compete for the same connection limit; a pooler (PgBouncer) in front of the database absorbs this.
When to use — and when not
- Vertical: a single measured resource limit, no availability requirement beyond one machine, and headroom below the largest instance.
- Horizontal: the largest instance is in sight, one machine loss must not be an outage, or load swings enough that elasticity pays.
- Both, in order: scale up until the next step is disproportionately expensive, then scale out the stateless tier.
- Horizontal scaling of a stateful service that has not been made stateless — you get sticky sessions and random logouts, not capacity.
- Vertical scaling as the availability plan; a bigger box is still one box.
- Either, before the bottleneck is identified: a missing index or an N+1 query is not fixed by any amount of hardware.
Tradeoffs
Vertical scores 1 on complexity and 5 on the failure-domain risk; horizontal is the reverse. The rating above is the blend a real system ends up with: stateless tiers out, databases up first.
How it fails
- Scaling out a service with in-memory sessions: capacity goes up, users get logged out, and stickiness is bolted on.
- Resizing a database instance during peak: the restart plus a cold buffer pool produces minutes of timeouts.
- Connection multiplication: 30 instances × 20 pooled connections overwhelms a database sized for 200.
- Autoscaling on CPU while the bottleneck is the database: more instances add more load to the thing that is already saturated.
- Running the largest instance at 90% with no next step; the vertical ceiling arrives as an outage instead of a plan.
How it scales
- Stateless services: near-linear with instance count until the shared tier (database, cache) limits them.
- Databases: vertical first, then read replicas (near-linear for reads), then a cache, then sharding for writes.
- Autoscaling policies should key on the real bottleneck metric — queue depth, p99, connection wait — not CPU alone.
How it interacts with databases, queues, caches, APIs and external systems
- Database: the tier that resists horizontal scaling; replicas and a connection pooler are the first horizontal moves it tolerates.
- Cache (Redis): the shared store that makes stateless copies possible; scales out by cluster hashing.
- Load balancer: required for horizontal, irrelevant for vertical.
- Queue: decouples bursty producers from a worker tier that scales horizontally on queue depth.
- Orchestrator/autoscaler: turns horizontal capacity into elastic cost; needs readiness probes and drain to be safe.