Building a Capacity Model
Turning request rate, CPU, memory, connections, queue throughput, network and storage into one defensible statement of safe capacity.
The question, the obvious approach, and why it breaks
Every lesson starts where the work starts: an operational problem, a first attempt that is entirely reasonable, and the way production disagrees with it.
How do you turn a pile of resource limits into a single number you can plan against?
Capacity questions arrive as one number — "can we handle the launch?" — while the system has a dozen independent limits, each in different units.
Take the peak request rate the fleet has served, add a comfortable margin, and call that capacity. It is real data from real traffic, which feels stronger than a model.
The observed peak is a lower bound on capacity, not a measurement of it. The system was not at its limit, so the number says only "at least this much".
- The observed peak is a lower bound on capacity, not a measurement of it. The system was not at its limit, so the number says only "at least this much".
- It carries the traffic mix of the day it was recorded. A launch whose traffic is 90% writes is a different workload against the same fleet.
- It hides which resource was closest to its ceiling, so it cannot tell you what to buy more of.
- It cannot be extrapolated. Doubling the instances does not double capacity if the constraint is a shared database or a fixed quota.
- It gives no answer to the question people actually ask next: what if we lose a zone while that traffic is arriving?
What is actually happening
Underneath the tooling, which is the part that survives a change of tool.
- A capacity model has three parts: the dimensions (the resources that can run out), the per-unit cost of a request in each dimension, and the usable limit in each dimension after reserve is removed.
- Safe capacity in each dimension is the usable limit divided by the per-request cost in that dimension. The system's safe capacity is the smallest of those — the binding constraint. Everything else is slack.
- The per-request cost is an average over a traffic mix, which is why the mix is part of the model. A model that does not state its mix is not reproducible.
- The usable limit is not the physical limit. It is the physical limit minus reserve held for bursts, failure and deploys (Headroom), and minus whatever the resource cannot actually deliver near its ceiling — a disk near full, a pool at its edge, a CPU being throttled.
- Some dimensions are per-instance and scale with the fleet (CPU, memory, local disk). Some are shared and do not (database connections, broker throughput, provider quotas, NAT capacity). Scaling out relieves the first group and worsens the second — which is the single most useful distinction in the whole model.
The dimensions, in the units they actually come in
Each row is a resource that can run out. The column that matters most is the last one: whether adding instances helps or hurts.
| Dimension | Limit is expressed as | Per-request cost | Scaling out |
|---|---|---|---|
| Request rate | Requests per second the tier can serve | One request | Helps, until a shared dimension binds |
| CPU | Cores, or a throttled quota per instance | CPU-seconds per request | Helps directly |
| Memory | Bytes per instance before the limit kills it | Peak resident bytes per in-flight request | Helps, if concurrency per instance drops too |
| DB connections | A server-wide maximum, shared by everyone | Connections held per in-flight request | Hurts — pools are per-instance (The Connection Budget) |
| Queue throughput | Messages per second consumers can drain | One message, plus its downstream work | Helps at the consumer, moves load downstream |
| Network | Bandwidth and packets per second per instance | Bytes in and out per request | Helps per-instance, may hit a shared gateway |
| Storage | Bytes free, and IOPS | Bytes written per request, retained for a period | Neutral — usually a shared volume or service |
| Third-party quota | Calls per interval, set by the provider | Outbound calls per request | Hurts — the quota is for the whole account |
The arithmetic, and what it is allowed to claim
The model is one expression applied per dimension and then minimised. Written symbolically it is impossible to mistake for a measurement, which is the point — the numbers are yours, and only yours, because they come from your load test.
The two subtractions in the usable limit are where most of the honesty lives. The reserve is a policy decision made once and applied everywhere (Headroom); the derating is an admission that resources misbehave near their ceilings.
1for each dimension d:2 usable_d = hard_limit_d - reserve_d - derating_d3 cost_d = resource units of d consumed per request, for a stated mix4 capacity_d = usable_d / cost_d # requests per second5 6safe_capacity = min over d of capacity_d7binding = the d that produced the min8 9# the two outputs are equally important:10# safe_capacity tells you whether the launch fits11# binding tells you what to buy, fix or shedEverything on the right-hand side is measured by you. Nothing in this lesson supplies a value for any of them, because a number that did not come from your system is not evidence about your system.
Building one, once
The first model is a day of work and pays for itself the first time someone asks whether a campaign will fit. The steps matter in this order: enumerating before measuring stops you from measuring precisely the wrong thing.
- 1Enumerate dimensions
List every resource that can run out, including shared and third-party ones.
fails by The dimension you omit is disproportionately likely to be the one that binds.
evidence A written list a second engineer cannot add to.
- 2State the traffic mix
Fix the ratio of endpoints, read/write split and payload sizes the model describes.
fails by An unstated mix makes the result irreproducible and unfalsifiable.
evidence The mix is recorded next to the result and matches production telemetry.
- 3Measure per-request cost
Run that mix at moderate load and record consumption per dimension.
fails by Measuring on a warm, idle-adjacent system understates cold cost.
evidence Consumption per request from telemetry, not from a calculator.
- 4Establish hard limits
Find the real ceiling for each dimension — instance class, server config, provider quota.
fails by Assuming a default that was changed, or a quota that was never checked.
evidence Each limit traced to a configuration value or a provider console.
- 5Subtract reserve
Apply the headroom policy and any derating near ceilings.
fails by Modelling to 100% of the hard limit, which no system can actually use (Headroom).
evidence The reserve figure is the one from the policy, and the policy is written down.
- 6Minimise and publish
Report safe capacity and the binding dimension together.
fails by Publishing the number without the constraint, so nobody knows what to fix.
evidence A model page linked from the runbook (Runbooks).
- 7Validate
Load test to actual saturation and compare with the prediction.
fails by Skipping this, leaving a model that has never been contradicted or confirmed.
evidence Observed saturation point and observed binding dimension, next to the predicted ones.
How to do it properly
Most important first.
- Enumerate the dimensions before measuring anything. A dimension you did not list cannot be the answer.
- Measure per-request cost from a load test with a stated traffic mix, at a load high enough to be representative and below the knee.
- Mark every dimension as per-instance or shared. The shared ones are where scaling betrays you (The Connection Budget).
- Compute safe capacity per dimension and publish the minimum together with which dimension produced it — the second half is the actionable part.
- Re-run the model when the workload changes shape, not on a calendar. A new N+1 query changes the database dimension by more than a year of traffic growth.
- State the model's assumptions next to its answer so a reader can tell whether it still applies.
How much can this affect
Every production change has a blast radius. Stated as a scale so it is comparable between changes rather than adjectival — and paired with what actually contains it, because a wide scope with a real containment mechanism is a different situation from a wide scope with none.
A wrong model is contained only by the load test that would have contradicted it; nothing in production stops a plan built on it.
What can go wrong
- A model with correct arithmetic and a wrong mix, which is worse than no model because it is quoted with confidence.
- Per-request cost measured on a warm system, so the first minutes of a real peak — cold caches, cold pools, JIT warm-up — exceed the model immediately (Scale to Zero).
- A shared dimension modelled as if it were per-instance, so the plan says "add instances" and adding instances makes it worse.
- Storage modelled as percentage-full rather than time-to-full, so the alert arrives with no time to act.
- The mitigation failing: the model becomes a spreadsheet nobody owns, correct on the day it was written and quietly wrong for a year.
- "The model says we can handle it, so we are fine." The model describes steady state. Failover, deploys and cold starts are not steady state (Capacity During Failover).
- "We should model every dimension precisely." Model them all coarsely, then refine only the one that binds and the one behind it.
- "Capacity per instance times instance count is capacity." True only for per-instance dimensions, which is the half of the model that never causes the incident.
- "We have historical peaks, so we do not need a model." History tells you what happened. A model tells you what would happen to a mix you have not seen yet.
Operating it
Evidence is the signal, not the intention. Rollback is sometimes 'you cannot, and that is the point'.
- A written model naming its dimensions, its traffic mix, and which dimension binds.
- A load test whose observed saturation point matches the model's prediction within an order that the team finds acceptable — and a note when it does not.
- Dashboards showing current level against usable limit for the binding dimension and the next one behind it.
- A launch that was planned against the model and behaved as the model said it would.
- The model itself carries no rollback — it is a document. What needs a rollback path is every change made because of it: a raised pool size, a larger instance class, a higher quota.
- Version the model with the system. When you revert a change, revert the assumption it created, or the next planner inherits a model describing a fleet that no longer exists.
- Automate collection: per-request resource cost, current levels and limits should come from telemetry, not from someone reading a console.
- Automate the recomputation so the published safe capacity moves when the inputs move.
- Keep the choice of traffic mix human. It encodes a prediction about the business, and a script cannot know that next month is a sale.
- A model is only as good as its worst dimension estimate, and the dimensions that are hardest to measure — shared services, third-party quotas — are exactly the ones that bind.
- Precision here has sharply diminishing returns. Knowing which dimension binds is worth a great deal; knowing safe capacity to two significant figures is usually worth very little.
- Modelling costs load-test infrastructure that looks like production, which is one of the more expensive environments a team runs (Parity That Is Worth Paying For).
Where this applies
This domain is unusually tool- and organisation-dependent. These labels say what each claim is specific to, and what a different platform, provider or organisation does instead.
- GENERALThe min-over-dimensions structure holds anywhere. The dimension list changes: a serverless workload has concurrency and downstream quotas but no instance memory to size, while a stateful service adds disk and IOPS.
- SIMPLIFIEDThis treats per-request cost as an average and dimensions as independent. Real systems have interacting limits — memory pressure raises CPU through GC, throttled CPU lengthens connection hold time — so a real model is conservative rather than exact.
Where the depth lives
This domain teaches delivery and operation, and hands the mechanism off to the domain that owns it.
- — System Design — estimating capacity for a system that does not exist yet, where every input is an assumption rather than a measurement.