Cost Per Request
Infrastructure cost divided by successful requests — the unit that lets you compare architectures instead of comparing bills.
The question, the obvious approach, and why it breaks
Every lesson starts where the work starts: an operational problem, a first attempt that is entirely reasonable, and the way production disagrees with it.
How do you tell whether a system got cheaper, when the business got bigger at the same time?
Total spend rises with growth, so it cannot distinguish a system becoming more expensive from a business becoming more successful. Every cost conversation stalls on that ambiguity.
Track total monthly spend and compare month to month. It is the number on the invoice and everybody understands it.
Total spend conflates efficiency and volume. A month that is 20% more expensive because traffic grew 30% describes an improvement, and the total says the opposite.
- Total spend conflates efficiency and volume. A month that is 20% more expensive because traffic grew 30% describes an improvement, and the total says the opposite.
- It cannot compare two architectures, because they were never run at the same traffic level.
- It gives no target. "Spend less" is not something a team can design against; "serve the same work for less" is.
- It hides the fixed floor. A system with a large fixed component looks efficient at high traffic and terrible at low traffic, and total spend never reveals which regime you are in.
- It counts failed work as if it were valuable. A month where a tenth of requests errored costs the same as one where they succeeded, and the totals look identical.
What is actually happening
Underneath the tooling, which is the part that survives a change of tool.
- The metric is infrastructure cost for a period divided by the number of successful requests served in that period. Successful is the load-bearing word: failed work consumed resources and delivered nothing, so counting it flatters the number exactly when the system is behaving worst.
- Its value is comparability. Two architectures, two regions, two versions or two tenants can be compared on the same axis even when their volumes differ by orders of magnitude.
- Cost per request is not a constant. Total cost has a fixed part that exists at zero traffic and a variable part that scales with work, so cost per request falls as traffic rises and rises as traffic falls. Quoting one value without the traffic level it was measured at is meaningless.
- That decomposition is the useful output. A high fixed share means the lever is consolidation or scaling down when idle; a high variable share means the lever is per-request work (Cost Drivers).
- The denominator is a choice with consequences. Requests, transactions, active users, jobs completed or business events each give a different picture, and the right one is whatever unit the business actually cares about producing.
- It is a comparison unit, not a target. Driving it down by shedding expensive requests, or by degrading a feature, improves the metric while making the product worse — which is why it is read next to reliability, never alone.
The definition, and where each part goes wrong
The formula is trivial and every difficulty is in the definitions. Write both halves down explicitly; the exercise of doing so surfaces most of the disagreements before they become arguments about a chart.
1cost_per_unit(period) = attributable_infrastructure_cost(period)2 / successful_units(period)3 4numerator includes compute, storage, transfer, managed services,5 observability, and an agreed share of platform cost6 excludes people, licences, and anything you cannot attribute7 honestly - state the exclusions8 9denominator successful units only10 a unit is whatever the business produces:11 requests | transactions | jobs | active users | events12 failed work is counted separately, not discarded13 14report with the traffic level, because:15 cost(period) = fixed + variable_per_unit x units16 => cost_per_unit falls as units rise, with no change to the systemThe last two lines are why a single quoted value is not a fact about the system. The same architecture reports very different numbers at different traffic levels.
Choosing the denominator
The denominator decides what the metric is capable of telling you. Pick the unit that the business would notice losing, and be honest that a service with genuinely different kinds of work may need more than one.
| Denominator | Best for | What it hides | Failure it invites |
|---|---|---|---|
| Successful HTTP requests | Stateless APIs with uniform work | Wide variation between endpoints | A single expensive endpoint dominating the average |
| Business transactions | Systems where a transaction is the product | The read traffic that supports each one | Ignoring the cheap-but-enormous read path |
| Jobs completed | Batch and asynchronous workloads | Job size variation | Optimising job count instead of work done |
| Monthly active users | Products billed per seat | Very different usage per user | Cost that tracks usage, not seats |
| Bytes processed | Data pipelines | The value of the processing | Rewarding compression over usefulness |
| Tokens or model calls | Agent and inference workloads (Agent Cost in Production) | Retries and failed generations | A cheap-per-call system that calls constantly |
What it is good at, and what it is bad at
The metric earns its place in one specific job: comparing two things that serve different volumes. It is poor at almost everything else that people try to use it for, and the failures are all the same failure — treating a ratio as a goal.
goal: reduce cost per request by a fixed amount -> shed expensive requests -> drop an expensive feature -> reduce headroom and redundancy -> metric improves -> reliability falls, product narrows -> nobody objects, the number is green
question: is architecture A cheaper than B for the same work? -> measure both at a stated traffic level -> decompose fixed and variable -> read next to error rate and latency -> choose, and record why -> re-measure after the change lands
As a target it is trivially satisfiable by doing less, and doing less is invisible in a ratio. As a comparison it answers a question that total spend genuinely cannot answer, which is why the metric exists at all.
How to do it properly
Most important first.
- Define the denominator once, write it down, and keep it stable. A metric whose definition drifts cannot be compared over time, which was the entire point.
- Count successful units only, and track the cost of failed work separately — it is a real number and it is worth seeing (Cost Awareness).
- Report cost per unit alongside the traffic level it was measured at, so the fixed-versus-variable regime is visible.
- Decompose into fixed and variable at least once. It tells you which lever exists and stops a year of pointless optimisation of the wrong half.
- Use it to compare candidate architectures during design, where the numbers are estimates but the shapes are honest.
- Watch it per tenant or per endpoint where the business cares — one expensive endpoint can dominate a service average.
- Read it next to error rate and latency. A cost-per-request improvement that came with a reliability regression is not an improvement (Capacity or Efficiency: Which Problem Are You Solving? in Performance).
How much can this affect
Every production change has a blast radius. Stated as a scale so it is comparable between changes rather than adjectival — and paired with what actually contains it, because a wide scope with a real containment mechanism is a different situation from a wide scope with none.
Nothing breaks on the day it is computed — the risk is a wrong architecture decision made from it, contained only by reading it next to reliability signals.
What can go wrong
- A denominator that includes failed requests, so the metric improves during an outage.
- A numerator that includes only compute, so storage, transfer and observability — often a large share — are invisible (Cost Drivers).
- Comparing two measurements taken at very different traffic levels and attributing the difference to architecture rather than to the fixed floor.
- Optimising the metric by shedding the expensive requests, which is a product change disguised as an efficiency win.
- Shared platform costs allocated by a rule nobody agreed with, making the per-service numbers unarguable in the wrong way (FinOps).
- The mitigation failing: the metric becomes a target on a dashboard, and teams start managing the number rather than the system.
- "Lower is always better." Lower with the same reliability and the same product is better. Lower because you shed load or removed a feature is a different change wearing the same number.
- "We can compare our cost per request to another company's." Different products, different work per request, different accounting boundaries. The comparison that means something is against your own past.
- "It is going up, so we are getting less efficient." Check the traffic level first: falling traffic raises cost per request with no change to the system at all.
- "Include everything in the numerator." Include everything you can attribute honestly, and state what is excluded. A precise number over an arbitrary allocation is false precision.
Operating it
Evidence is the signal, not the intention. Rollback is sometimes 'you cannot, and that is the point'.
- A stated definition of numerator and denominator, in a document, that has not changed this quarter.
- The metric plotted against traffic, so the fixed and variable regimes can be seen rather than assumed.
- A design decision that was made by comparing two candidate architectures on this unit.
- The metric plotted next to error rate and latency on the same dashboard.
- The metric itself changes nothing in production. What needs a rollback path is the change it motivated — a smaller instance class, a removed cache, a consolidated fleet — each of which is a capacity change (Overprovisioning).
- If the definition must change, keep both series for an overlap period. Silently redefining a metric destroys the history that made it valuable.
- Automate the computation from billing and request telemetry, per service, on the same cadence as the cost data.
- Automate the joint display with reliability signals so the number is never read alone.
- Keep the denominator definition human and deliberately hard to change, because its stability is the source of its value.
- Attribution is required before the numerator means anything, and shared costs have no objectively correct allocation (FinOps).
- The metric is easy to game and, being a single number, invites exactly that. It is safest when it is a comparison tool used by engineers and least safe when it is a target reported upwards.
- Maintaining a stable denominator across a changing product is real work: endpoints appear, semantics shift, and the unit has to be defended.
Where this applies
This domain is unusually tool- and organisation-dependent. These labels say what each claim is specific to, and what a different platform, provider or organisation does instead.
- CLOUD-SPECIFICAssumes metered billing that can be attributed per service. With owned hardware the numerator is an amortised capital and power figure, which changes the arithmetic and makes the fixed share far larger.
- SIMPLIFIEDTreating cost as fixed plus a constant per request is a teaching model. Real per-request cost varies by endpoint, by payload, by cache state and by time of day, so a single average hides distributions that sometimes matter more than the mean.
Where the depth lives
This domain teaches delivery and operation, and hands the mechanism off to the domain that owns it.
- — System Design — using a cost-per-unit estimate to choose between candidate designs before either exists.