API Performance: The Levers You Actually Own
Most API latency is decided by the contract, not the code: how many round trips a task needs, how many bytes each carries, and how often a request can be skipped entirely. The levers are payload, compression, request count, caching, serialization and field selection.
Frame the contract
API design starts with a consumer, a design question and a guarantee — never with a URL.
Where the milliseconds actually go
When a consumer says "the API is slow", the server's handler time is usually the smallest number in the trace. A single call from a mobile client pays connection setup (up to 2–3 RTTs cold: TCP plus TLS), one RTT for the request itself, server time to first byte, transfer time proportional to payload size over the link, and client-side parse time proportional to payload size again. On an 80ms-RTT connection, a cold call that transfers 200KB and spends 40ms in the handler costs roughly 240ms of handshakes + 80ms request RTT + 40ms server + ~300ms transfer on a 5 Mbps link — the handler is 6% of the total.
This arithmetic is why backend optimization so often disappoints: cutting the 40ms handler to 20ms wins 3%, while cutting the payload to 20KB or reusing a warm connection wins ten times that. The levers with leverage are contract-shaped — how many requests, how many bytes, whether a request happens at all — and they can only be pulled where the contract allows it.
The same arithmetic explains why performance is per-consumer, not per-API. The internal service calling over a 0.5ms datacenter hop feels none of this; for it, serialization CPU and connection pooling dominate. Naming the consumer before naming the fix is the whole discipline (see Consumer-First Design).
DNS + TCP + TLS ~240ms (3 RTTs, cold connection)
Request ~80ms (1 RTT)
Server TTFB 40ms (the part backend profiling sees)
Transfer 200KB ~320ms (5 Mbps ≈ 625 KB/s)
Client JSON parse ~8ms (mid-range phone, ~25 MB/s)
──────
~690ms → handler time is 6% of the experienceThe five levers, priced
Every API performance conversation reduces to five levers, and each one is a contract feature with a cost. The order below is roughly the order of leverage for network-bound consumers: the best request is the one that never happens, the second best is the one that shares a round trip with another.
Notice that none of these are implementation tricks. ?fields=, batch endpoints, cursors, ETag support and Content-Encoding negotiation are all clauses consumers program against — which means adding them later is easy (additive) but *relying* on them later requires consumers to change code. Shipping the levers with the API is cheap; retrofitting the consumers is not.
| Lever | What it saves | What it costs | Where it lives |
|---|---|---|---|
| Skip the request (caching, conditional GET) | Everything: RTTs, bytes, server work | Staleness rules you must state; cache-key discipline | Caching as a Contract Clause, Conditional Requests: ETags, 304 and 412 |
| Fewer requests (aggregation, batching) | RTTs × per-request overhead (auth, logging, rate checks) | Coarser endpoints to own; partial-failure semantics | API Granularity and the Chatty API, Batch APIs and Partial Failure |
| Fewer bytes (field selection, pagination) | Transfer + parse time, linear in payload | Response variability; more query surface to validate | Payload Size: 20KB, 200KB, 5MB, Over-Fetching and Under-Fetching |
| Cheaper bytes (compression) | 5–10× on JSON transfer | CPU on both ends; inverts on small or pre-compressed data | Compression: Cheaper Bytes, Not Fewer |
| Cheaper serialization (binary formats) | CPU + bytes for high-volume internal calls | Tooling, debuggability, a schema pipeline | gRPC: Schema, Codegen and Streams |
Design the budget, then spend it
The practical technique is a written latency budget per consumer task: "dashboard render ≤ 800ms on p75 mobile" decomposes into "≤ 2 round trips, ≤ 50KB total, cacheable for 30s". Now every contract decision has a test. An endpoint that returns 200KB fails the budget at review time, before any consumer measures it — and the review argument is arithmetic, not taste.
Budgets also stop the most common failure: optimizing a lever nobody is bottlenecked on. Compressing an internal datacenter API saves bytes nobody was waiting for while spending CPU someone will page about. Batching for a consumer who makes one call a minute adds partial-failure complexity for zero saved RTTs. The lever must match the consumer's bottleneck, and the bottleneck is measurable per task (see API Metrics: Rate, Errors, Duration, Sizes).
1GET /users/42 → 34KB (every column)2GET /users/42/projects → 120KB (all projects, all fields)3GET /users/42/activity → 210KB (unbounded history)4 5# 3 serialized round trips, 364KB, ~1.6s on p75 mobile6# the server spent 60ms; the contract spent the rest1GET /dashboard?fields=user,projects.name,activity.recent2Cache-Control: private, max-age=303ETag: "v81"4 5→ 200 OK · 28KB · one round trip6→ next render within 30s: no request at all7→ after 30s: If-None-Match → 304, ~200 bytesNothing about the second design is faster code — it is a contract that lets the consumer skip requests, share round trips and receive only the fields the screen reads. The 1.6s → 0.3s win happened at design review, not in a profiler.
Key points
- For network-bound consumers, RTT count and payload size dominate handler time — often by 10× — so performance is mostly a contract property.
- Five levers, in leverage order: skip the request, share round trips, send fewer bytes, send cheaper bytes, serialize cheaper.
- Every lever is a contract clause (
?fields=, batch endpoints,ETag,Content-Encoding) that consumers must code against — ship them early, additively. - Write latency budgets per consumer task and review contracts against them arithmetically.
- The lever must match the consumer's bottleneck: compressing a datacenter API or batching a once-a-minute caller spends complexity on a non-problem.
- Measure per task, not per endpoint: a fast endpoint called eight times serially is a slow task (see API Granularity and the Chatty API).
Payload Size Visualizer
Change the contract and observe which guarantee moves.
Follow the failure
How the contract fails or gets misused, hop by hop — and what it costs when it completes.
- 1Consumers → team: report "the API is slow" from mobile; the trace shows a 40ms handler, so the report is disputed.
- 2Team → backend: spends a sprint on query tuning and shaves the handler to 25ms; consumers measure no change.
- 3Team → infra: adds capacity and a regional replica; connection setup improves slightly, the 364KB of payload does not.
- 4Consumers → workarounds: mobile team builds its own aggregation proxy and cache with its own bugs and staleness rules.
- 5Team → v2: a "performance rewrite" finally changes the contract — under pressure, breaking consumers the levers would have served additively.
- User-perceived latency stays pinned to round trips × RTT + bytes ÷ bandwidth, no matter what the backend does.
- Provider capacity is spent serving fields and requests nobody needed: per-request overhead × N, payload × every caller.
- Trust erodes between teams: backend dashboards say "fast", consumer dashboards say "slow", and both are measuring honestly.
Design, observe, evolve
A contract decision is incomplete until you know how you would notice it failing and how it changes later.
- • Set a per-task latency budget (round trips, total bytes, cacheability) with the consumer, and review every endpoint against it.
- • Ship the levers as contract features from day one: field selection or task-shaped aggregates, pagination, `ETag`/`Cache-Control`, compression negotiation.
- • Instrument the consumer side of the budget (requests per task, bytes per task, cache hit rate) — server-side p99 alone cannot see the problem.
- • Choose the lever by the consumer's bottleneck: RTT-bound → fewer calls and caching; bandwidth-bound → fewer/cheaper bytes; CPU-bound internal → serialization and pooling.
- • Trace whole consumer tasks with [[request-ids]]: requests-per-screen, serialized depth and total bytes are the metrics that match user experience.
- • Track payload percentiles per endpoint and per consumer; p50 payload growth release-over-release is contract bloat announcing itself.
- • Compare server TTFB against client-measured total per task — a widening gap means the network share is growing and the contract levers are the fix.
- • All five levers add compatibly: `?fields=`, new aggregate endpoints, `ETag` support and compression can appear without breaking anyone — defaults must stay unchanged.
- • Removing weight later (slimming a fat default response) is the breaking direction; add the lean shape alongside and migrate consumers with usage telemetry (see [[consumer-driven-evolution]]).
- • Budgets are re-negotiated as consumers change: a new TV client or an offline-sync mode re-runs the arithmetic, not the architecture.
- • Contract-level levers add surface: field selection, batch semantics and cache headers all need validation, documentation and tests.
- • Caching and aggregation trade freshness and coupling for speed — each skipped request is a staleness decision someone must own.
- • Per-consumer budgets take coordination that a single "p99 < 200ms" SLO avoids; the single number is simpler and answers the wrong question.