PerformanceGENERALFRAMEWORK-SPECIFICSCALE-SPECIFIC

N+1 as a Design Problem

The classic N+1 is treated as a query bug and fixed with an eager-load hint. Often it is an interface that only knows how to answer about one thing at a time, called from a loop that had no alternative.

Constraints & invariants Who owns what What the next change costs

The requirement, the obvious build, and why it breaks

Every lesson starts where the work starts: someone asked for something, and the first implementation that comes to mind survives until the requirement changes.

The question

Why does the same N+1 keep coming back after it is fixed, and what in the design is producing it?

The requirement

"The order list should show each order's customer name and shipping status." Two extra columns, and the loader for each already exists as a per-order function.

The obvious build

Loop the orders and call the loader for each. It is the obvious code, it is correct, it reads clearly, and the loader is right there. When it turns out slow, add the eager-load hint the ORM documentation shows.

Why it breaks

The hint fixes this call site. The interface that produced the shape is unchanged, so the next list page, the next export and the next report each rediscover it — this is why the same N+1 appears three times a year in a codebase that has "fixed" it (Shotgun Surgery).

How it breaks as requirements change

The hint fixes this call site. The interface that produced the shape is unchanged, so the next list page, the next export and the next report each rediscover it — this is why the same N+1 appears three times a year in a codebase that has "fixed" it (Shotgun Surgery).
Half the instances are not ORM-shaped at all. A per-item HTTP call, a per-item cache lookup, a per-item permission check and a per-item feature-flag evaluation all have the same shape and none of them has a hint (Observability & Performance owns what they look like in a trace).
Eager loading over-fetches in the other direction: the hint that fixes the list page loads associations the detail page never needed, and now every read pays for the worst caller (Cost-Aware Interfaces).
When the fix is a hint rather than a shape, nothing prevents a regression. Someone removes an include while refactoring and nothing fails — the code is still correct, just fifty times more expensive.

Requirement→Constraints→Invariants→Responsibilities→Boundaries→Interfaces→State→Dependencies→Failure→Implementation→Tests→Feedback→Evolution

What limits the solution, and what must never stop being true

This domain leads with these two. A design that ignores its constraints is not a design, and an invariant nobody named is one nothing is protecting.

Constraints

The per-item loaders are used by a dozen other call sites where fetching one thing is exactly right (Fan-in and Fan-out).
The list is paginated at fifty, so the problem is fifty extra round trips rather than a million — big enough to matter, small enough that nobody treats it as urgent.
The shipping status comes from another service, so its N+1 is over the network rather than the database and no ORM hint can fix it.

Invariants

The number of round trips a request makes must be a function of the request, not of the number of rows it happens to return.
Whatever batching happens, the result per item must be identical to the result the per-item call would have produced (What Refactoring Actually Is).

Who owns what, and where the seams fall

Responsibilities decide boundaries; boundaries decide what an interface has to say.

Responsibilities

The data-access interface owns being answerable about a *set*, not only about an individual, whenever anything renders lists — which is almost every system.
The caller owns not calling a single-item operation in a loop; that is a real responsibility and it is unreasonable to place it there alone, which is why the interface has to help.
Nothing owns "remembering to add the hint". A responsibility that lives in a reviewer's memory is the thing this lesson is trying to eliminate (A Review Checklist Worth Reading).

Boundaries

The seam is at the data-access interface: it can offer byId and byIds, and the presence of the second is what makes the loop avoidable.
A second seam sits at the request boundary: a per-request loader that collects ids, batches, and resolves is the general answer when call sites cannot be restructured (Backend Engineering owns the mechanism for carrying it).
The domain boundary must not move. Batching is an access concern; if domain code starts taking lists of ids because the store prefers it, the persistence model has escaped (Dependency Direction).

The loop is not the bug

Look at the call site and the loop is obvious in hindsight. Look at what was available to the person writing it and there was no other option: the only operation that existed answered about one customer. The loop is not a mistake, it is the only way to use the interface.

That is the reframe. Fixing the loop fixes today; adding the set-shaped operation removes the possibility, and the second one is cheap precisely because it is the same query with a different predicate.

The interface that makes the loop inevitable, and the one that does not

1// only knows how to answer about one thing
2interface Customers { byId(id: CustomerId): Promise<Customer | null> }
3 
4for (const o of orders) {
5  const c = await customers.byId(o.customerId)   // one round trip each
6  rows.push({ ...o, name: c?.name })
7}
8 
9// answers about a set; the single case is the special case
10interface Customers {
11  byIds(ids: CustomerId[]): Promise<Map<CustomerId, Customer>>
12  byId(id: CustomerId): Promise<Customer | null>   // -> byIds([id])
13}
14 
15const found = await customers.byIds(orders.map(o => o.customerId))
16const rows = orders.map(o => ({ ...o, name: found.get(o.customerId)?.name }))

The second version makes the request cost constant in the number of orders — one round trip for orders, one for customers, one for statuses. Note what it does *not* do: it does not make anything faster for a single order, and it forces every caller to handle a missing key. The set-shaped operation is the honest one and it is slightly worse to use, which is why it has to be designed in rather than hoped for.

Where the shape actually gets decided

It is worth tracing this backwards, because teams usually intervene at the last box and the decision was made at the first. Every arrow here is cheap to change except the one at the end, and the one at the end is where the incident happens.

The same picture explains why the fix does not hold: patching the call site leaves the first two boxes intact, so the next feature walks the identical path.

The hint fixes one call site; the interface fixes the class of call sites (Change Amplification).
A per-request batching loader is the same fix applied at the boundary instead of at the interface, for when the interface cannot be changed.
If the data lives behind another team's service with no batch endpoint, this is not a code problem any more — it is a contract negotiation (Contract Tests).

From an interface choice to fifty round trips

UserLLMAgentToolDataDecisionHumanGuardrail

The variants that keep coming back

SIMULATEDThe rows describe shapes observed in Engineer Atlas's model codebases, not measurements from a production system; the relative claim — cost proportional to rows rather than to the request — is what transfers, and any specific latency figure would not.

Once you look for the shape rather than for the ORM symptom, it turns up in places that have nothing to do with a database. Each row below is the same design fact — a per-item operation reached from an iteration — and each is invisible in the diff that introduces it.

The same shape, wearing different clothes

Trigger	Symptom	Cause	Response
Rendering a list with an association	Query count scales with page size	Lazy association: iteration triggers a load, so there is no loop in the source at all	Set-shaped read plus a query-count assertion; the hint alone regresses silently.
A permission check per row	Authorisation dominates a list endpoint	`can(user, row)` is per-object and the list has many objects	Filter by policy in the query, or resolve the policy once per request (Least Privilege as a Design Decision).
A feature-flag or config lookup inside a mapper	A cheap-looking function is the top of the flame graph	Evaluation is per call and the call is per item	Resolve once at the request boundary and pass the value inward (Feature Flags and What They Cost).
A cache lookup per item	The cache is hit constantly and latency is unchanged	Network round trip per key defeats the point of caching a set	Multi-get. A cache with only a per-key interface has the same design flaw as the repository (Cost-Aware Interfaces).
Enriching from another service	Timeouts under load, and retries make it worse	One HTTP call per row, and no batch endpoint exists	Batch endpoint, or change the data flow so the enrichment is not per request (Partial Failure).
A nested list — orders, then items per order	Cost is multiplicative, not additive	The batch fix was applied at one level only	Batch per level: one round trip per level of nesting, not per node (Module Granularity).

How to build it

Most important first.

Design set-shaped operations from the start where lists exist: byIds(ids) next to byId(id), returning a map. It is the same query with a different predicate and it prevents an entire class of call site.
Make the individual operation a special case of the set operation, not the other way round, so there is one query to tune and one place where the shape lives (DRY: Knowledge, Not Lines).
Where a loop is unavoidable — deep object graphs, per-item authorisation — put a batching loader at the request boundary that collects and resolves in one round trip. The pattern is old and well understood (Decorator describes the shape it usually takes).
Count round trips in review. It is a design property readable from the code, and once a team asks the question routinely it stops shipping the shape (Review as Design Feedback — and Why It Arrives Too Late).
Assert it in a test, because this regresses silently. A query-count assertion around a list endpoint is the cheapest guard there is (What a Unit Is).
Do not batch by default. byIds over a set of three, on a page nobody loads, is machinery for nothing — the trigger is that a call site iterates (Premature Optimization, Reclaimed).

What the next change costs

The field this whole domain exists for. A structure is only better if it makes the change after this one cheaper — and it is worth saying which changes it does not help.

Cost of the next change

Adding a column to the list under a batch-shaped interface: one more batched read, and the request cost goes from k round trips to k+1 — constant in the number of rows.
Adding a column under the per-item interface: fifty more round trips, discovered later, and fixed with a hint that has to be added at every call site that renders a list.
Moving one of these reads behind a network boundary — the shipping status becoming a separate service is exactly this — is nearly free if the interface was already set-shaped, and is a rewrite of the call site if it was not. That is the strongest argument for designing set-shaped early (What Changes at the Network Boundary).
The cost that does not go away: byIds has to define what happens for ids that do not exist, and every caller has to handle it. A per-item call answers that with null and no ceremony.

What the recommended approach costs

Set-shaped interfaces are less pleasant for the common single-item case: a map lookup, a missing-key branch, and a call that reads less directly than find(id).
A batching loader adds machinery and a request-scoped cache with its own staleness rules, which is a real cost paid for by call sites that mostly did not need it.
Designing every read set-shaped before any list exists is speculative generality with a performance justification, and it is wrong as often as it is right (Speculative Generality).

What can go wrong

Failure modes

Batching is added and the implementation loops internally, so the interface is batch-shaped and the cost is unchanged. This passes review reliably, because the signature looks correct.
The batch query takes a list of ids so large it becomes its own problem — a query with fifty thousand parameters, or one that exceeds a limit and fails at a size no test used.
Order or duplicates are lost in the map round trip, turning a performance fix into a correctness bug — the most common way a batching change gets reverted.
A request-scoped loader caches within the request, and a mutation happens mid-request, so the second read returns a stale value. Batching quietly introduced a cache (Database Engineering owns what invalidating that costs).
The team fixes N+1 everywhere and creates 1+N-shaped over-fetching: one giant query that joins everything, for pages that needed a third of it (Over-Design and Under-Design).

Dependencies, and their direction

Batching couples callers to a map-shaped result and to the possibility of a missing key, which is real API surface and slightly less pleasant than an object per call.
A request-scoped loader introduces a dependency on request context in places that previously had none, which is a genuine architectural cost and the reason some teams refuse it (Hidden Global State).
Cross-service batching couples you to the other service having a batch endpoint. If it does not, the design problem is theirs and the honest options are caching or a different data flow (API Stability).

Misreads

"N+1 is an ORM problem." ORMs make it easy and invisible; they do not cause it. A hand-written HTTP call inside a loop is the same design, with no framework involved (Backend Engineering covers the framework half).
"Eager loading is the fix." It is *a* fix, per call site, and it over-fetches for every caller that did not need the association. It also regresses silently the moment someone removes it.
"So always batch." Batching a set of one is worse than not batching, and most reads are of one thing. The trigger is a loop, not a principle (Premature Optimization, Reclaimed).
"A join solves it." A join solves the round trips and can multiply rows, so the same data comes back once per combination. The shape changed and the cost may not have (Observability & Performance owns when a join is the cheaper shape).

Smells this explains

shotgun-surgery
feature-envy

Testing it, and how it ages

What to test, and at which boundary

Assert query count around list endpoints with a fixture of more than a couple of rows. This is the one performance-adjacent test worth having in the normal suite, because it is deterministic and fast (Testing as Design Feedback).
Test the batch operation for missing ids, duplicate ids and order independence — the three ways a batching change becomes a correctness bug.
Test the batch and single operations return equivalent results for the same input, so the special-case relationship is enforced rather than assumed (Property-Based Testing).
Do not assert wall-clock time here. The property being protected is the number of round trips, which is a structural fact and stable across machines.

How this design ages

N+1 migrates. Fixed at the database, it reappears at the cache, then at the service call, then at the permission check — because the shape belongs to the interface style rather than to any one store (Observability & Performance calls the general phenomenon bottleneck migration).
As systems decompose into services, per-item interfaces become expensive in a way no local optimisation fixes, and set-shaped interfaces are the thing that survives the split (Finding Seams).
Eventually some lists want a denormalised read model rather than batching, and that is a bigger decision with a consistency cost — worth making deliberately, and much later than teams usually make it (Consistency Boundaries).

Where this applies

This domain's advice is contested more than most. These labels say what each claim is specific to — and where CONTESTED appears, the note gives the strongest form of the opposing view rather than a caricature.

GENERALThe shape — a per-item operation called from a loop — is independent of storage technology and appears over databases, HTTP services, caches, file systems and permission checks alike; what differs is whether a framework offers a hint that hides it.
FRAMEWORK-SPECIFICORMs with lazy associations produce this without any loop being visible, because iteration over a collection triggers loads; in a codebase with explicit queries the loop is right there in the diff, so the design fix matters more in the first case and the review fix is sufficient more often in the second.
SCALE-SPECIFICAt page sizes of ten against a local database this is measurable and unimportant; the same shape against a service across a network, or over a page size of a thousand, is the difference between a page loading and a request timing out, and the design argument only becomes compelling at the second scale.

Where the depth lives

This domain teaches the codebase-level structure and hands the rest off.

Backendn-plus-one eager-loading-and-batching orm-mechanics orm-tradeoffs request-coalescing

Databasejoins query-execution query-tuning reading-explain

Performancen-plus-one-traces database-performance-signals trace-waterfall join-performance critical-path fan-out-latency

Domains that do not exist yet

— System Design — the per-item call across a network is also a fan-out reliability problem: fifty dependent calls make the request's success probability the product of fifty, which is a separate reason to batch that has nothing to do with latency.