LayersGENERALSCALE-SPECIFICRUNTIME-SPECIFIC

Fat Controllers

The 300-line handler is a real production problem for a specific reason: it puts a payment call inside a database transaction and nobody can see it.

What actually happensHow to build it

The requirement, the obvious build, and why it breaks

Every lesson starts where the work starts: someone asked for something, and the first implementation that comes to mind has a problem.

The question

What actually goes wrong when the handler does everything, beyond it being hard to read?

The requirement

The checkout endpoint has grown over eighteen months: coupons, tax, gift cards, fraud checks, partner referrals, a receipt email and an analytics event. It works.

The obvious build

Keep adding to it. The whole flow is in one file, in order, and anyone can read it top to bottom — which is more than can be said for a version split across six files.

Why it breaks

The transaction opens at line 40 and closes at line 260. Between them are two HTTP calls to payment and tax providers. Every in-flight checkout holds a pooled database connection for the duration of a third party's p99, and at 30 concurrent checkouts the pool is empty and unrelated endpoints start timing out (Connection Pool Exhaustion).

How it breaks in production
  • The transaction opens at line 40 and closes at line 260. Between them are two HTTP calls to payment and tax providers. Every in-flight checkout holds a pooled database connection for the duration of a third party's p99, and at 30 concurrent checkouts the pool is empty and unrelated endpoints start timing out (Connection Pool Exhaustion).
  • The handler cannot be called from anywhere else, so the retry job for failed payments has a second copy of the flow with the fraud check missing.
  • Testing the "coupon expired while a gift card was applied" branch requires an HTTP fixture, an auth token and a running server, so it is not tested. It is the branch that broke.
  • Error handling is per-branch: fourteen try/catch blocks, three of which swallow and continue, and one of which returns 200 with an empty body (Error Boundaries: Three Translations, Not One).
  • Four engineers edit the same 300 lines every sprint, so every merge is a conflict in the middle of the transaction block.
  • The trace is one flat span called POST /checkout lasting 4 seconds, with no way to attribute the time (Tracing From the Backend's Side).
RequirementAPI ContractApplication LogicData AccessExternal DepsConcurrencyFailureSecurityObservabilityDeploymentScale

What is actually happening

  • The readability complaint is the least important one. The real problem is that a long handler hides boundaries: where the transaction starts and ends, where the process leaves for the network, where a failure is recoverable.
  • The specific killer is a network call inside a transaction. A pooled connection is a finite, shared resource; holding it across a call you do not control converts a slow dependency into a total outage (External Calls Inside a Transaction, Connection Pools).
  • A handler is a transport adapter. Its job is protocol translation: parse the request into a command, call something, translate the result into a status and a body. Everything else it does is work that has no other caller by construction.
  • The mirror failure exists and is real: the anemic handler, where the handler forwards to a service that forwards to a repository that calls the ORM, with no behaviour at any level. That is When the Repository Is Just Indirection applied to the whole stack, and it is not the cure.
  • The cure for a fat handler is not "more layers". It is moving the *specific* things that have another caller, another lifetime, or a boundary that must be visible.

The edit that matters, and the edit that only looks like it matters

There are two different changes people mean by "fixing a fat controller", and only one of them stops an outage. Moving code into a service file changes who can call it. Moving the network call outside the transaction changes what happens when the payment provider is slow.

If you only ever make one of these changes, make the second. It is smaller, it is safer, and it is the one that shows up in the pool metrics the same afternoon.

Where the transaction ends
Network calls inside the transaction
app.post('/checkout', async (req, res) => {
  await db.transaction(async (tx) => {
    const cart  = await tx.cart.findUnique({ where: { id: req.body.cartId } })
    const tax   = await taxApi.quote(cart)        // ~200ms, someone else's p99
    const fraud = await fraudApi.score(cart)      // ~150ms
    const charge = await stripe.charges.create({  // ~500ms, sometimes 8s
      amount: cart.total + tax.amount,
    })
    await tx.order.create({ data: { chargeId: charge.id } })
    await mailer.sendReceipt(cart.email)          // in the request path too
  })
  res.status(201).json({ ok: true })
})
The transaction holds only database work
app.post('/checkout', async (req, res) => {
  const cmd = parseCheckout(req)                  // untrusted -> typed
  const result = await checkout(deps, cmd)
  res.status(result.ok ? 201 : STATUS[result.reason]).json(toBody(result))
})

async function checkout(deps, cmd) {
  const cart   = await deps.carts.find(cmd.cartId)          // no transaction yet
  const tax    = await deps.tax.quote(cart)                 // outside
  const fraud  = await deps.fraud.score(cart)               // outside
  if (fraud.reject) return { ok: false, reason: 'rejected' }
  const charge = await deps.payments.charge(cart, tax, cmd.idempotencyKey)

  return deps.db.transaction(async (tx) => {                // milliseconds
    const order = await deps.orders.insert(tx, cart, charge.id)
    await deps.outbox.append(tx, 'order.placed', order.id)  // email via the outbox
    return { ok: true, order }
  })
}

In the first version each in-flight checkout holds one of a fixed number of pooled connections for roughly 850 ms of someone else's latency, so a 10-connection pool saturates at about a dozen concurrent checkouts and every other endpoint in the service starts waiting on the pool. In the second, the connection is held for the two inserts only — a few milliseconds — so the same provider slowdown makes checkout slow and leaves the rest of the service working. The email moves to the outbox because it must not be able to fail the order, and must not be re-sent when the request is retried (The Transactional Outbox).

What a handler is allowed to do

A useful handler has a shape, and the shape is short enough to check in review. Four steps, in order, with everything else delegated. The value is not the brevity — it is that a fifth responsibility is immediately visible as a fifth block.

Each step has a characteristic failure, and they are worth knowing separately because they produce different-looking incidents.

The four things a handler does
  1. 1
    Parse

    Turns the untrusted request into a typed command, rejecting what cannot be meaningful (Parse, Do Not Validate).

    fails by Passing req.body onward unparsed, so every layer below re-checks or trusts it (Mass Assignment and Over-Posting).

  2. 2
    Establish the actor

    Resolves the authenticated principal and the tenant from verified credentials, never from the body (The Trust Boundary).

    fails by Reading body.tenantId; or checking authentication and calling it authorization (Authentication vs Authorization).

  3. 3
    Delegate

    Calls one application function with the command and the actor.

    fails by Doing the work here instead — the fat version; or calling six functions and orchestrating them, which is the same thing spread out.

  4. 4
    Map the result

    Turns a domain result into a status code and a response body (Status Codes From the Server's Side).

    fails by Returning internal error text or the raw ORM entity (Not Leaking Your Internals, Schema Leakage).

Cross-cutting concerns — logging, correlation id, rate limiting, error mapping — are not steps in this list because they belong to the pipeline around it (The Middleware Pipeline).

How it shows up in production

A fat handler does not page anyone for being long. It pages someone for one of these, and the connection between the symptom and the shape of the code is the thing to internalise.

Note how many of these are triggered by a dependency behaving badly rather than by your own code changing. That is the signature: the code was fine until something else was slow.

TriggerSymptomCauseResponse
Payment provider p99 goes from 400 ms to 6 sEvery endpoint slow; /health times out; database CPU is idleConnections held in idle in transaction across the external callMove the call outside the transaction; add a timeout to it (Timeouts)
Traffic doubles at a launchErrors start exactly at N concurrent requests and not beforeThe pool is the binding constraint and it is a hard limit (Connection Pools)Shorten transactions first; resize the pool second, with the database's own limit in mind
A coupon edge case in productionCannot be reproduced in a testThe branch is only reachable through HTTP with an auth tokenExtract the flow so the branch is callable directly (A Test Strategy Chosen by What Each Layer Can Prove)
A client retries a timed-out checkoutTwo charges, one orderThe flow is not keyed, and the retry re-runs the whole handlerIdempotency key threaded to the payment provider (Idempotency Keys)
Four engineers edit checkout in one sprintEvery merge conflicts inside the transaction blockOne file is the whole featureSplit by boundary, not by line count
An unhandled branch throws500 with a database error message in the bodyOne of fourteen catch blocks returns err.messageOne error boundary in middleware, mapping taxonomy to status (Error Boundaries: Three Translations, Not One)

How to build it

Most important first.

  • Move the network calls out of the transaction first. This is the change that stops the outage, and it can be made without reorganising anything else.
  • Extract by boundary, not by line count. A 200-line handler with one transaction, no external calls and one caller may be perfectly fine; a 40-line one that charges a card inside a transaction is not.
  • Give the handler a fixed shape — parse, authorize, delegate, map — so that a reviewer can see at a glance whether it has grown a fifth responsibility (What a Handler Is Responsible For).
  • Move deferred work to a job rather than into a helper: the receipt email and the analytics event do not need to be in the request path at all (Request or Background?).
  • Push cross-cutting concerns into the pipeline: authentication, correlation ids, rate limiting and error mapping belong to middleware, not to fourteen catch blocks (What Belongs in the Pipeline, The Error Boundary).
  • Extract the pieces with a second caller, and leave the pieces without one inline. Extraction that produces single-caller passthroughs has traded one problem for a duller one.

What can go wrong

Failure modes
  • Extracting into "helpers" that still take req and res. The handler is now short and nothing has moved — the transport dependency is intact and there is still only one possible caller.
  • Splitting a transaction across two extracted functions and losing atomicity, so a failure leaves a half-written order (Where the Transaction Boundary Goes).
  • Extracting so far that the flow is unreadable: eleven files to follow one checkout, each one line long (Alternatives to Layering).
  • Moving the email to a background job without making the job idempotent, so a retry sends two receipts (Job Idempotency).
  • Leaving the fat version in place for one caller "temporarily" while the new one is used elsewhere, so both drift.
What can race
  • A long handler usually reads state at the top and writes at the bottom, with hundreds of milliseconds of external calls between. That gap is the largest check-then-act window in most codebases (Backend Races).
  • Retries against a fat handler re-execute the whole flow, including the parts that already succeeded, unless the operation is keyed (Idempotency Keys).
Security
  • Long handlers are where authorization checks go missing: with fourteen branches, one path reaches the write without passing the check, and code review does not catch it because the check is visibly present three branches up (Object-Level Authorization).
  • Per-branch error handling leaks internals: one of the fourteen catches returns err.message, which on a database error is a table and column name (Not Leaking Your Internals).
  • A handler that passes req.body into an update is mass assignment; the distance between the parse and the write is what makes it invisible (Mass Assignment and Over-Posting).
Misreads
  • "Handlers should be under 20 lines." Line count is a proxy, and a bad one. A long handler with no transaction and no external call may be the clearest thing in the codebase.
  • "Extract everything into services." Single-caller passthroughs are the anemic mirror image and are not an improvement (When the Repository Is Just Indirection).
  • "The problem is readability." The problem is that a pooled connection is held across a third party's latency. Readability is what stopped anyone noticing.
  • "Moving code out of the handler fixes the pool problem." Only if it also moves out of the transaction. A helper called inside db.transaction holds the connection exactly as long.
  • "This is a code-quality issue, so it can wait." The pool exhaustion version is an availability issue with a code-quality appearance (Backend Code Smells).

Operating it

How you see it in production
  • Look at the transaction duration histogram, not the handler length. pg_stat_activity showing sessions in idle in transaction for seconds is the fat handler's signature (Connection Pool Saturation: Waiting in Front of an Idle Database).
  • Look for a single span covering most of the request. Once steps are extracted and instrumented, the same trace attributes the 4 seconds to the tax provider (Reading the Waterfall).
  • Count distinct authors per file per quarter. A file with four regular authors and 300 lines is a coordination cost you can measure before it is an argument.
  • Count branch coverage on the handler. Low coverage on a high-branch file is the specific risk, not the line count.
What changes at 10x and 100x
  • At 10x traffic the transaction-inside-network-call pattern is the first thing to fail, and it fails as a full outage rather than as gradual slowdown: the pool is a hard limit (Resource Limits).
  • At 10x team size the single-file bottleneck dominates. The code is not worse; the queue of people waiting to change it is.
  • Extraction does not change throughput. Moving the external call out of the transaction changes it enormously, and those are different edits — do the second one first.
What this costs
  • A well-extracted flow costs you the ability to read the whole thing top to bottom. That was a genuine benefit of the fat version and it is genuinely lost.
  • Moving the external call outside the transaction usually means accepting a window where the charge succeeded and the order row does not exist yet, which you must then reconcile (The Dual Write Problem, The Transactional Outbox).
  • Deferring the email to a job introduces a queue, a worker, a retry policy and a dead-letter path — real operational surface in exchange for a shorter request (Dead-Letter Queues).

Where this applies

Backend advice is context-sensitive. These labels say what each claim is specific to, and where a different stack or scale would differ.

  • GENERALHolding a pooled connection across an uncontrolled network call is bad on every runtime, language and framework.
  • SCALE-SPECIFICFlips on concurrency versus pool size, and it flips sharply rather than gradually. Below the pool size in concurrent checkouts, a transaction wrapping an external call is invisible — latency is normal and nothing queues. Above it, every additional request waits for a connection and endpoints with no relationship to checkout start timing out. A 10-connection pool and a 500 ms provider means the wall is at roughly 20 checkouts per second, and there is no warning on the way to it.
  • RUNTIME-SPECIFICHow the wall feels differs: on Node the awaited external call yields the loop, so the process stays responsive while every request piles up on the pool and the symptom is uniform slow queries; with a thread-per-request runtime the blocked threads are themselves the scarce resource, so the process stops accepting work and the symptom is refused connections. Same cause, different-looking incident.

Where the depth lives

This domain teaches the application-side mechanism and hands the rest off.