The Connection Budget
A database accepts a finite number of connections. Every instance, worker, job and console session spends from the same pool — so the pool sizes have to add up.
The question, the obvious approach, and why it breaks
Every lesson starts where the work starts: an operational problem, a first attempt that is entirely reasonable, and the way production disagrees with it.
If the database allows N connections and we run M instances, what may each instance's pool be set to?
Pool size is configured per application, as if the application were the only consumer. The database has one limit, shared by everything, and nobody owns the total.
Set the pool to something generous so requests never wait for a connection — a hundred per instance sounds safe. The database will handle it.
The moment the application scales out, total demand multiplies. Ten instances at a hundred each is a thousand connections against a limit that is usually far smaller.
- The moment the application scales out, total demand multiplies. Ten instances at a hundred each is a thousand connections against a limit that is usually far smaller.
- When the limit is reached the database refuses *new* connections, which takes down healthy instances, background workers, the migration runner and the operator trying to log in and fix it.
- The failure looks like a total outage with an idle database, which sends everyone in the wrong direction for the first ten minutes.
- Autoscaling makes it worse exactly when you need it least: load rises, instances scale out, connection demand crosses the limit, and the scaling event causes the outage.
What is actually happening
Underneath the tooling, which is the part that survives a change of tool.
- This is arithmetic, not tuning. The database exposes a maximum connection count; every consumer holds some; the sum must stay under the maximum with a reserve left over.
- The consumers are more numerous than people count: application instances multiplied by pool size, background workers, scheduled jobs, the migration runner, the analytics or BI tool, connections held by a proxy, replicas, monitoring agents, and human sessions.
- The reserve is not optional. Superuser slots and an operator's ability to connect during an incident are the difference between diagnosing and guessing.
- A connection is not free. Depending on the engine it costs a process or a thread plus working memory, so raising the maximum trades the database's own memory for concurrency it may not be able to use.
- Pool size should be driven by concurrent in-flight queries, not by request rate. A pool larger than the database can execute concurrently just moves queueing from the application into the database, where it is harder to see and harder to shed (Queueing: Why Systems Get Slow Before They Get Broken in Observability).
- A connection pooler in front of the database decouples the two sides — many short client connections multiplexed onto few server connections — which is what makes serverless and high-instance-count architectures workable at all.
The arithmetic, written out
The reason this lesson exists is that the sum is almost never written down. Once it is, the answer to "what should our pool size be" stops being a preference and becomes a division.
Fill in the left column from your own system. The relationships hold at any size; the numbers are yours to measure.
1 database max connections MAX2- superuser / reserved slots (engine reserves some)3- operator sessions during an incident (reserve deliberately)4- monitoring and agents (one or more per host)5- migration runner and deploy tooling (peaks during deploys)6- analytics / BI / ad-hoc tools (spiky, often unmanaged)7- background workers and scheduled jobs (workers x their pool)8= budget available to the request-serving fleet BUDGET9 10 max pool per instance = BUDGET / MAX_INSTANCES11 12where MAX_INSTANCES is the autoscaler's ceiling, not today's count.13 14Two consequences people find surprising:15 * scaling out REDUCES what each instance may hold16 * during an incident, scaling out can be the thing that ends serviceThe division is the lesson. Everything else in this file is about who owns the numerator and who is allowed to spend from it.
Two ways to configure the same fleet
The difference is not that one number is smaller. It is that one configuration has an owner for the total and a defined behaviour when it is exhausted, and the other has neither.
service-a: pool = 50 # "generous, so we never wait" service-b: pool = 50 # copied from service-a workers: pool = 20 # nobody asked bi-tool: unbounded # not in any config we own autoscaler: 2 -> 40 instances # total at peak: unknown, unbounded, and larger than the limit
# budget.yaml - reviewed when tiers or scaling policy change limit: <from the instance tier> reserve: operators: <slots kept free for incidents> monitoring: <agents x hosts> migrations: <deploy tooling peak> allocations: service-a: <share> max_instances: <ceiling> service-b: <share> max_instances: <ceiling> workers: <share> bi-tool: <share, enforced by its own credential> rules: pool_size = allocation / max_instances acquisition_timeout set, so starvation errors fast CI fails a service whose pool x ceiling exceeds its allocation
The left version cannot be checked, because no artefact states the total. The right version makes the constraint reviewable, makes exceeding it a build failure rather than an outage, keeps an operator able to connect while it is happening, and — most importantly — makes the autoscaling ceiling part of the configuration rather than an unrelated setting owned by a different team.
How connection exhaustion presents
It is worth rehearsing the symptom, because it does not look like a database problem and the instinctive response makes it worse.
- The first row is the one to internalise: during connection exhaustion, adding capacity removes capacity.
- A reserve is what lets you observe any of this while it is happening. Spend it last.
- Attribution matters more than totals during an incident — you need to know which consumer took the budget, which means per-consumer connection metrics before you need them.
| Trigger | Symptom | Cause | Response |
|---|---|---|---|
| Traffic rises; autoscaler adds instances | Errors across all services; database CPU low | Instances x pool exceeded the limit; new connections refused | Scale in, not out; cap pool sizes; add a pooler before the next event |
| A deploy starts during peak | Both old and new instances failing to connect | Rolling deploy briefly doubles instance count and therefore connections | Budget for the overlap, or use a deploy strategy that does not double the fleet (Rolling: Two Versions, One Database) |
| An analyst runs a heavy query set | Production intermittently cannot get connections | An unmanaged consumer spending from a shared budget | Give ad-hoc tools their own credential with its own connection cap, and prefer a replica |
| A downstream dependency slows down | Pool exhausted although query rate is unchanged | Connections held for the duration of a slow request, so effective concurrency collapsed (Timeouts in Backend Engineering) | Timeouts at every layer; never hold a database connection across an external call |
| Idle-in-transaction sessions accumulate | Connections consumed with no query activity | A worker opened a transaction and stalled | Set an idle-in-transaction timeout at the database; alert on the count |
| Database tier resized down for cost | Exhaustion at a traffic level that was previously fine | The connection limit is derived from instance size | Recompute the budget as part of the resize change, not after the incident |
How to do it properly
Most important first.
- Write the budget down as an explicit sum, and treat the maximum instance count from autoscaling as the number that matters, not the current one (Autoscaling).
- Reserve headroom for operators, migrations and monitoring before allocating anything to applications.
- Enumerate every consumer, including the ones nobody thinks of: the BI tool, the ad-hoc console, the cron container, the sidecar.
- Size pools from concurrency, not from traffic. A small pool with a short acquisition timeout fails fast and visibly; a large pool fails slowly and invisibly.
- Cap application scale so that maximum instances multiplied by pool size still fits the budget, or put a pooler in the middle so that scaling does not translate into connections.
- Set an acquisition timeout so a starved pool produces a fast, attributable error instead of a hung request (Connection Pool Exhaustion in Backend Engineering).
- Alert on connection utilisation against the limit, with enough margin to act — this is a leading signal for an outage rather than a capacity curiosity.
How much can this affect
Every production change has a blast radius. Stated as a scale so it is comparable between changes rather than adjectival — and paired with what actually contains it, because a wide scope with a real containment mechanism is a different situation from a wide scope with none.
A reserve plus an acquisition timeout contains it: healthy services fail fast and attributably, and an operator can still connect. Without a reserve, nothing contains it — including your ability to fix it.
What can go wrong
- A scaling event exhausts connections, and the health checks that would show the problem cannot connect either.
- A single misbehaving consumer — an analytics tool, a runaway job — consumes the whole budget and starves everything else.
- "A bigger pool means more throughput." Throughput is bounded by what the database can execute concurrently. Beyond that, a bigger pool moves the queue somewhere less visible.
- "We are nowhere near the limit." Check at maximum autoscale, with the batch job running, plus the analytics tool. The limit is hit in the combination, not in the steady state.
- "The database is idle, so this is not a database problem." Connection refusal is a database limit presenting as an application-wide failure.
- "A pooler fixes connection problems." It fixes the arithmetic. It does not fix a query holding a connection for a long time, and it introduces modes that break session-dependent code.
Operating it
Evidence is the signal, not the intention. Rollback is sometimes 'you cannot, and that is the point'.
- A written budget: limit, reserve, per-consumer allocation, and the maximum-instance assumption it depends on.
- Peak observed connections plotted against the limit, with the gap visible on the operator dashboard.
- Pool acquisition wait time and timeout counts exported per service, so starvation is attributable to a consumer.
- A load test or a real scaling event in which instance count reached its maximum and connections stayed within budget.
- Reducing pool sizes is safe and fast; increasing the database limit is a parameter change that usually requires a restart or failover, which is a production event of its own.
- If you are already at the limit during an incident, the immediate levers are: kill idle-in-transaction sessions, cap the offending consumer, and scale the application *in* rather than out — which is counterintuitive and correct.
- Adding a pooler is a topology change, not a config tweak. It should be introduced and verified before you need it.
- Automate: connection utilisation alerting, per-consumer connection attribution, and a CI check that a service's declared pool size fits its allocation.
- Automate the arithmetic itself — derive the maximum allowed pool size from the declared budget and the configured maximum replica count rather than letting each team pick a number.
- Keep human: allocating the budget between competing consumers, which is a prioritisation decision about who degrades first.
- Small pools mean requests occasionally wait for a connection; large pools mean the database occasionally refuses everyone. The first is a latency cost, the second is an outage.
- A pooler removes the arithmetic constraint and adds a component on the critical path, with its own failure modes, its own limits, and transaction-mode restrictions the application must respect.
- A bigger database tier raises the limit and costs more continuously, while leaving the underlying "nobody owns the total" problem in place.
Where this applies
This domain is unusually tool- and organisation-dependent. These labels say what each claim is specific to, and what a different platform, provider or organisation does instead.
- DATABASE-SPECIFICWhat a connection costs differs by engine: PostgreSQL forks a backend process per connection with its own memory, so high connection counts are expensive and poolers are near-mandatory at scale; MySQL uses threads and tolerates more connections; some engines multiplex natively and make this arithmetic much less sharp.
- CLOUD-SPECIFICManaged services usually derive the connection limit from instance size, so resizing changes the budget without anyone editing a config file. Serverless and function-based compute makes the instance count effectively unbounded, which is why providers ship dedicated poolers for that case.
- PLATFORM-SPECIFICOn a fixed fleet, maximum instance count is known. Under an autoscaler or a per-request compute model it is a policy number, and the budget must be computed against the policy maximum rather than today's count.
Where the depth lives
This domain teaches delivery and operation, and hands the mechanism off to the domain that owns it.
- — System Design — where to put the pooling layer, and what it costs to make an inherently stateful resource look stateless to a fleet that scales freely.