The question this answers
If the producer is faster than the consumer and the queue has no limit, what actually stops it — and how does that failure present at 3 a.m.?
A log-shipping process reads lines from an application socket and enqueues them for a batching uploader. Arrival is 40 000 lines/s in a bad minute; the uploader manages about 25 000/s when the upstream API is healthy.
The queue buffer and the process heap it lives in. Every item still in the queue is retained memory that the garbage collector or allocator cannot reclaim, plus everything each item transitively references.
Resident memory attributable to the queue stays within a limit the operator chose, and the age of the oldest queued item stays inside a bound the system can state.
Synchronization exists to preserve this sentence. If a schedule can make it false, the code is wrong no matter which primitive it uses.
What "unbounded" actually means
An unbounded queue makes put() a total function: it always succeeds, it never blocks, and it never returns an error. That is exactly why people choose it — the enqueue site needs no error handling and no policy decision. The cost is that the queue has quietly taken on responsibility for something it cannot do: it has promised to store an arbitrary amount of data in a process with a finite heap.
So the backpressure did not disappear. It moved, and it changed type. Instead of a producer blocking for 200 ms or a caller receiving a 429 it can retry, you get a process that grows until the allocator fails or the kernel's OOM killer selects it, at which point every item in the queue is lost at once, along with whatever else the process was doing. An unbounded queue is not "no backpressure" — it is backpressure via OOM. That is the sentence worth remembering from this lesson.
And memory is the *second* failure. The first is latency. By Little's Law the average time an item spends in the system is the queue length divided by the throughput; if the queue is 400 000 items deep and the uploader does 25 000/s, the head item has been waiting 16 seconds and every new item will wait longer. Long before the heap is exhausted, the pipeline is emitting data that is minutes stale and nobody has noticed, because the only symptom is that the numbers on the dashboard are old. See Little's Law as Working Intuition and Depth Is Not an Emergency; Age Is in Performance.
time arrival/s drain/s depth rss head_age 10:31:00 26,100 25,300 4,120 512 MB 0.16s 10:32:00 39,800 25,100 884,300 1.9 GB 35.2s <- upstream API slowed; depth is now the story 10:33:00 41,200 24,800 1,870,000 3.6 GB 75.4s <- data is over a minute stale, no alert fired 10:34:00 40,600 24,900 2,810,000 5.4 GB 112.9s 10:34:47 - - - - - <- SIGKILL from the OOM killer; 2.8M lines gone 10:34:52 4,300 0 0 68 MB 0.00s <- restarted clean. every graph looks healthy again.
What bounding forces you to decide
Putting a bound on the queue does not solve the rate mismatch. Nothing solves a rate mismatch except making the consumer faster or the producer slower. What the bound does is force the decision *at design time*, in code you wrote, instead of at 3 a.m. in a mechanism you did not choose. When the queue is full, one of four things happens, and you pick which: the producer blocks, the enqueue is rejected, the oldest item is evicted, or the newest is dropped.
For the log shipper, blocking is wrong — it would stall the application socket reader and eventually the application itself, which is a much worse outcome than losing log lines. Rejecting means the reader gets an error it must handle. Dropping the oldest keeps the most recent logs, which is usually what an operator wants during an incident. Dropping the newest is the one you almost never want and the one several default executor policies pick for you. Whichever you choose, the drop must be counted, because a silent drop is indistinguishable from "nothing happened".
The bound itself is a memory budget, not a magic number. Work backwards: how much RSS can this queue own, and how big is one item including everything it references? 256 MB of budget at 400 bytes per line is roughly 650 000 items. That is a number you can defend in review, unlike "10 000, which felt right". Size it in bytes if the library lets you, because item size is the variable that changes when someone adds a field.
| # | Reader (producer) | Bounded queue (cap 650k, reject) | Unbounded queue | Uploader (consumer) | State |
|---|---|---|---|---|---|
| 1 | enqueue burst — 40k/s arriving, 25k/s draining | · | · | · | bounded_depth=620k unbounded_depth=620k bounded_rss=248 MB |
| 2 | · | depth reaches capacity | · | · | bounded_depth=650k bounded_rss=260 MB |
| 3 | · | reject enqueue; increment dropped_total | · | · | bounded_depth=650k dropped_total=15k bounded_rss=260 MB |
| 4 | · | · | accept enqueue; depth grows | · | unbounded_depth=1.2M unbounded_rss=2.3 GB dropped_total=0 |
| 5 | · | · | · | drain 25k from each | bounded_depth=625k unbounded_depth=1.18M |
| 6 | · | · | depth grows past 2.8M | · | unbounded_rss=5.4 GB head_age=112s ✕ Memory budget exceeded and the oldest item is nearly two minutes old — the queue is no longer bounded in memory or in latency. |
| 7 | · | · | allocation fails / OOM killer selects the process | · | unbounded_depth=0 lost=2.8M items unbounded_rss=— ✕ Every accepted item was lost at once, including the ones already acknowledged to the producer. |
| 8 | · | burst ends; queue drains normally | · | · | bounded_depth=12k dropped_total=190k bounded_rss=5 MB |
The cases where unbounded is defensible
The honest version of this lesson admits there are cases. An unbounded queue is fine when the total amount of work is *known and small* — a fixed fan-out over 50 shards, a batch job that enqueues one item per input row of a bounded input. Here "unbounded" is a statement that you have already bounded it elsewhere, and adding a capacity would be ceremony.
It is also common inside single-threaded event-loop runtimes, where the microtask queue and the callback queue are effectively unbounded and the runtime relies on the fact that each task is short. That works right up until an unbounded fan-out floods it, which is Unbounded Concurrency and Blocking the Event Loop rather than a queue problem. The rule of thumb is not "always bound", it is: if the input rate is controlled by something outside your process, the queue must be bounded, because that is the definition of a queue you cannot reason about.
The matrix below is the review checklist. Note the middle row — the one that catches people — where the queue is bounded but the *items* are not, so a 10 000-item cap holds 10 000 objects that each retain a 2 MB response body. The bound is on count and the failure is still memory. Bound what actually grows.
| Situation | Bound? | Full-queue policy | What you must also do |
|---|---|---|---|
| Input rate controlled by external clients or another service | Yes, always | Reject or block, depending on whether the caller can retry | Export depth, age and rejection counters; alert on age, not depth |
| Bounded count, unbounded item size (queued HTTP bodies, buffers) | Yes — bound bytes, not items | Reject on byte budget | Measure real item size including retained references; a count cap is not a memory cap |
| Fixed, known fan-out (one task per shard, per input row of a finite batch) | Not required | n/a — it cannot fill | Assert the size assumption; a "finite" input that becomes a stream is a silent regression |
| Telemetry, metrics, live sampling | Yes | Drop oldest | Count drops. A gap in a dashboard with no counter is unfalsifiable |
| Internal stage-to-stage handoff inside one pipeline | Yes, small | Block | Ensure the stall propagates to a place that can shed load — see Backpressure |
Key points
- An unbounded queue is not "no backpressure"; it is backpressure whose mechanism is memory exhaustion and whose signal is a process restart.
- Latency fails before memory does: by Little's Law a deep queue means stale data long before the heap is exhausted.
- Bounding does not fix a rate mismatch. It converts an uncontrolled failure into a policy you chose and can count.
- A count-based bound on items that hold large payloads is not a memory bound — bound bytes when item size varies.
- Unbounded is defensible only when the input is already bounded by something outside the queue, and that assumption should be asserted.
The loop, answered
Every field is required, which is why no lesson here can recommend concurrency without naming the interleaving that breaks it, the complexity it adds, and the simpler thing to consider first.
- • A bounded queue stores capacity alongside the buffer and checks it inside the same critical section as the insert, so the check and the insert cannot interleave.
- • On a full queue the configured policy runs: wait on a "not full" condition, return a rejection to the caller, or evict an existing item and record the eviction.
- • A consumer removing an item signals "not full", which admits at most one blocked producer per signal — a bounded queue is a semaphore with a payload (Semaphores: Counting Permits as a Resource Limit).
- • An unbounded queue skips the capacity check entirely, so the enqueue path allocates: the growth is heap allocation, and its ceiling is whatever the process is allowed to allocate.
- • When allocation fails, behaviour is runtime-specific: an exception, a hard abort, or the kernel killing the process — none of which unwinds the queued work.
- • Producer at 40k/s, consumer at 25k/s, unbounded: depth grows by 15k every second, monotonically, with no interleaving that recovers it. The absence of a failing schedule *is* the failure — nothing in the code can stop it.
- • Bounded, blocking: P attempts put on a full queue and waits; C removes one and signals; P inserts. Throughput is now pinned to the consumer's rate, which is the intended coupling.
- • Bounded, rejecting: P attempts put on a full queue, receives a rejection, and increments a counter. The loss is deliberate and observable at the exact moment it occurs.
- • The subtle one — bounded queue, drop-oldest policy: C reads the head item, and before C finishes, P's eviction removes what C is holding a reference to. If eviction and dequeue are not in the same critical section, an item is processed *and* counted as dropped.
- • Bounded queue as a deadlock: a consumer that enqueues follow-up work into the same full queue blocks forever, because the only thread that could drain it is the one now blocked (The Four Conditions).
- • A bounded queue guarantees an upper bound on the *number* of items retained. That is all it guarantees about memory.
- • It does NOT bound latency. Capacity 650 000 at 25 000/s drain is a 26-second worst-case wait, which may be far outside anything the system can promise.
- • It does NOT bound memory when item size is variable — the bound is count × size, and size is the term nobody re-checks after a schema change.
- • It does NOT prevent data loss. Rejecting and dropping are losses; blocking converts loss into stalling, which may be worse for a request path.
- • An unbounded queue guarantees exactly one thing:
put()never fails. It makes no promise at all about the process surviving that.
- • The full-queue condition is a second waiting point: producers now contend on "not full" as well as on the queue lock, and under sustained overload every producer thread is parked there.
- • Blocked producers are usually request threads or connection handlers, so queue contention converts directly into connection-pool and socket-accept pressure upstream (Connection Pool Saturation: Waiting in Front of an Idle Database in Performance).
- • An unbounded queue has less lock contention and more allocator contention — a growing buffer means reallocation and GC pressure that shows up as pauses rather than as waits.
- • Drop-oldest policies contend at the head, where consumers also operate, so the eviction path is the one most likely to be raced if it was bolted on later.
- • Memory exhaustion: allocation failure or an OOM kill, losing every in-flight item at once.
- • Unbounded latency: the queue is functionally healthy and the data coming out of it is minutes stale.
- • Silent data loss when a drop policy is chosen and never instrumented.
- • Deadlock when a consumer feeds the bounded queue it consumes from and capacity is reached.
- • GC death spiral in managed runtimes: a large live queue makes every collection scan more, which slows the consumer, which grows the queue.
- • Restart amnesia — the process comes back with an empty queue and healthy graphs, so the incident leaves almost no trace to investigate.
- • A bound helps whenever the arrival rate is outside your control, which is the normal case for anything fed by clients, another service, or a message broker.
- • It helps most at the moment of overload, which is exactly when you have the least attention available to make a decision — the decision is already made and encoded.
- • A small bound helps stage-to-stage inside a pipeline: it keeps the whole pipeline's memory proportional to the number of stages rather than to the burst size.
- • Unbounded genuinely helps when the work set is provably finite and small, and a capacity would only add a failure branch nobody will test.
- • A bound hurts when it is set without a memory model behind it — an arbitrary number produces arbitrary rejections and teaches the team that rejections are noise.
- • Blocking on a full queue hurts on a request path: the caller experiences a 30-second hang instead of a fast, retryable error.
- • A tight bound on a bursty-but-balanced workload rejects work that would have drained fine in two seconds. Bound for the burst, not for the mean.
- • Bounding without a drop counter is worse than not bounding, because you have converted a loud failure into a quiet one.
- • Queue depth *and* head age together. Depth alone cannot distinguish "big buffer, fast drain" from "data is two minutes old".
- • Rejection and eviction counters, as monotonic counters, with an alert on a rate rather than on a threshold.
- • RSS attributable to the queue: depth × measured item size. Measure item size once, in a heap dump, not by reading the struct definition.
- • Producer block time, if the policy blocks. Non-zero block time on a request thread is a user-facing latency source.
- • The restart signal: a process that dies with the exit code for SIGKILL and comes back clean is the OOM-killer signature. Correlate restarts against the depth graph from just before ("What Changed?" — Deploy Markers and the Invisible Deploys, OOM Kills and CPU Throttling in Cloud).
- • Bounding introduces an error path at every enqueue site, and every one of those sites now needs a decision about what to do with the rejected item.
- • The bound becomes a tuning parameter with an owner, a rationale and a review requirement — a number nobody can justify will be doubled at the first incident and never revisited.
- • Byte-based bounds require measuring item size, which requires a heap profile, which is work most teams do only after the first OOM.
- • Drop policies require a loss budget conversation with whoever consumes the data, and that conversation is usually the reason the queue was left unbounded in the first place.
- • Make the consumer faster or add consumers — the only actual fix for a sustained rate mismatch. A bound is damage control, not a solution (Worker Pools Beyond Threads).
- • Shed load at the entry point instead of at the queue: reject or sample at admission, where the caller still has context to retry or degrade (Rate Limiting in Architecture).
- • Spill to disk or to a durable broker, trading memory for latency and operational surface, when the burst is genuinely large and losing it is unacceptable (Message Queues in Architecture).
- • Remove the queue: if the producer can simply run the work itself when the pool is saturated ("caller runs"), the producer slows down naturally and no buffer exists to overflow.
The producer is faster than the consumer
What people believe, and what is true
Unbounded queues have no backpressure.
They have exactly one backpressure mechanism: memory exhaustion. It is slow, invisible, and takes the whole process rather than one request.
If it never OOMs in production, the unbounded queue is fine.
It has not been tested at the arrival rate that matters. The property you need is not "it has not failed", it is "the failure mode is one I chose".
A capacity of 10 000 caps memory.
It caps count. If each item retains a response body, 10 000 items can be 20 GB. Bound the thing that grows.