The question this answers
I need ordered scans across a partitioned dataset. What does keeping keys in order cost me?
Keys are totally ordered within a partition and across partition boundaries, so a scan over [a, b) touches only the partitions whose ranges intersect [a, b). There is no guarantee of even distribution: partition load follows the data distribution and the access pattern, both of which change without warning.
Everything below is bought to hold this sentence. "Strongly consistent" with no scope attached is a slogan, not a guarantee — read what it actually covers, and what it explicitly does not.
A node knows the boundaries of the ranges it owns and the split points it has been told about. It does not know the global key distribution — it can only observe its own ranges’ size and traffic, which is why split decisions are local heuristics rather than a global optimisation.
A node knows its own state and the messages that arrived. Everything else is inference from evidence that was already stale. "B has not replied in five seconds" is knowledge; "B is down" is a decision — and usually the bug.
The trade in one line
Hash partitioning destroys order to buy uniformity. Range partitioning keeps order and gives up uniformity. Everything else in this lesson follows from that sentence.
Concretely: with ranges, WHERE ts BETWEEN '2026-08-01' AND '2026-08-02' reads a contiguous run of keys from one or two partitions. Under hashing, the same query fans out to all of them and merges. For time-series, logs, event feeds, ledgers, leaderboards and cursor pagination — all of which are ordered access patterns — that difference is the difference between a workable system and an unworkable one.
The corresponding cost: a hash guarantees uniformity mathematically, whereas ranges require you to *choose split points*, and the right split points depend on a distribution you do not control and cannot predict.
| Property | Hash partitioning | Range partitioning |
|---|---|---|
| Point lookupprotocol | One partition, no lookup needed | One partition, needs a boundary lookup |
| Range scanprotocol | Fan-out to every partition, then merge | Only the intersecting partitions |
| Balance of bytesassumption | Uniform by construction | Follows the data; requires splitting to maintain |
| Balance of writestypical | Uniform unless one key is hot | Catastrophically skewed for monotonic keys |
| Adding a nodetypical | Reassign partitions or ring segments | Reassign ranges — same cost, plus possible splits |
| Metadatatypical | A function, or a small map | A boundary table that changes as ranges split and merge |
The monotonic key disaster
Here is the failure that catches every team once. You range-partition an events table by timestamp. At any moment, *every* write has a timestamp near now, so every write goes to the partition that owns the current time. One node takes 100% of the write load; the other nineteen serve historical reads and are otherwise idle. Then that partition splits, and the new right-hand half takes 100% of the write load. You have built a single-writer system with extra steps.
The identical shape appears with auto-increment ids, ULIDs and UUIDv7 (both are deliberately time-ordered), snowflake ids, and any key prefixed by a sequence number. It is not an edge case; it is the default outcome of range-partitioning on the most natural key.
Three ways out, each with a cost worth stating precisely:
- Reorder the composite key.
(device_id, ts)instead of(ts, device_id). Writes spread across devices; scans within one device remain contiguous and cheap. Scans across *all* devices for a time window now fan out. This is the right answer far more often than it is used. - Salt the prefix. Prepend
hash(key) % 16so writes land in 16 buckets. Write load is spread 16×; every time-range scan must now read 16 ranges and merge them. You have partially converted to hashing and pay a partial fan-out. - Bucket by a coarse time unit plus a spreading field.
(yyyy-mm-dd, shard_id, ts). Scans of one day touch a bounded number of partitions; writes spread acrossshard_id. A good compromise, and the usual shape in production time-series schemas. - Accept it and size for it, if the write rate genuinely fits one node. A hotspot that fits is not a problem, and this is a legitimate choice with a clear tripwire: alert on that partition’s write rate approaching node capacity.
- [2026-01 … 2026-04) — reads only · 4% CPU
- [2026-04 … 2026-07) — reads only · 6% CPU
- [2026-07 … 2026-08) — reads only · 9% CPU
- [2026-08 … ∞) — 100% of writes · 97% CPU · queue growing
- r4believes “I am overloaded and should split”✓ and it is true
- r4believes “splitting will halve my write load”✕ and it is false
Every node above is acting on what it believes. Nothing in the cluster tells the mistaken one that it is mistaken.
Splitting and merging: the part that is actually hard
Range partitioning is dynamic in a way hashing is not. Ranges grow, so they must split; ranges shrink or empty out, so they should merge. Systems like Bigtable, HBase, CockroachDB, TiKV and Spanner all run this loop continuously, and it is the source of most of their operational complexity.
A split is a metadata change plus a data reorganisation. You pick a midpoint, create two ranges, update the boundary table, and tell every router. The window between "the split committed" and "every router knows" is a window in which requests arrive at a node that no longer owns the key — the same stale-map problem as everywhere else in this module, and the reason these systems return an explicit *retry with new routing information* error rather than serving or failing.
Choosing the midpoint is a sampling problem. You want the point that halves the *load*, but you can cheaply measure only size. Splitting a hot range by size does nothing if the heat is on one key: you get one hot half and one cold half, then the hot half splits again. Systems that split on load rather than size do better, and none of them can split below a single key.
Merging is riskier than splitting and often skipped. Merging two ranges requires them to be adjacent and co-located, requires a window where neither accepts writes, and can immediately re-split if the estimate was wrong. Many systems merge lazily or never, and slowly accumulate thousands of small ranges — each with its own metadata, replication group and heartbeat traffic.
Where range partitioning is simply correct
It is easy to read the hotspot section as an argument against ranges. It is not. Ranges are the right choice whenever ordered access dominates, and for a large class of systems it does.
Time-series and observability data, where every query is "this series, this window". Ledgers and event logs replayed in order. Cursor pagination, which is *defined* by ordered keys — under hashing, WHERE id > cursor ORDER BY id LIMIT 100 must fan out to every partition and merge, on every page. Multi-tenant systems keyed by (tenant, …), where ranges give you tenant locality for free: one tenant’s data is contiguous, so it can be scanned, exported, migrated or isolated as a unit. That last property is worth a great deal and hashing cannot offer it at all.
The honest summary: hashing is the safer default for opaque point-lookup workloads; ranges are the better fit for ordered ones, and they demand active operational attention in exchange.
Key points
- Ranges keep order, so scans touch only the partitions that intersect the range.
- Ranges give up uniformity: load follows the data distribution and the access pattern.
- A monotonic partition key — timestamp, auto-increment, ULID — concentrates 100% of writes on one partition, and splitting does not help.
- Splits and merges make the partition map a continuously changing thing, so stale routing is a permanent condition, not an incident.
- A node that no longer owns a key must say "not the owner", never "not found".
The chain, answered
Every field here is required, which is why no lesson in this domain can recommend a design without naming what an operator sees when it fails, what survives the partition, what repairs it afterwards, and the simpler thing to consider first.
- • Define a total order over the partition key and cut it into contiguous ranges at chosen boundaries.
- • Store the boundary table somewhere routers can read it — itself usually a replicated, consensus-backed range.
- • Route by binary search over the boundaries rather than by arithmetic.
- • Monitor each range’s size and load; split when it exceeds a threshold, choosing a midpoint by sampled key distribution.
- • Commit the split as a metadata change, then let routers discover it lazily via a
NotOwnerresponse and a boundary refresh. - • Optionally merge adjacent under-full ranges, which requires both to be quiescent briefly.
- • A router uses stale boundaries and sends a key to a node that has split it away.
- • A split commits but the data reorganisation does not complete, leaving a range that two nodes partially serve.
- • The chosen midpoint does not halve the load, so the hot half splits again immediately.
- • A range cannot be split because all its load is on one key.
- • Merge and split race: a range is merged just as its neighbour is being split, and the boundary table has two conflicting proposals.
- • Write hotspot on the newest range: one node at 95% CPU with a growing write queue while the rest of the cluster is under 10%. Aggregate cluster utilisation looks healthy on every dashboard, so the problem is invisible until that node starts timing out.
- • Split storm: a fast-growing range splits, both halves grow, both split again. The operator sees the range count climbing steeply, rebalancer streams never going idle, and metadata write rate spiking — the cluster is busy reorganising itself instead of serving.
- • Range that cannot split: one key exceeds the size or throughput limit alone. The operator sees a split repeatedly attempted and abandoned in the logs, with the range staying hot.
- • Range-count bloat after data deletion: tens of thousands of tiny ranges remain because merging was never enabled. The operator sees memory and heartbeat traffic on every node scaling with range count rather than with data volume.
- • Apparent data loss during a split window: a client with stale boundaries reads from the old owner and gets an empty result, and the application treats the key as deleted.
- • The boundary table is shared mutable state that every router depends on, so changes to it must be totally ordered — this is the one piece of a range-partitioned system that genuinely needs consensus.
- • A split must be atomic with respect to routing: there must never be a moment when both zero and two nodes claim a key. Systems achieve this by making the metadata commit the single ordering point.
- • Routers do not need to be *current*, only *correctable*. The
NotOwnerresponse is what makes eventual metadata propagation safe — a cheap, self-healing design worth copying. - • Merges need more coordination than splits, because two ranges must be quiesced together. That asymmetry is why splitting is automatic in most systems and merging is not.
- • Order is preserved regardless of failures: a partial fan-out returns a correct prefix of the range, not a scrambled sample.
- • A range whose replicas are all unavailable makes a *contiguous* slice of the keyspace unavailable — which is easier to explain to users but concentrates impact on whoever owns that slice, e.g. one tenant.
- • An in-progress split that stalls leaves the pre-split range as the source of truth, provided the metadata commit is the only cutover point.
- • Detect: alert on per-range write rate and on range count trend, not on node averages.
- • Contain: pause automatic splitting during an incident. A split storm during overload adds load precisely when the cluster cannot take it.
- • Recover: for a monotonic-key hotspot, the only real recovery is a key-design change — salting or reordering the composite key — which means a rewrite, so treat the tripwire as a deadline.
- • Reconcile: verify boundary continuity — no gaps, no overlaps — after any stalled split or merge. A gap is silent data unavailability.
- • Verify: run an ordered full scan and confirm it visits every range exactly once and in order.
- • Write and read rate per range, with the ranges sorted by key order — a hotspot on the right-hand edge is visually obvious and invisible in any aggregate.
- • Range count over time, and the size distribution of ranges. A long tail of tiny ranges means merging is not keeping up.
- • Split and merge events per hour, with their chosen midpoints. Repeated splits at nearly the same point mean the heat is on one key.
- •
NotOwnerresponses per second, by client. A steady low rate is normal and healthy; a spike means the boundary table changed and the fleet has not caught up. - • Boundary-table read rate — if routers are refreshing constantly, their cache TTL is wrong.
- • Time-series, logs, metrics, and any workload whose dominant query is "one series over one window".
- • Cursor pagination and ordered iteration over a large dataset.
- • Multi-tenant systems keyed by tenant first: one tenant’s data is contiguous, so it can be exported, migrated or isolated as a unit.
- • Datasets where you want small, movable units of data and are willing to run an active split/merge loop to get them.
- • Any monotonically increasing partition key, unless the total write rate comfortably fits one node.
- • Workloads with no ordered access at all, where you are paying boundary metadata and split machinery for nothing.
- • Highly bursty key distributions — a viral tenant makes one range enormous, and unlike with hashing, the imbalance is durable rather than statistical.
- • Teams without the operational appetite to watch range counts and split behaviour. Range partitioning is not fire-and-forget.
- • Hash partitioning (Hash Partitioning and the Modulo Trap) when point lookups dominate and you want uniformity for free.
- • A composite key — hash the high-order component, range the low-order one — which is the practical middle ground and what most real schemas end up with.
- • Explicit, manually chosen static ranges, when you know the distribution and it is stable. No split machinery, no surprises, and it goes wrong quietly when the distribution shifts.
- • Keep ranges for the primary data and add a hash-partitioned secondary index for the point-lookup access pattern, accepting the write amplification.
- • Time-bucketed tables rotated on a schedule, with old buckets moved to cheaper storage — often simpler than dynamic ranges for retention-bounded time-series.
Range partitioning: scans you keep, hotspots you inherit
What people believe, and what is true
Splitting a hot range fixes the hotspot.
It fixes a *size* problem. If the load is on the newest keys, the right-hand half inherits all of it; if the load is on one key, neither half helps.
Range partitioning is the same as an ordered index.
An index orders rows within one storage engine. Range partitioning orders them across machines, which additionally means the boundaries are distributed state that can be stale.
A UUID primary key is safe for range partitioning.
UUIDv4 is, and gives you no ordering benefit at all. UUIDv7 and ULID are time-ordered by design, so they reproduce the timestamp hotspot exactly.
More, smaller ranges are strictly better for balance.
Each range carries a replication group, heartbeats, metadata and a rebalance unit. Past a point the fixed per-range cost dominates and the cluster spends its capacity on bookkeeping.
Go deeper
Only the levels this lesson can honestly fill — a missing level is a claim nobody had.
Overview
Range partitioning keeps keys sorted, so range scans stay cheap. In exchange, load follows your data, and a timestamp key sends every write to one partition.
Practical
Never make a monotonic value the leading component of a range partition key. Put a high-cardinality, non-monotonic field first — device, tenant, user — and the time component second. Alert on per-range write rate ordered by key, not on node averages.
Advanced
Treat the boundary table as the single ordering point for ownership. A split is safe precisely because the metadata commit is atomic and every stale router is corrected by a NotOwner response. Systems that let nodes serve keys they might not own trade a routing race for a data-divergence bug.
Internals
Midpoint selection is a streaming quantile problem over a key distribution you can only sample. Size-based sampling is cheap and load-blind; load-based sampling requires per-key counters, which is why systems that offer load-based splitting maintain a sketch per range. Neither can split below one key, which is the boundary condition Hot Partitions: The Skew Hashing Cannot Fix takes up.
Apply it
- 🔧 Take your busiest table and decide whether its primary key is monotonic. If it is, write down what the write rate would be on the newest range at 10× today’s traffic.
- 🔧 Design a composite key that supports both "one tenant, recent first" and "all tenants, one hour" queries, and state honestly which of the two you have made expensive.
- ⚡ A logging cluster splits ranges every few minutes and the range count has gone from 400 to 40,000 in a week. Writes are timing out. Diagnose it.
- ⚡ After a range merge, a scan returns duplicate rows for a small key window. What invariant was violated and how would you detect it automatically?
- 💬 You are storing IoT telemetry and want fast per-device time-window queries. Design the partition key and justify the order of its components.
- 💬 A range-partitioned cluster has one node at 100% CPU. It splits the hot range and nothing improves. What are the two possible explanations?
- 💬 Why must a node that has split away a key return "not the owner" rather than an empty result?