Load Test Shapes: The Shape Is the Hypothesis
Ramp, spike, step and soak are not stylistic choices. Each shape exposes a different failure: ramps find the knee, spikes find scaling lag and cold starts, soaks find leaks and drift. Choosing a shape is choosing what you are willing to find out.
Frame the diagnosis
Performance work starts from a symptom and a signal — never from a resource dashboard.
Four shapes, four hypotheses
Every load shape is a hypothesis about how the system fails. A ramp says "I think there is a capacity limit and I want to find it". A spike says "I think sudden arrival is different from gradual arrival". A soak says "I think something accumulates". A step says "I think behavior changes discontinuously at some level, and I want to hold each level long enough to see steady state".
A steady-rate test at expected peak is the shape that tests the fewest hypotheses. It confirms the system can do the thing you already expected it to do, from a starting state you controlled, for a duration too short for anything to accumulate. It is the right first test and a poor only test.
The practical approach is a small suite: a ramp to locate the knee, a spike at your measured worst-case burst ratio, and a soak at a realistic sustained level. Three tests, three different classes of bug, and each one is cheap once the environment exists.
| Shape | Hypothesis | Exposes | Misses |
|---|---|---|---|
| Ramp (gradual rise to failure) | There is a capacity knee somewhere | Saturation point, first resource to saturate, degradation mode | Anything time-dependent or burst-dependent |
| Spike (instant multiple, brief) | Sudden arrival differs from gradual arrival | Autoscaling lag, cold caches, connection storms, thundering herds | Steady-state capacity; slow accumulation |
| Step (hold each level) | Behavior changes discontinuously with load | Level-dependent effects: GC regime changes, pool exhaustion thresholds | Transient burst behavior |
| Soak (hours at a realistic level) | Something accumulates over time | Memory leaks, fragmentation, disk fill, connection churn, replication lag drift | Peak capacity and burst response |
The ramp finds the knee
A ramp raises offered load in steps and holds each step long enough to reach steady state — typically a few minutes, long enough for queues to stabilize and autoscaling to settle if it is enabled. The output is the throughput-latency curve, and the knee in that curve is your capacity number.
Hold time matters more than people expect. Ramping too fast means each level is measured during a transient, and the curve smears: latency at each step reflects the queue left over from the previous step rather than steady-state behavior at that rate. If your ramp shows latency climbing steadily from the very first step, suspect the ramp rate before you suspect the system.
Read three things off the result: where throughput stops tracking offered load, which resource hit its limit at that point, and what happened to error rate past it. That is a capacity number, a bottleneck identification and a failure-mode assessment from one test.
offered throughput p50 p99 app CPU pool waiters note 500 500 42 ms 88 ms 18 % 0 1,000 1,000 45 ms 96 ms 34 % 0 1,500 1,498 51 ms 130 ms 51 % 0 2,000 1,980 68 ms 210 ms 67 % 3 first waiters appear 2,200 2,040 124 ms 740 ms 71 % 48 knee: throughput flattens 2,400 2,050 310 ms 1,900 ms 72 % 190 past knee: queueing only 2,800 2,045 980 ms 8,400 ms 72 % 520 timeouts begin KNEE ~2,050 req/s FIRST CONSTRAINT database connection pool (CPU never passed 72 %) PAST-KNEE BEHAVIOR accepts everything, queues, times out at 30 s — no shedding USABLE CAPACITY ~1,800 req/s to stay inside a 300 ms p99 objective
Spike and soak find what a ramp cannot
A spike test jumps straight to a multiple of baseline and holds it briefly. It reproduces the class of failure a ramp is specifically designed to avoid: everything that depends on *rate of change* rather than level. Autoscaling lag (Autoscaling Lag: The Gap Where the Outage Lives), cold caches on newly arrived instances, connection pools filling from empty, and thundering herds all appear here and nowhere else. Set the multiple from your own traffic history — the fastest observed rise — rather than a round number.
A soak test holds a realistic sustained level for hours. It is the only shape that finds accumulation, and accumulation is a large family: heap growth (Memory Leaks: Growth That Does Not Come Back), allocator fragmentation, log or temp-file disk fill, connection churn exhausting file descriptors, replication lag creeping up, and cache working sets slowly exceeding memory. None of these are visible in a ten-minute run, and all of them are visible by hour six.
The comparison below is the whole argument for soak testing: identical offered load, identical throughput, and a system that is quietly on its way to an incident. Every one of those signals is flat during the first ten minutes.
| Signal | Value | What it tells you | Verdict |
|---|---|---|---|
| Throughput | 1,500 → 1,500 req/s | Unchanged — the test looks like it is passing | normal |
| p99 latency | 140 ms → 380 ms | Degrading under constant load: something is accumulating | smoking gun |
| Heap after GC | 420 MB → 1.9 GB | Live set growing under stable workload — leak signature | smoking gun |
| GC pause time share | 0.4% → 6.1% | Consequence of the growing live set, not a separate problem | suspect |
| Open file descriptors | 1,100 → 7,800 | Connection churn without reuse; heading for a hard limit | smoking gun |
| Replication lag | 40 ms → 2.4 s | Drifting under sustained write load — invisible in a short test | suspect |
Key points
- Each load shape encodes a hypothesis: ramps test for a capacity limit, spikes for rate-of-change effects, soaks for accumulation.
- A steady-rate test at expected peak tests the fewest hypotheses of any shape and is the most commonly run.
- Hold each ramp step long enough to reach steady state, or the curve measures transients rather than capacity.
- Spike multiples should come from your own fastest observed traffic rise, not a round number.
- Soak tests are the only way to find leaks, fragmentation, descriptor exhaustion and lag drift — all invisible in ten minutes.
Load-Test Shapes
Change an input and watch which number moves — and which one does not.
Increase load in steps and hold each step long enough to reach steady state. 100 → 500 → 1k → 5k → 10k req/s.
Where the knee is: the load at which latency stops being flat and starts climbing. This is the number capacity planning actually needs.
Anything that only appears over hours (leaks, disk fill, cache drift) and anything caused by a sudden change rather than a level.
Follow the diagnosis
The causal chain, hop by hop — and the readings that invite the wrong conclusion.
- 1Shape → coverage: a steady ten-minute test at peak exercises neither rate-of-change nor accumulation failure modes.
- 2Ramp → knee: throughput flattens at 2,050 req/s while CPU sits at 72%, so CPU is not the constraint.
- 3Knee → constraint: pool waiters climb from 3 to 190 across the knee, naming the database connection pool.
- 4Spike → lag: an instant 2.4× jump outruns scale-out, and the standing fleet absorbs the whole burst (Autoscaling Lag: The Gap Where the Outage Lives).
- 5Soak → accumulation: at constant load, heap after GC grows from 420 MB to 1.9 GB over six hours while throughput stays flat.
- • "The ramp showed latency climbing from the first step, so the system is weak" — more likely the ramp rate outran steady state.
- • "The spike test passed because throughput held" — check recovery time and error rate during the burst, not just the plateau.
- • "The soak passed, throughput was constant" — throughput is the last thing to move; read the trend lines instead.
- • "We tested at 3× peak so we are covered for bursts" — a ramp to 3× and an instant jump to 3× exercise different code paths entirely.
Measure, fix, validate
An optimization is not finished until the metric that motivated it has moved.
- • Throughput, p50/p99 and per-tier resource utilization at each ramp step after it reaches steady state.
- • Time to recovery after a spike: how long until p99 returns to baseline once load drops back.
- • Trend lines during a soak: heap after GC, file descriptors, disk usage, replication lag, p99 — all plotted against elapsed time.
- • Your own worst-case burst ratio from traffic history, to parameterize the spike test honestly.
- • Run a small suite rather than one test: a ramp for the knee, a spike at your measured burst ratio, a soak for accumulation.
- • Set ramp hold times long enough for queues and autoscaling to settle, and verify by checking that latency is flat within each step.
- • Parameterize the spike from traffic history — fastest observed rise — and include the recovery period in the measurement window.
- • Run soaks long enough for the slowest accumulation you care about; overnight is a reasonable default for leak detection.
- • Plot trend lines against elapsed time for soaks, not just aggregate percentiles, since averages over the whole run hide monotone drift.
- • Confirm each ramp step reached steady state by checking that latency within the step is flat, not still rising when the step ends.
- • After a spike, verify p99 returns to baseline within the expected window and no work was silently dropped.
- • For soaks, confirm the trend lines are flat after a fix, not merely that the final aggregate looks acceptable.
- • A three-shape suite costs several hours of environment time per release cycle, which is real scheduling friction.
- • Soak tests need long-lived environments and someone to look at trend lines afterwards, or they become a job that always passes.
- • Spike tests deliberately push a system past capacity, so they need an environment where degradation is safe.
- • Keep the three-shape suite in the release process for services where burst or long-run behavior matters, and track the knee over time.
- • Alert on soak-test trend slopes (heap growth per hour, descriptor growth per hour) rather than on a pass/fail at the end.
- • Re-derive the spike multiple periodically from recent traffic, since burst shape changes as the product changes.
Accuracy
Performance numbers are conditional. These are the conditions.
- ILLUSTRATIVEThe ramp table and the soak comparison are teaching examples showing what each shape reveals. The step values, knee position and drift rates are invented to make the shapes legible.
- WORKLOAD-SPECIFICAppropriate hold times, spike multiples and soak durations depend on your queue depths, autoscaling reaction time and the timescale of whatever accumulates in your system.