The nightly rebuild no longer finishes
A nightly job rebuilds a fact table from all history. It has been getting slower for a year and now regularly runs past the morning deadline. The team has already increased the warehouse size once, which bought a few months.
What you would do first
Answer before revealing anything. The value of the exercise is entirely in committing to a diagnosis you can be wrong about.
- 1Plot run duration against total row count over the past year. A straight line through the origin means the job scales with history, not with new data.
- 2Establish what actually changes per run: new rows, plus corrections to a bounded recent window, plus rare historical restatements.
- 3Check whether the transformation is deterministic and re-runnable for a bounded range, which decides whether incremental processing is available at all.
- 4Find out what depends on the full rebuild — often it is one late-arriving correction path that nobody has separated out.
What is actually going on
The trap
The fix that looks right. Read it even if you got the answer — especially then.
Increase the warehouse size again. It restores the deadline for another few months, the cost per night rises immediately, and the growth curve is unchanged — so the same conversation happens again with a larger number and less remaining headroom.