Thought Experiments
Run the system in your head at one user, a hundred, a million — not to build for a million, but to see which assumptions hold at each size and where the design would first bend. A thought experiment is the cheapest experiment there is; it finds the assumption, and a real experiment then checks it.
The situation, the reflex, and why it stalls
Every lesson starts where being stuck starts: someone has a problem, and the first move that comes to mind feels like progress.
How do you understand what a design assumes without building it — and without sliding into building for a scale you do not have?
The store works for you, in test mode, alone. The founder asks "will it hold up when we launch?" and you honestly do not know. Two voices answer at once: one says the code is fine and you will find out, the other says you should add a cache and a queue now to be safe. Neither has looked at anything.
Add capacity now. A cache for the product list, a queue for order processing, maybe a read replica — the things every scaling article names. It feels responsible: better to have it and not need it, and it produces visible architecture.
Every component is added without knowing what it protects against. The cache is in front of a product list that a single database would serve for years; the queue makes checkout asynchronous, which changes what the customer sees, in exchange for solving a problem that was never measured.
- Every component is added without knowing what it protects against. The cache is in front of a product list that a single database would serve for years; the queue makes checkout asynchronous, which changes what the customer sees, in exchange for solving a problem that was never measured.
- The assumptions that would actually bend under load — a total recomputed on every cart view, a stock check that locks a row for the length of a payment call — are untouched, because they are in the code, not in the architecture diagram.
- The system becomes harder to reason about before anyone has reasoned about it. Each added component has its own failure modes, and now "what if the cache is stale?" is a question with no answer because nobody asked why the cache was there.
- When the founder asks again, the answer is "we added a cache", which is not an answer to "will it hold".
The move
Precisely enough to apply it to a problem you have never seen — not a slogan.
- Instead of building for a scale, *imagine* the system at several sizes and ask, at each one, which assumptions are still true. One user: everything is sequential, nothing collides, every table fits in memory. A hundred: two customers can act at once, the admin's price change lands mid-checkout. A million: nothing fits in memory, every per-request computation is multiplied, every external call is a queue in disguise.
- For each size, walk the core workflow and name the first thing that stops being true. Not "the database will be slow" — that is a feeling — but "the cart total is recomputed from live prices on every page view, so at N views the price table is read N times per second". The output of a thought experiment is an *assumption with a size attached*.
- Then decide what to do with each assumption: nothing, because the size where it bends is far beyond the constraint you were given (Time, Users, Data); measure it, because you are not sure of the size; or change it now, because it is cheap to change now and expensive later — a captured price instead of a live one is cheap today.
- Keep the real experiment separate from the imagined one. The thought experiment finds the assumption; a load test or a query plan checks the number. Confusing the two produces either premature architecture (imagined numbers treated as measured) or paralysis (nothing believed until measured).
The store, imagined at three sizes
The board below is the thought experiment's output for the store. The known column is what the walk established; the unknowns are the assumptions that bent, each rewritten as a question with the observation that would answer it. None of the unknowns is "will it scale?" — that was the vague form, and the walk sharpened it.
- ✓At one user nothing collides; every assumption in the draft holds.
- ✓At a hundred, the two-step stock decrement and the live-price total are correctness problems, fixed now: a conditional update and a captured price.
- ✓The constraint the founder gave is a launch in the hundreds, not the millions; capacity assumptions that bend past that are noted, not built.
- ~Product prices change rarely — a few edits a day by one admin. If prices become dynamic, the captured-price decision is revisited.
- ~One database serves reads and writes for the launch. Written down so that a replica, if it arrives, arrives for a reason.
? Will the product page be slow?
becomes How many rows does one product-page view read, and at what request rate does that read exceed what one database instance serves?
experiment Run the page's queries with a plan explanation, count rows touched, then load-test the single endpoint until latency turns — one afternoon, one graph.
? Will checkout be slow?
becomes How much of checkout latency is the payment provider and the mail sender, and what is their slow tail?
experiment Time each external call in checkout for a day of test traffic and read the tail; only if the tail is unacceptable does a queue for email become a candidate.
? Will the database fall over?
becomes Which single query in the core workflow reads the most rows per request, and does it have an index?
experiment List the queries of the core workflow, explain each, sort by rows read; the top one is the first thing to index, and probably the only one for now.
Three vague worries became three afternoons of measurement, and none of them became a component. That is the thought experiment working.
The cache that the experiment talked you out of
The reflex reached for a cache. The why ladder is what the thought experiment does to that reach: it asks what the cache is for, and finds — at this store, at this size — that the answer is an assumption nobody has checked. The ladder also says when the cache would have been right, because the device is not "caches are bad".
“We need a cache in front of the product catalog before launch.”
- ↓Why a cache? Because the product list will be read a lot and the database will be slow.
- ↓Why will the database be slow? Because at a million users it would be reading the products table constantly.
- ↓Why a million? It is the size I imagined in the thought experiment. The launch constraint is hundreds.
- ↓Why would hundreds be slow? It would not, if the products query is indexed and reads a few rows. I have not checked that it is.
the claim was right when The load test shows the endpoint turning below the constraint, or the catalog is served from a slow upstream (a third-party product feed) where every read is an external call — then a cache is the measured answer, and it comes with its own ledger entries about staleness.
Thought experiment, then real experiment
The two kinds of experiment have a fixed order and a fixed division of labour. The thought experiment is fast and produces assumptions; the real one is slow and produces numbers. Skipping the first wastes the second on the wrong question; skipping the second builds on imagination.
- 1Pick three sizes
Now, the constraint, one order of magnitude beyond it
fails by Only the absurd size is imagined, and the design is built for it
- 2Walk the workflow at each
Name the first assumption that bends, in one sentence with a size
fails by "The database will be slow" — a feeling, not an assumption
- 3Sort: correctness or capacity
Races and stale reads are fixed now; throughput is measured first
fails by Everything is treated as capacity and the race ships
- 4Attach an observation
Query plan, load test, external-call timing — the cheapest thing that gives a number
fails by The imagined number is written down as if measured
- 5Decide, and record in the ledger
Nothing / measure / change now — with the reason
fails by The note is filed and the founder's next question restarts from zero
The growth lab runs the second half of this pipeline for you: it shows what the measured bottleneck is at each traffic level and which added component would actually move it.
How to do it
Most important first.
- Pick three sizes that bracket the constraint you were given: the size you have now, the size the founder hopes for, and one order of magnitude past that. The third is not a target; it is there to make the assumptions visible.
- Walk the core workflow at each size and write the first assumption that bends. One sentence each. If you cannot find one at the largest size, you have not walked carefully enough — every design bends somewhere.
- Separate assumptions about *correctness* (two customers, last unit) from assumptions about *capacity* (the price table read per view). Correctness assumptions bend at a hundred users and are fixed now; capacity assumptions bend at a million and are measured first (Add Complexity Only When Required).
- For each bending assumption, name the cheapest observation that would tell you the real size: a query plan, a load test of one endpoint, a count of external calls per checkout (Unknown, Question, Experiment).
- Write the result in the notebook as "assumes X; bends around Y; checked by Z" and stop. The thought experiment's deliverable is that sentence, not a component (The Complexity Ledger).
- Revisit when the constraint changes. The founder's "a thousand customers" becoming "a national retailer" is the moment to run the experiment again, not to reach for last year's notes.
Worked on a concrete problem
The move has to produce something. This is what it produced.
- The store at one user: the cart total is computed from live prices on each view, checkout reads stock and decrements it in two steps, order confirmation emails are sent inside the request. All true, all fine, nothing collides.
- At a hundred concurrent users: two customers can now hit the last unit together, so the two-step stock decrement is a correctness bug, not a capacity one — fixed now, because the fix is a conditional update and costs an afternoon. The admin can change a price during someone's checkout — the live-price total becomes a decision about snapshots, also cheap now.
- At a million: the live price lookup is a million reads per page view per second in the imagined worst case — a capacity assumption, written as "assumes product prices are read per view; bends when views exceed what one database serves; check with a query plan and a load test of the product page". Nothing is built; a note exists. The email-in-request assumption becomes "checkout latency includes the mail provider" — also noted, with "measure the provider's p99 before deciding on a queue".
- The founder's question now has an answer: "it will hold for the launch you described; the two things that would bend first are noted with how we would know, and one of them is a correctness fix we are doing now regardless." No cache, no queue, and the architecture is smaller and better understood than before the conversation.
How you know it worked
What now exists that did not before, and what question you can now ask.
- You can name, for the current design, the first assumption that bends at each of three sizes — and the sizes are numbers with a reason, not feelings.
- Correctness assumptions have been separated from capacity ones, and the correctness ones are being fixed now.
- Every capacity assumption has an observation next to it that would check the real number.
- The answer to "will it scale?" is a list of what would bend and how you would know, rather than a list of components.
The questions you can now ask
The field this whole domain exists for. After this lesson, these are the questions to put to an unfamiliar problem.
- ?At one user, a hundred and a million, what is the first assumption in this design that stops being true?
- ?Is that assumption about correctness or about capacity — and does that decide whether I fix it now or measure it first?
- ?What is the cheapest observation that would tell me the real size at which it bends?
- ?Which size did the constraint actually name, and which of these assumptions bend before it?
What can go wrong
- The thought experiment becomes design for the largest size. Imagining a million users, and then building the sharded database, is the reflex wearing a lab coat. The largest size exists to expose assumptions, and the constraint decides which ones matter.
- Imagined numbers are treated as measured ones. "The price table will be read a million times a second" is a scenario, and the cache built on it is built on nothing. The thought experiment produces a question for a measurement, not a measurement.
- Only capacity is imagined. The most valuable finding at a hundred users is usually a race, and a thought experiment that only asks "will it be fast?" walks past it.
- The experiment is never re-run. The assumptions were correct for the launch; the retailer deal arrives and the notes from the launch are treated as still true.
- A thought experiment is free and therefore easy to trust more than it deserves; its conclusions are only as good as the walk through the workflow, and a missed step is a missed assumption.
- Naming the assumptions that would bend invites the question "so why not fix them now?", and the honest answer — because we have no evidence they will — takes discipline to hold under a nervous founder.
- Separating correctness from capacity means doing the correctness work immediately, which is real time spent on cases no user has hit yet.
- "So never think about scale until it happens." Think about it now, in your head, and write down what bends; build for it when the measurement says so. The slogan "don't design for a million users" is precise only as "don't *build* for a size the constraint did not name and no measurement supports" — the thinking is always cheap.
- "The thought experiment tells me the bottleneck." It tells you the candidate. The growth lab exists to show that the imagined bottleneck and the measured one often differ, and that a cache added for the wrong one changes nothing (Scale Thought Experiments).
- "A million is an absurd size for this store." It is — which is why it is the right size for the experiment. An absurd size makes assumptions visible that a plausible size hides; the constraint then decides which ones to act on.
Where this applies
Problem-solving advice is stated as universal far more often than it is. These labels say what each method is specific to — and where CONTESTED appears, the note gives the strongest form of the opposing view.
- GENERALImagining a system at several sizes to find the assumption that bends applies to anything with a size axis — users, rows, files, messages, team members; the sizes change, the question does not.
- ILLUSTRATIVEOne user, a hundred and a million, the afternoon for the conditional update and the store's workflow are invented to show the move; the numbers are for the shape of the argument, not capacity estimates.
- STAGE-SPECIFICBefore launch the experiment replaces measurement because there is nothing to measure; on a running system it is the first step *before* measurement — it says where to point the profiler — and a team with real traffic should trust the graph over the imagined walk whenever they disagree.
Where the depth lives
This domain asks the question and hands the answer off by name.