Documentation Decay
Documentation is trusted, so wrong documentation is acted on with confidence. That makes a stale document worse than no document — and nothing fails when one goes stale.
The requirement, the obvious build, and why it breaks
Every lesson starts where the work starts: someone asked for something, and the first implementation that comes to mind survives until the requirement changes.
Why is a wrong document more expensive than a missing one, and what actually keeps a document true?
An on-call engineer follows the runbook at 3am. Step four says restart the worker pool. The worker pool was replaced eight months ago and the command now restarts the queue consumers, which makes the incident considerably worse.
More documentation is better. Write everything down — someone will find it useful, and worst case it is out of date and they check the code.
"Worst case they check the code" is exactly what does not happen. A document's function is to save the reader from reading the code; a wrong one performs that function perfectly and returns the wrong answer.
- "Worst case they check the code" is exactly what does not happen. A document's function is to save the reader from reading the code; a wrong one performs that function perfectly and returns the wrong answer.
- Absence produces correct behaviour. An engineer who finds no runbook goes and reads the code, asks someone, and proceeds carefully. An engineer who finds a confident, plausible, wrong runbook proceeds quickly.
- As the system changes, the document does not, and nothing anywhere reports the divergence. The failure is discovered at the moment of highest cost, by the person least able to verify it, because that is the only time anyone reads it.
- Volume makes it worse rather than better: the more pages exist, the less any one of them is reviewed, and the higher the chance that the page someone happens to trust is one of the rotten ones.
What limits the solution, and what must never stop being true
This domain leads with these two. A design that ignores its constraints is not a design, and an invariant nobody named is one nothing is protecting.
- Nothing detects a document going wrong. There is no compiler, no test and no failing build, so decay is silent by construction.
- The people best placed to notice — the ones changing the code — are the ones least likely to read the document, because they already know the answer.
- Documents accumulate readers you cannot enumerate, so you cannot warn them when one becomes untrustworthy.
- Anything a reader is told to act on must be true or must not exist. There is no acceptable third state, and "probably still right" is that third state wearing a disguise.
- A document that describes the present must have an owner and a mechanism by which it can be found wrong.
Who owns what, and where the seams fall
Responsibilities decide boundaries; boundaries decide what an interface has to say.
- The change that makes a document wrong owns fixing it. If that change cannot see the document, the document is in the wrong place, which is the whole argument for proximity (Docs Close to Code).
- Every document describing current behaviour has a named owner. "The team" is not an owner; a person or a CODEOWNERS entry is (Code Ownership).
- Whoever discovers a document is wrong owns either fixing it or deleting it, in that moment. Leaving it for later is how it stays wrong for another eight months.
- The defensible line is between documents that describe the present and documents that describe the past. The first kind rots; the second cannot, which is why an ADR is safe and a runbook is not (Architecture Decision Records).
- The other line is between documents inside the change's blast radius and documents outside it. A document in the same pull request as the code has a chance; one in a separate system does not (Docs Close to Code).
- Generated documentation is on the far side of both lines: it cannot diverge, because it has no independent existence (OpenAPI: Describing the Contract, Not Designing It).
The lifecycle of a documented fact
Decay is not an event, it is a state machine, and the useful thing about drawing it is that the transitions people assume exist mostly do not. Nothing moves a document from wrong back to correct on its own, and nothing moves it from unverified to trusted except an actual verification.
The forbidden transitions below are the design content. Each one is an edge your process must not contain, and each corresponds to a specific practice that removes it.
| From | On | To | Guard | Effect |
|---|---|---|---|---|
| Absent | someone writes it from the code | Fresh | dated and owned | a new maintenance obligation exists |
| Fresh | the code or infrastructure it describes changes | Unverified | the change did not touch the document | silent — nothing reports this |
| Unverified | a drill, a regeneration or a real verification against the system | Fresh | — | — |
| Unverified | enough divergence that the steps no longer work | Wrong | — | — |
| Wrong | someone is burned and fixes it | Fresh | — | usually an incident |
| Unverified | expiry review, or an owner deciding not to maintain it | Retired | — | — |
| Wrong | deletion — the cheapest correct action available | Retired | — | — |
- Fresh → Wrong — A direct edge means a code change can invalidate a document without anything noticing. This is the disease. Remove the edge by putting the document where the change must touch it, or by generating it — then the only path out of Fresh runs through a review that can see both.
- Unverified → Fresh — Not forbidden as an edge, but forbidden without a verification event. If a page can be promoted to trusted because someone glanced at it and it looked plausible, the state is decorative and a quarterly review will manufacture confidence without accuracy.
- Absent → Wrong — This edge fires when a document is written from memory rather than from the system. It is the reason "more documentation is better" is false: a page can be born wrong, and on day one it has maximum credibility.
The only outbound edges from Wrong are an incident and a deletion. A process that contains no scheduled verification has, in effect, chosen incidents as its repair mechanism.
What proximity actually buys, priced
The argument for co-location is usually made on principle. It is more convincing priced: take one small policy change and count what each design requires a human to notice.
The retry policy for outbound provider calls changes from three fixed attempts to five with exponential backoff and jitter, and support needs to be able to explain the new behaviour to customers.
Two of the three call sites have their own copy of the numbers, and the wiki page is invisible to review. Nothing in the pull request can tell a reviewer the page is now wrong. It stays wrong until support quotes it to a customer, or on-call follows it during an incident.
The numbers have one home, the callers reference it, and the only external page is a pointer that cannot go stale because it makes no claim of its own.
The four things that actually work
Everything that reliably keeps documentation true does one of four things, and none of them is trying harder. They are ordered by how much they buy relative to what they cost.
- 1Generate
Derive the document from the code, so it has no independent existence and cannot diverge.
fails by Covering only structure. A generated field list is always accurate and says nothing about intent, which is the part readers need most.
- 2Co-locate
Put the document where the change that invalidates it must pass, so review sees both at once.
fails by Audiences who cannot read the repository. Support and product will not find it, so something still has to exist outside (Docs Close to Code).
- 3Date and own
Let a reader discount an old page and give a named person the obligation to maintain it.
fails by Ownership moving on. An owner who left the company is worse than none, because the name creates false assurance (Code Ownership).
- 4Verify or delete
Exercise it — a drill, an onboarding run, a regeneration — or remove it. This is the only defence for documents that cannot be generated or co-located.
fails by Becoming ceremony. A review that marks pages read without executing them raises confidence without raising accuracy, which is the worst outcome available.
Notice that deletion appears as a first-class defence rather than a failure. A page removed is a page that can no longer mislead anyone, and it is the only intervention that costs nothing to maintain afterwards.
How to build it
Most important first.
- Generate anything that can be generated. A schema, a type, a route table and a CLI help text already contain the reference material, and a generated document has no failure mode of this kind (Schema-First vs Code-First).
- Move the document into the diff. If the change that invalidates it also touches it, review catches the divergence for free — that is the single highest-leverage move available (Docs Close to Code).
- Date and own everything that describes the present, at the top, visibly. A reader can then discount an undated page from three years ago, which is a much better outcome than trusting it.
- Prefer executable documentation: a runbook that is a script, an example that is a test, an invariant that is an assertion. Executable documentation fails loudly, which converts silent decay into a build failure (Testing as Design Feedback).
- Delete aggressively. Deletion is the cheapest correct action available and it is almost never taken, because deleting feels like losing information while leaving feels free — the accounting is backwards.
- Verify what you cannot generate: a drill for a runbook, an onboarding run for a README, a periodic review with a real date for domain rules (Restore Drills).
What the next change costs
The field this whole domain exists for. A structure is only better if it makes the change after this one cheaper — and it is worth saying which changes it does not help.
- Under documentation that lives away from the code, the next change costs nothing extra today and creates a liability with no due date. That is precisely why it keeps happening: the cost is real, deferred and charged to somebody else.
- Under co-located and generated documentation, the next change costs a few extra lines in the same pull request, every time — a small, visible, recurring tax in exchange for removing the silent failure.
- The cost of repair grows with time in a specific way: the longer a document has been wrong, the more decisions have been made on top of it, so fixing the page no longer fixes what the page caused.
- Aggressive deletion loses genuine context. Some of what you delete was the only record of a reason, which is the argument for writing ADRs rather than for keeping stale pages (Architecture Decision Records).
- Generated documentation is accurate and often unhelpful: it describes shape perfectly and intent not at all, so it cannot replace the narrative layer that decays.
- Requiring documentation changes in the same pull request slows every change slightly and produces perfunctory edits under deadline pressure. The tax is real; it is simply cheaper than the incident.
What can go wrong
- The runbook is followed and makes the incident worse — the canonical case, and the reason this lesson is the sharpest point in the module.
- A stale architecture diagram teaches a new hire a model that has not been true for two years, and they design against it for months before anyone notices.
- A wrong glossary entry propagates into code, tickets and a schema, at which point the wrong meaning becomes the real one and correcting it is a migration.
- The mitigation itself fails: a quarterly documentation review becomes a box-ticking exercise where every page is marked reviewed and none is read, which produces confidently-dated wrong documents — strictly worse than undated ones.
- Every document describing the present depends on the system it describes, with no link back. That one-way, undetectable dependency is the entire disease.
- Documents that live outside the repository additionally depend on a separate tool, permission model and search index, none of which are in the change's path (Docs Close to Code).
- Generated documentation inverts the dependency: the document is produced by the code, so the code cannot change without it changing.
- "So do not write documentation." The conclusion is to write less and keep it true, not to write none. Absence is better than wrong, and correct is better than both.
- "Just review the docs quarterly." A calendar review with no execution produces pages marked as reviewed that nobody read, which raises confidence without raising accuracy — the worst possible movement.
- "The code is the documentation." The code is the truth about what happens and says nothing about why, what was rejected, or what to do at 3am. Those are the documents that decay, and they are the ones that matter most (Comments).
- "This is a discipline problem." It is a design problem. Teams do not fail to update documents because they are careless; they fail because the change and the document are in different places, and the fix is structural (Docs Close to Code).
Testing it, and how it ages
- Compile or run examples in CI. A doc example that no longer compiles is the rare piece of documentation with an automatic decay detector (Contract Tests).
- Execute runbooks in drills, with someone who did not write them, and treat every point of confusion as a defect in the document rather than in the person (Restore Drills).
- Assert generated documentation matches the code by regenerating it in the build and failing on a diff.
- Nothing tests a prose explanation of why. Accept that, keep those short, and date them so a reader can judge.
- Every document trends toward wrong. That is the base case, and any process that does not include deletion or verification is only choosing how long it takes.
- Systems that survive tend to converge on generated reference plus a small number of hand-written narrative pages, because that is the only combination anyone maintains for a decade.
- The wiki page nobody has opened in two years is not neutral. It is a trap that a search engine will eventually deliver to someone in a hurry (Docs Close to Code).
Where this applies
This domain's advice is contested more than most. These labels say what each claim is specific to — and where CONTESTED appears, the note gives the strongest form of the opposing view rather than a caricature.
- GENERALThe mechanism — a document is trusted, so a wrong one is acted on rather than checked — is a property of how people use documentation, so it holds across languages, organisations and document formats.
- SCALE-SPECIFICOn a team of three, decay is partly self-correcting: the person who changed the system is also the person reading the document, and they notice. Past roughly a dozen engineers, or the moment another team reads your documents, the reader and the changer are different people and nothing connects them — which is where the structural fixes stop being optional.
- CONTESTEDThe strongest opposing view: "wrong docs are worse than none" is used to justify writing nothing, and in practice teams with sparse documentation lose more time to repeated archaeology than teams with somewhat stale documentation lose to being misled — because experienced engineers do treat documents as hints and verify anything consequential. That is a fair description of ordinary development work. It is not a fair description of an incident at 3am, where verification is exactly what the reader has no time for, and that asymmetry is why the sharp form of the claim survives.
Where the depth lives
This domain teaches the codebase-level structure and hands the rest off.
- — Testing & Reliability Engineering — a runbook that has never been executed is an untested code path with a human interpreter, and the practice that fixes it is a drill rather than a review.