Reverse Engineering a System
Where does the program start? What receives input? Where is data stored? Which modules change state? What external systems exist? Five questions recover the design of a system whose design was never written down — from the running thing, not the folder names.
The situation, the reflex, and why it stalls
Every lesson starts where being stuck starts: someone has a problem, and the first move that comes to mind feels like progress.
You have inherited a system with no documentation and no author to ask. How do you recover what it is and how it works, without reading all of it?
The store's original team is gone. There is a repository, a running deployment, a database nobody has a diagram of, and a request from the business to "add a second warehouse". Nobody can say what that would touch.
Draw the architecture from the folder structure. The folders have names — services, models, controllers — and a diagram of them looks like an understanding of the system.
The folder names describe the framework's conventions, not this system's responsibilities. Every application built with the framework has the same three folders; the diagram is of the framework.
- The folder names describe the framework's conventions, not this system's responsibilities. Every application built with the framework has the same three folders; the diagram is of the framework.
- The diagram omits everything the folders do not show: which tables exist, which of them are written by which module, which external systems are called, and what actually runs in production versus what is left over.
- "The code is the documentation" is the slogan under this reflex; made precise it says the code is the *source* of the documentation, and the documentation still has to be produced by asking questions of it — the code answers, it does not volunteer.
- The warehouse request is estimated from the diagram, which does not know that inventory is decremented in three places, and the estimate is wrong by a multiple.
The move
Precisely enough to apply it to a problem you have never seen — not a slogan.
- Recover the system from observations rather than from names. The questions are the ones you would ask when designing it — Systems Thinking run backwards — and each has a concrete way to answer it from the running thing: where does it start (the process entry and the route table); what receives input (every handler, job, webhook and queue consumer); where is data stored (the schema, the object store, the caches); which modules change state (the writers to each table); what external systems exist (every outbound call and credential).
- Answer them in an order that lets each check the last. The entry and the inputs give you the surface; the storage gives you the entities; the writers tell you which parts of the surface can change which entities; the external systems tell you where the boundary is. A module that appears in none of the answers is either dead or a library.
- Build the picture as data — a table of inputs, a table of stores with their writers, a list of external systems — before drawing a diagram. The diagram is then a rendering of facts, and every box on it can be pointed at.
- Test the recovered design against a change. "Add a second warehouse": which stores hold inventory, which writers decrement it, which inputs trigger them? If the recovered design answers that, it is good enough for the request; if not, the gap says where to look next.
The five questions as a tree
Recovering a design is a decomposition of the question "what is this system?" into questions that have observable answers. The leaves are testable in the sense that matters here: each names the observation that would confirm the answer is true of the running system, not merely of the repository.
- ├Where does it start?
- └HTTP server and route tabletestable Every route in the table answers a request in the running deployment; routes that 404 are dead.
- └Scheduled jobs and consumerstestable The scheduler's configuration lists them, and their log lines appear at the scheduled times.
- ├What receives input?
- └Customer and admin routestestable Each has a request shape; sending it produces a response, and the handler is reachable from the route.
- └Provider webhooktestable A test-mode event from the provider reaches the handler and is recorded.
- ├Where is data stored?
- └Live schematestable The tables listed by the running database, with row counts, match the entities the handlers read and write.
- └Object store and cachestestable The credentials in config point at buckets and caches that contain what the code expects.
- ├What changes state?
- └Writers per tabletestable For each table, the search for inserts and updates yields call sites, and exercising each call site changes the row it claims to.
- ├What is outside?
- └Outbound clients and credentialstestable Each credential in the config corresponds to an outbound call in the code, and disabling it makes a specific feature fail.
A module that appears under none of the five is either a library or dead; either way it is not part of the design.
The state-change map
Of the five answers, the writers table is the one that most often surprises and most often decides an estimate. It says, for each store, which inputs can change it — and therefore which inputs a change to that store must consider.
| Store | Written by | Triggered from | Surprise |
|---|---|---|---|
| products | admin product service | admin routes | none |
| carts | cart service | customer cart routes | items stored as JSON, shape changed by last week's refactor |
| orders, order_items | checkout service | POST /checkout/pay | the nightly job also updates status to cancelled |
| payments | payment gateway; webhook handler | pay request; provider webhook | two writers for one row — the webhook can arrive before the request returns |
| inventory | admin stock endpoint; checkout service; nightly reconciliation | admin routes; pay request; schedule | three writers, one quantity per product, no location — the second-warehouse change touches all three |
The recovered system, drawn from the tables
The diagram comes last and adds nothing that is not in the tables; its value is that it can be looked at. Every box is an answer to one of the five questions and can be pointed at in the running system.
How to do it
Most important first.
- Start: the process bootstrap, the route table, the scheduled jobs, the queue consumers — everything that can begin execution. List them.
- Inputs: for each starting point, what arrives — an HTTP body, a webhook payload, a message, a file. This is the system's surface (Which Components Must Communicate?).
- Storage: the schema from the live database, not from the migrations folder; the buckets; the caches. Each is an entity or a projection of one (Data Discovery).
- Writers: for each table, search for the code that inserts, updates or deletes into it. The result is the state-change map, and it is usually the most surprising artefact of the exercise (Who Owns This State?).
- External: every outbound HTTP client, SDK and credential in the config. Each is a boundary and a failure mode (Inside and Outside the System).
- Check it: pick the pending change and walk it through the recovered design; where the design cannot say what the change touches, read there.
Worked on a concrete problem
The move has to produce something. This is what it produced.
- Starts: one HTTP server with a route table; a nightly job that reconciles payments; a webhook handler for the provider. Inputs: customer routes (catalog, cart, checkout), admin routes (products, stock), the provider webhook, the nightly job's own query. Storage:
products,carts,orders,order_items,payments,inventory, plus an object store for images. Writers toinventory: the admin stock endpoint, the checkout service on order creation, and — unexpectedly — the nightly reconciliation job, which restocks cancelled orders. External: the provider (charge, refund, webhook), an email API, the object store. - The warehouse request, walked through: inventory is one table with a quantity per product and no location; three writers decrement or restock it; the checkout writer assumes one stock number per product. A second warehouse touches the schema, all three writers, and the admin UI — and the estimate now has a basis, which is the writer list, not the folder count.
- A module that appeared in no answer:
services/recommendations. Not routed, not scheduled, not a writer, not a caller. Dead code from a feature that was cut; it would have been a box on the folder diagram, and it is not on the recovered one.
How you know it worked
What now exists that did not before, and what question you can now ask.
- You have a list of every way execution starts, every input, every store with its writers, and every external system — each item pointing at code or configuration.
- You can say, for a proposed change, which of those it touches, and where the recovered design is not sure.
- The picture distinguishes what runs from what merely exists in the repository.
- The diagram, if you drew one, could be regenerated from the tables.
The questions you can now ask
The field this whole domain exists for. After this lesson, these are the questions to put to an unfamiliar problem.
- ?Where can execution begin in this system, and have I listed every way?
- ?What does each starting point receive, and from whom?
- ?What is stored, and which code writes to each store?
- ?What does this system call that it does not own, and what happens when that call fails?
- ?For the change in front of me, which of these does it touch — and where can the recovered design not say?
What can go wrong
- Recovering the whole system when the request touches one corner. Five questions asked of the entire store is days; asked of "everything that touches inventory" is an afternoon, and the request only needs the afternoon.
- Trusting the migrations folder over the live schema, or the route file over the running process. Systems drift; the running thing is the truth and the repository is a claim about it.
- Missing writers that go through indirection — an ORM hook, a trigger, a raw SQL string built at runtime. The search for writers has to include these or the state-change map is falsely reassuring.
- Drawing the diagram first and then looking for facts to support it, which is the folder reflex with a delay.
- Recovering from observations is slower than reading a good design document, when one exists. The move is for when it does not, or when it is not trusted.
- The recovered design is a snapshot. Without maintenance it drifts like the README did, and the writer list in particular goes stale with every new feature.
- Searching for writers is thorough only if the indirections are known; a system with dynamic SQL or database triggers hides writers from a text search, and the exercise has to include the database's own view of who changes what.
- "Reverse engineering means reading the binary." Here it means recovering design from a system whose design was never written; the code is available and the question is what it amounts to.
- "Once recovered, the design is documented." It is recorded for the questions you asked. The warehouse change needed inventory's writers; a payments change will need the webhook's and the reconciliation job's, which the first pass may have skimmed.
- "The folder structure is useless." It is a weak hint about intent, and intent is one input. It is not evidence about behaviour, which is what the five questions gather.
Where this applies
Problem-solving advice is stated as universal far more often than it is. These labels say what each method is specific to — and where CONTESTED appears, the note gives the strongest form of the opposing view.
- GENERALStart, inputs, storage, writers, externals recover a web application, a data pipeline or a desktop tool alike; for a pipeline the "inputs" are files and topics and the "writers" are stages.
- SCALE-SPECIFICFor a single service with one database the whole exercise is an afternoon and the writer list fits on a page; across a dozen services with shared tables, the writers span repositories and the state-change map is the most valuable and most expensive artefact — often worth a tool rather than a search.
- ILLUSTRATIVEThe nightly reconciliation job, the three inventory writers and the dead recommendations module are invented to show the recovery producing facts the folders did not; no real system is described.
Where the depth lives
This domain asks the question and hands the answer off by name.
- — An Architecture domain link for "recover the current architecture before changing it" would go here; use the writer table as the input to any architecture discussion about the store.