Where a Test Must Be Real
Some abstractions are load-bearing precisely because the thing underneath them is complicated. Replacing those with a double tests your belief about the dependency rather than the dependency.
The requirement, the obvious build, and why it breaks
Every lesson starts where the work starts: someone asked for something, and the first implementation that comes to mind survives until the requirement changes.
Which parts of this design are only meaningfully tested against the real thing?
A repository test replaces the database with an in-memory list. Every test passes. In production the first save fails on a unique constraint the fake did not have.
Everything gets a fast unit test. Where a dependency is inconvenient, substitute an in-memory version — a dictionary for the database, a list for the queue, a recording object for HTTP. The suite runs in eight seconds.
The fake implements the interface, not the semantics. A dictionary has no unique constraints, no transactions, no isolation level, no null handling, no collation, no case-sensitivity and no timezone behaviour — and every one of those has produced a production incident somewhere.
- The fake implements the interface, not the semantics. A dictionary has no unique constraints, no transactions, no isolation level, no null handling, no collation, no case-sensitivity and no timezone behaviour — and every one of those has produced a production incident somewhere.
- Serialisation is where it bites hardest. A round trip through the fake is identity; a round trip through the real store loses timezone, precision, key order and type, and the code that was correct in the test is wrong in production (Leaky Abstractions).
- The fake drifts. It was accurate the day it was written, the real schema gained a check constraint, and nothing failed — the fake has no schema to gain it.
- It hides the N+1. A dictionary lookup is free, so a loop that issues 400 queries in production looks fine in the test and is only discovered under load (N+1 as a Design Problem).
What limits the solution, and what must never stop being true
This domain leads with these two. A design that ignores its constraints is not a design, and an invariant nobody named is one nothing is protecting.
- The real database is Postgres; an in-memory substitute is not it, whatever the ORM claims.
- CI runs on machines that can start containers, but every container costs seconds of every build.
- Some dependencies genuinely cannot be run locally — a payment provider, a partner bank — so "just use the real one" is not universally available.
- A test that passes must give real evidence about production. A green test against a fake that does not share the real thing's semantics is worse than no test, because it is believed.
- Constraints enforced by the database are part of the design and must be exercised by something (Enforcing Invariants).
Who owns what, and where the seams fall
Responsibilities decide boundaries; boundaries decide what an interface has to say.
- The integration test owns the claim "this abstraction is faithful" — it is the only test that can make that claim at all.
- The unit tests own the rules, which do not need the real dependency and should not wait for it.
- The abstraction itself owns hiding the dependency from the rest of the codebase, which is exactly why its own fidelity has to be checked somewhere (Information Hiding).
- The line is not "fast versus slow". It is: does this code contain a decision, or does it contain a translation? Decisions are tested in isolation; translations are only real when translated.
- Every place your code hands meaning to something that does not share your type system — SQL, HTTP, a serialiser, a filesystem, a queue — is a translation, and translations are where fakes lie (Boundary Adapters).
- The boundary of the integration test is deliberately the same as the boundary of the abstraction, so the test is testing the promise the abstraction made (Designing a Module Interface).
What a fake cannot know
The case against in-memory substitutes is not theoretical, and it is not about purity. It is a list of specific behaviours that the real dependency has, the substitute does not, and production code depends on without saying so.
Read the right-hand column as a list of incidents. Every row is something a team shipped green.
| The real dependency does this | The in-memory double does this | What ships |
|---|---|---|
| Rejects a duplicate on a unique index | Overwrites the key silently | Two accounts with the same email, discovered by a customer |
| Rolls the whole transaction back on failure | Keeps whatever was already mutated | Half-applied writes after any error path (Partial Failure) |
| Stores a timestamp with a timezone and returns it in another | Returns the same object you put in | Renewals firing an hour early twice a year |
| Truncates or rejects a string over the column length | Stores any length | Silent data loss on the one long address |
| Enforces a foreign key | Has no notion of one | Orphan rows that only the reporting job notices |
| Compares strings by collation, sorts accordingly | Sorts by code point | Pagination that skips rows for non-ASCII names |
| Issues one query per loop iteration | A hash lookup, free | A page that is fine at 10 rows and times out at 10,000 (N+1 as a Design Problem) |
| Takes a lock and blocks a concurrent writer | Cannot block | Lost updates under concurrency, unreproducible locally (Concurrency by Design) |
Deciding what gets a real dependency
This is a budget, not a principle. Real dependencies buy fidelity and cost seconds, flakiness and infrastructure; the question is which parts of the design are worth spending it on.
The scores below are a way of laying out the exchange, not a measurement — there is no unit in which "testability 4" means anything absolute.
| Option | Simplicity | Testability | Operational | Performance | Note |
|---|---|---|---|---|---|
| In-memory fake of the repository interface | Fast and easy, and tells you nothing about the translation. Fine for testing code that *uses* the repository; not evidence about the repository. | ||||
| Real dependency in a container, per build | The strongest evidence available short of production. Costs build time, CI configuration and occasional container flakiness. | ||||
| Shared long-lived test database | Faster to start, but tests interfere unless carefully isolated, and the isolation mechanism becomes its own source of flakes. | ||||
| Fake plus a contract test against the real thing | The right answer when the real dependency cannot run locally — a payment provider, a partner API. Weaker: the contract test runs less often, so the fake can be wrong for a while (Contract Tests). |
caveat The scores compare mechanisms, not situations, and they cannot express the one variable that decides it: how much of your correctness lives in the translation. A system whose invariants are enforced by database constraints has almost all of its risk in row two and should pay for it; a system whose database is a dumb key-value store for JSON blobs has almost none, and the same investment there is waste. Read the axes as a shape, never as a score.
What the test should actually assert
An integration test that saves a row and reads it back proves the connection string is right. The valuable assertions are the ones about semantics you are relying on and did not write down anywhere else.
The test below is short on purpose: it is a list of the promises the repository is making on behalf of the rest of the codebase.
1// what the rest of the codebase believes about this repository2test('email is unique, case-insensitively', async () => {3 await repo.save(user({ email: 'A@x.com' }))4 await expect(repo.save(user({ email: 'a@x.com' })))5 .rejects.toBeInstanceOf(DuplicateEmail) // not a raw driver error6})7 8test('a failed step leaves nothing behind', async () => {9 await expect(service.register(badPayload)).rejects.toThrow()10 expect(await repo.count()).toBe(0) // transaction, not cleanup11})12 13test('renewal date survives a timezone round trip', async () => {14 const saved = await repo.save(sub({ renewsOn: '2026-03-29' }))15 const read = await repo.byId(saved.id)16 expect(read.renewsOn).toBe('2026-03-29') // not 03-28T23:00Z17})18 19test('loading an order does not issue a query per line', async () => {20 await withQueryCounter(async (n) => {21 await repo.orderWithLines(id)22 expect(n()).toBeLessThanOrEqual(2)23 })24})Each test names a promise the repository makes to callers — a typed duplicate error rather than a driver exception, atomicity, date fidelity, bounded query count. None of them can be made by a substitute, and all four are things callers already assume (Cost-Aware Interfaces).
How to build it
Most important first.
- Run the real dependency where you can. A containerised Postgres of the same major version is not a heroic amount of infrastructure and turns a class of production surprises into build failures (Where a Test Must Be Real).
- Where you cannot run the real one, run a fake and verify the fake against the real one on a schedule — that verification is the contract test, and without it the fake is an assertion nobody checks (Contract Tests).
- Keep them few and thick. Integration tests should cover the paths where the translation matters, not re-cover every business rule that is already tested in a millisecond upstairs.
- Test the constraint, not just the happy write. Insert the duplicate. Roll back the transaction. Store the timestamp in one timezone and read it in another. Those are the assertions that would have caught the incident.
- Design so there is a small amount of translation to test. If SQL is spread through forty files, the integration surface is forty files wide; if it is behind a repository, it is one (Repository Structure).
What the next change costs
The field this whole domain exists for. A structure is only better if it makes the change after this one cheaper — and it is worth saying which changes it does not help.
- Adding a field with a new constraint costs: one migration, one repository change, and one integration test that fails immediately if the mapping is wrong. Minutes, and the feedback is at build time.
- Under the in-memory fake the same change costs the same minutes plus, with some probability, a production incident with a rollback and an incident review. That is the actual comparison — not "eight seconds versus ninety seconds".
- What stays expensive either way: changing the database itself. Integration tests are written against Postgres semantics, so a move to a different store invalidates them, and that is correct rather than a flaw — they were always testing the translation (Data Migration).
- Integration tests are slow, occasionally flaky, and need infrastructure in CI. That is a real, daily, recurring cost, and teams that minimise them are responding to something true.
- They fail for reasons unrelated to your change — a container that did not start, a port collision — which erodes trust in the suite, and an untrusted suite is close to useless.
- Running the real dependency locks the tests to that dependency's semantics, which slightly raises the cost of ever changing it. That is a price worth naming rather than pretending away.
What can go wrong
- The suite becomes all integration tests, takes forty minutes, and gets skipped — a slow suite is a suite that stops being run, which is a worse outcome than a fast shallow one.
- Tests share a database and interfere; someone adds a
TRUNCATEbetween tests and the suite becomes serial and slow, or they do not and it becomes flaky. - The container version drifts from production, so the test now faithfully verifies a database you do not run.
- The fake-plus-contract-test approach is adopted and the contract test is never run against the real provider, so the fake is still unverified and now has a reassuring name.
- The build now depends on being able to start a real dependency, which is a genuine operational dependency and a real cost when it breaks.
- The translation layer depends on the real thing's semantics in ways that are not in the type system — the test is the only place that dependency is written down.
- Business rules depend on none of this, which is the whole reason to have separated them (Functional Core, Imperative Shell).
- "So test everything against a real database." No: the rules do not need it, and putting them there makes a fast suite slow for no additional evidence (What a Unit Is).
- "An in-memory implementation of the same interface is equivalent." It shares the interface and not the semantics, and every bug in this lesson lives in that gap (Test Doubles, Precisely).
- "SQLite is close enough to Postgres." It differs in types, constraints, concurrency and date handling. Close enough is exactly the property that makes it dangerous — it passes until it does not.
- "The ORM abstracts the database, so the database cannot surprise us." The ORM abstracts the syntax. Constraint violation, isolation, lock contention and query plans all come straight through (Leaky Abstractions).
Testing it, and how it ages
- One integration test per translation, exercising the parts of the semantics you rely on: constraints, transactions, ordering, timezone, null handling.
- Round-trip tests for anything serialised — write it, read it, assert equality of the domain object rather than of the bytes (Value Objects).
- A query-count assertion on the paths that load collections, so an N+1 introduced by a lazy-loading change fails the build (Cost-Aware Interfaces).
- Business rules stay upstairs and untouched by any of this (What a Unit Is).
- The integration suite grows slowly and stays small if the translation surface stays small. If it is growing fast, that is a signal that infrastructure knowledge is leaking upward into code that should not have it (Invariant Leaks).
- It ages badly against managed services with no local equivalent. The honest response is a fake plus a periodically-run contract test, and the honest admission is that this is weaker evidence (Contract Tests).
- When the suite gets slow, the first move is almost never "delete integration tests" — it is to check how many of them are re-testing rules that have unit tests already.
Where this applies
This domain's advice is contested more than most. These labels say what each claim is specific to — and where CONTESTED appears, the note gives the strongest form of the opposing view rather than a caricature.
- GENERALThat a substitute sharing an interface does not thereby share semantics is true of any dependency with behaviour of its own, so the argument holds for databases, queues, filesystems and HTTP alike.
- DOMAIN-SPECIFICFor a CRUD system whose logic is mostly persistence, almost all the risk is at this boundary and the integration suite is the primary suite. For a system whose value is a complex calculation over data loaded once, the boundary is thin and the integration tests are a handful — the same advice produces very different suite shapes.
- CONTESTEDA serious opposing view holds that fast isolated tests are so much more valuable per second of CI that a small, well-maintained fake plus a thin smoke test beats a real dependency: containers are flaky, they serialise the build, and the failures they catch are also caught by a staging deploy an hour later. That is a coherent position and it is right when the translation layer is genuinely thin and the schema is stable. It fails specifically where constraints and transactions carry business invariants, because then a staging deploy catches the bug after it has already been written into the design.
Where the depth lives
This domain teaches the codebase-level structure and hands the rest off.
- — Testing & Reliability Engineering — test data management, database isolation between parallel tests, container lifecycle in CI and flake triage are that domain's problems. This lesson only argues about which boundary deserves a real dependency.
- — System Design — deciding what to run in CI versus what to catch in a staging environment or a canary is a delivery-pipeline design question with its own economics.