The Three Validations
Well-formed, permitted, and consistent are three different questions with three different enforcement points — and only the last one survives a race.
The requirement, the obvious build, and why it breaks
Every lesson starts where the work starts: someone asked for something, and the first implementation that comes to mind has a problem.
When someone says "we validate that", which of three completely different checks do they mean?
Users sign up with an email address. The rules are: it has to look like an email, the invite it came from has to still be open, and nobody else may already have that address.
Add a validation schema at the endpoint. It checks the email format, checks the invite, and checks the database for an existing user. One place, one function, done.
The uniqueness check is a SELECT followed by an INSERT. Two simultaneous signups both find nothing and both insert, so the "validated" rule produced exactly the state it forbade (Database Constraints).
- The uniqueness check is a
SELECTfollowed by anINSERT. Two simultaneous signups both find nothing and both insert, so the "validated" rule produced exactly the state it forbade (Database Constraints). - The invite check runs at the endpoint. The bulk-import path and the SSO auto-provision path do not call that endpoint, so they create members with no invite at all.
- The format check is duplicated in the mobile client, the web client and the schema, and the three definitions of "an email" disagree — which is only discovered when a valid address is rejected.
- Someone adds a rule that needs a database read into the transport schema, so schema validation now issues queries and the endpoint's parse step can time out.
- Every check throws the same
ValidationError, so a genuinely malformed body, a forbidden state transition and a duplicate key all become 400 and the client cannot tell which to retry (Reporting Validation Failures).
What is actually happening
- Transport validation asks: *could this input possibly mean anything?* Types, shapes, formats, ranges, required fields, size limits. It needs no state at all, which is what makes it cheap, cacheable and safe to run first (Transport Validation).
- Business validation asks: *is this allowed, given what we know?* It needs loaded state and an identity: is the invite open, is the order in a status that can be cancelled, is the account under its seat limit (Business Validation).
- Database constraints ask: *is the resulting state consistent?* Uniqueness, referential integrity, check constraints, exclusion constraints. Enforced by the engine, inside the write, atomically (Database Constraints).
- The three differ on what they need: nothing, application state, and the storage engine's own guarantees. That ordering is why they run in that order.
- They differ on what they survive. Transport validation survives everything, because its answer does not depend on anything that can change. Business validation is a point-in-time reading and goes stale the moment it returns. Only a constraint is evaluated as part of the write itself, and only it survives concurrency.
- They are not substitutes. A unique index does not tell a user their email is malformed; a schema does not know whether an invite is open. Each is the only place its own question can be answered.
Three questions, three enforcement points
The single most useful table in this module. Read the last two columns first: what each layer can guarantee, and what it cannot. Most validation bugs are a rule placed in a layer whose "cannot" column contains the thing that mattered.
Note that the strength of the guarantee runs in the opposite direction to the cost of getting the answer. The cheapest check is absolute; the most expensive one is the only one that is atomic.
| Layer | The question | What it needs | Can guarantee | Cannot guarantee |
|---|---|---|---|---|
| Transport | "Is this well-formed?" | Nothing — the payload alone | Types, shapes, formats, ranges, size, unknown-field policy | That the value means anything in your domain |
| Business | "Is this allowed right now?" | Loaded state + the actor | Rules over state: status transitions, quotas, relationships | That the state is still true a millisecond later |
| Database | "Is the result consistent?" | The engine, inside the write | Uniqueness, referential integrity, check and exclusion constraints — atomically | A useful message, or any rule that needs application context |
| (Client) | "Should I even send this?" | Nothing | A faster, kinder form | Anything at all, from a security standpoint |
One signup, all three
Following one request through the three layers makes the division concrete. Each step rejects a different kind of wrong, at a different cost, with a different guarantee, and the last one is the only one that cannot be beaten by a second request arriving at the same moment.
The INSERT step is the one people leave out. Notice that the pre-check does not disappear — it exists to give a good error to the 99.9% of duplicates that are not races, while the constraint handles the rest.
- 1Size limit
Rejects an oversized body before it is parsed at all.
fails by A default limit assumed to be tuned; a 40 MB JSON body parsed into memory (Request Bodies and Streaming).
- 2Parse + shape
JSON to a typed command: email is a string matching a format, password meets length, no unknown fields.
fails by Passing unknown fields through, so
role: "admin"reaches an ORM create (Mass Assignment and Over-Posting). - 3Business rules
Loads the invite: does it exist, is it unexpired, unused, and for this email?
fails by Living in the handler, so the SSO provisioning path skips it entirely.
- 4Authorization
A different question with a different answer — may this caller create a member here? (Authentication vs Authorization)
fails by Being skipped because the business rule already "checked" something.
- 5Write inside a transaction
Inserts the user and marks the invite consumed as one unit (Where the Transaction Boundary Goes).
fails by Two statements, no transaction — the user exists and the invite is still open.
- 6Constraint decides
The unique index on
lower(email)succeeds or raises. This is the actual enforcement.fails by No index — two users with one email and no line of code that was wrong.
- 7Map the violation
Catches the constraint error and returns 409 with a stable code.
fails by Uncaught: a 500 containing the constraint name (Not Leaking Your Internals).
Steps 1-2 need no I/O, step 3 needs one query, step 6 costs an index write. Running them in this order means the cheapest check rejects the most obviously wrong input.
Where the rule actually goes
Given a new rule, the placement question has a short answer: what does it need in order to be evaluated, and what happens if two requests evaluate it at the same instant? Those two questions decide the layer.
The awkward cases — rules that span rows, or need a rule engine, or must be true across services — are worth naming as awkward rather than forcing into a layer that cannot hold them.
What does the rule need, and what does a concurrent request do to it?
when The rule is a function of the payload alone: type, format, length, range, enum membership, required fields.
cost Nothing at runtime. The cost is contract rigidity — strictness breaks clients on additive changes (Running Two API Versions in One Service).
when The rule needs loaded state or the actor: status transitions, quotas, relationships, "the invite is open".
cost Queries per request, and a point-in-time answer that can be stale before you act on it.
when The rule is about the resulting state and must hold under concurrency: uniqueness, referential integrity, non-negative balance, non-overlapping ranges.
cost A migration, index-maintenance write cost, and an unhelpful error you must catch and translate.
when A uniqueness or consistency rule that users hit routinely and deserve a good message for. The normal answer for signup emails, slugs and handles.
cost An extra query on every request; two places that state the same rule.
when The rule spans rows and no single constraint can express it — "at most five active projects per account".
cost Contention, and serialization failures the caller must retry (Pessimistic Locking, Optimistic Concurrency).
when The rule spans services or systems you do not write to atomically.
cost Detect and compensate instead of prevent, and say so explicitly (Eventual Consistency in Practice, The Dual Write Problem).
How to build it
Most important first.
- Run them in order and let each one narrow the input for the next: shape first (cheap, no I/O), then rules that need state, then the write with its constraints.
- Put transport validation at the edge, so nothing below has to wonder whether a field is a string (Parse, Do Not Validate).
- Put business rules where every entry point passes through — the service, not the handler — because the HTTP endpoint is rarely the only caller (The Service Layer).
- Back every business uniqueness or consistency rule with a real constraint, and treat the pre-check as user experience rather than as enforcement.
- Handle the constraint violation. A rule enforced only by an index produces a 500 unless the violation is caught and mapped to a meaningful response.
- Give the three different error shapes and different status codes, so a client can distinguish "fix your request" from "not allowed" from "someone got there first" (Reporting Validation Failures).
What can go wrong
- A business rule implemented only in the transport schema, so the job, the CLI and the admin path skip it entirely.
- A uniqueness rule implemented only in application code, which is correct in every test and wrong under concurrency (Backend Races).
- A constraint added without an application-side pre-check, so ordinary user mistakes surface as 500s with a database error message (Not Leaking Your Internals).
- Transport validation that reaches the database — an "async refine" that turns the parse step into a query, then a query per field, then an N+1 in a validator (The N+1 Query Problem).
- Business validation performed against a read replica, so a rule is evaluated against data that is seconds old (Eventual Consistency in Practice).
- The same rule implemented in all three places with three slightly different definitions, which is worse than any one of them alone.
- Every business validation is time-of-check/time-of-use. Between "the invite is open" and the insert, the invite can be revoked, and nothing in the application layer closes that gap (Database Constraints).
- Two requests can pass the same uniqueness pre-check concurrently. This is not a rare interleaving — a double-clicked submit button reproduces it (Duplicate Detection).
- A rule spanning two rows ("an account may have at most five active projects") cannot be enforced by a per-row constraint and needs a lock, a serializable transaction, or a denormalised counter with a check constraint (Pessimistic Locking).
- Only server-side checks count. Client-side validation is a user-experience feature and enforces nothing (The Trust Boundary).
- None of the three is authorization. "This field is a valid UUID", "this order can be cancelled" and "this id is unique" all say nothing about whether *this caller* may act on it (Object-Level Authorization, Authentication vs Authorization).
- Transport validation is a security control in its own right: size limits, field allowlists and type strictness are what stop oversized bodies, prototype pollution and mass assignment (Mass Assignment and Over-Posting, Request Bodies and Streaming).
- A validation error message that quotes the database error tells an attacker your table and column names (Not Leaking Your Internals).
- "We validate, so the data is correct." Validation is three different guarantees with three different strengths. Which one did you get?
- "The schema validates it, so it is enforced." Only for the callers that go through the schema.
- "Adding a unique index is a database concern." It is the *only* implementation of "emails are unique" that is actually true.
- "Validation and authorization are both just checks." They fail for different reasons, need different information, and must produce different status codes (Business Validation).
- "Validate early, once." Validate early *for shape*. The rules that need state cannot be answered early, and the ones that need atomicity cannot be answered by the application at all.
Operating it
- Count rejections separately per layer:
validation.transport.rejected,validation.business.rejected{rule},db.constraint_violation{constraint}. One counter for "validation errors" tells you nothing actionable. - A rising constraint-violation rate with a flat business-rejection rate is the signature of a pre-check that is racing rather than a client that is misbehaving (Backend Races).
- Log the field path and the rule name, never the value — the value is frequently a password, a token or personal data (Secrets in Logs).
- Alert on transport rejections from a single client version. That is usually a deploy mismatch, not an attack (Deploys Are the First Suspect).
- Transport validation cost scales with payload size and request rate. It is almost never the bottleneck, and when it is, the cause is a payload that got large rather than a schema that got slow (What Serialization Costs).
- Business validation cost scales with the queries it needs. A rule that loads three aggregates per request is three queries per request at any traffic level.
- Constraint enforcement cost is index maintenance on write, which is real and usually worth it. A unique index on a very hot insert path is a contention point on the index's rightmost pages (Composite Indexes and the Leftmost-Prefix Rule).
- At 100x concurrency the check-then-act gap does not widen — it just gets hit. The rule that was theoretically racy becomes a daily support ticket.
- Three layers means a rule can be expressed in more than one place, and keeping them agreeing is ongoing work.
- Strict transport validation breaks clients on additive changes that a lenient parser would have tolerated — a real cost on a public API (Running Two API Versions in One Service).
- Constraints require migrations, and adding one to a table that already violates it fails. Retrofitting uniqueness means cleaning existing duplicates first (Expand and Contract Migrations).
- Pre-checking a constraint costs a query on every request to give a better message on the small fraction that fail.
Where this applies
Backend advice is context-sensitive. These labels say what each claim is specific to, and where a different stack or scale would differ.
- GENERALThe three questions, and the fact that only the third is atomic with the write, hold on every stack and every engine.
- DATABASE-SPECIFICWhich rules the third layer can express varies: Postgres has partial indexes,
CHECK, deferrable constraints andEXCLUDEwith GiST, so "no two overlapping bookings for one room" is a single constraint. MySQL/InnoDB has no exclusion constraints, and itsCHECKwas only enforced from 8.0.16. SQLite enforces foreign keys only whenPRAGMA foreign_keys=ON, which is off by default. A rule that is one constraint on one engine is application code plus a lock on another. - SCALE-SPECIFICFlips on concurrency, not team size. Below roughly one write per second to the same key, a check-then-insert is wrong and will not be observed to be wrong — many products run for years like that. Once two requests can plausibly touch the same key within the check-to-write window (tens of milliseconds), the constraint is the difference between a correct system and a duplicate-cleanup script. Double-submit and client retries put you in that regime at any traffic level.
Where the depth lives
This domain teaches the application-side mechanism and hands the rest off.