Filtering: An Allowlist With an Index Bill
Every filter parameter is a promise that a class of database queries will stay fast forever. Explicit, typed, allowlisted filters keep that promise affordable; a generic query language hands your query planner to strangers.
Frame the contract
API design starts with a consumer, a design question and a guarantee — never with a URL.
A filter parameter is a standing query contract
When the docs say GET /orders?status=paid&created_after=2026-01-01, they are not describing a convenience — they are promising that this *shape of query* is a supported operation at production scale, indefinitely. The client cannot know or care whether the predicate is served by an index seek or a 40-million-row scan; the contract made both look identical. Which means the filter list is really a list of query plans you have agreed to fund, and every field you add to it adds rows to that bill (see What an API Contract Actually Is).
This is why the design position is an allowlist: a small, explicit set of filterable fields, each with defined operators (status equals one of an enum; created_after/created_before bound a range) and each backed by a deliberate index decision. The opposite default — "we pass query params through to the WHERE clause" — ships fast and fails in two directions at once: performance (any unindexed field becomes a scan endpoint the moment someone filters on it) and security (client-controlled field names flowing toward SQL is the posture injection lives in, even when your ORM technically escapes everything).
Filters are input, so they get the full Validation Errors: Feedback, Not Verdicts treatment: typed values (created_after must parse as a date — a silently ignored typo'd date returns the *unfiltered* collection, which for a reconciliation job is a correctness bug, not a UX flaw), enum values checked against the enum, and unknown filter fields rejected, not ignored. Silently ignoring ?staus=paid returns every order while the client believes it asked for a subset — the most expensive kind of wrong answer, an all-2xx one.
1GET /orders?filter[shipping_address.zip]=902102GET /orders?filter[internal_margin][gt]=0.43GET /orders?staus=paid → 200, ALL orders (typo ignored)4 5# any column is filterable → any column is a6# promised query plan → the client just chose7# a 40M-row scan; also: internal fields leak,8# and field names flow toward the query layer1GET /orders?status=paid&created_after=2026-01-012 3Filterable (documented, exhaustive):4 status eq enum: pending|paid|refunded5 created_after range RFC 3339 timestamp6 created_before range RFC 3339 timestamp7 customer_id eq id8 9GET /orders?staus=paid10→ 400 { "code": "validation_failed",11 "details": { "fields": [ { "path": "staus",12 "rule": "unknown_parameter" } ] } }The good side is smaller and that is the point: four fields, each typed, each index-backed, each documented. The bad side's flexibility is a blank check — on performance (scans), on security (field names as input), and on evolution (every internal column that ever worked in a filter is now load-bearing).
The combinatorics: promises multiply, indexes do not
The subtle trap is combinations. Four filterable fields plus a sort order look like four promises; they are closer to dozens, because clients combine them: status + created_after, customer_id + status + sort by total… A composite index on (status, created_at) serves the first beautifully and does nothing for customer_id + total. Each *combination pattern* is its own query plan, and B-tree index mechanics (leftmost-prefix rules, one range condition, sort direction) decide which combinations one index can cover — the contract you can afford is downstream of index design, which is why filter design is a conversation with whoever owns the schema (see Cursor Pagination: An Opaque Bookmark, Not a Position for the same bill arriving via ordering).
You do not need an index per combination — you need a *policy* for combinations. The workable options: keep the filterable set small enough that the handful of real combinations are each covered (the usual answer); designate some filters as always-required so the index prefix is guaranteed (customer_id required on a per-tenant API — which is also the authorization boundary doing double duty, see Authorization Design in the Contract); or explicitly tier the contract — "these combinations are fast; anything else may be slow or rejected". What is not workable is silence, where the docs imply every combination is equal and the database disagrees at 2 a.m.
Filters also interact with Pagination: Choosing How Lists End in one way worth engineering deliberately: the filter set is part of the traversal's identity. A cursor minted under status=paid must not continue under status=refunded — bind the filter hash into the cursor and reject mismatches (see Cursor Pagination: An Opaque Bookmark, Not a Position). And every filtered traversal still needs the total ordering and tiebreaker; a filter that shrinks the result does not excuse the ordering contract.
| Policy | Consumer gets | Provider pays | Fails when |
|---|---|---|---|
| Small allowlist, all combinations indexed | Predictable speed everywhere | A few composite indexes; saying "no" often | Consumers genuinely need ad-hoc queries |
| Required tenant/anchor filter + allowlist | Fast within their partition | One index family; contract stiffness | Cross-tenant/admin queries need a separate path |
| Tiered: fast set + documented slow set | Flexibility with honest labels | Two SLOs; complexity in docs and limits | Clients ignore the labels and build on the slow tier |
Generic query language (?q=field:op:value…) | Arbitrary queries | Query-cost control, planner babysitting, injection surface | Any consumer discovers an expensive shape and automates it |
When consumers really do need arbitrary queries
Sometimes the requirement is genuinely open-ended — an analytics surface, an admin power-search, an integration platform. The answer is still not "pass params to the WHERE clause"; it is recognizing that you are now building a *query API*, a different product with different machinery: a defined grammar with a parser (never string concatenation), per-field capability declarations, query cost estimation with rejection of expensive shapes, tight result and time limits, and usually a separate datastore fed by replication so the analyst's five-way filter never competes with checkout for the primary's buffer pool. What GraphQL Costs tells the same story from the GraphQL side; Search Is a Different Contract Than Filtering tells it for relevance-ranked text.
The decision discipline: start from measured consumer tasks, not imagined flexibility. Filter parameters are trivially additive — shipping four fields today and adding a fifth next quarter when telemetry shows clients client-side-filtering on it is cheap (see the observe list below). Shipping twenty speculative fields today means twenty standing promises, most unused, some unindexed, all load-bearing the moment one consumer automates against them. Filters are the clearest case in API design where the small contract is the strong one.
- Open-ended querying is a product, not a parameter: grammar, cost control, limits, often a replica-fed store.
- Filters are additive — ship the measured few, add on demand; removal is the breaking direction (see Backward Compatibility: The Real Rules).
- Watch what clients fetch-then-discard: client-side filtering is the demand signal for the next filter field.
- Filterable ≠ returnable: a field can be in responses but not filterable — and occasionally filterable but redacted in responses (existence is then still leaked; decide deliberately).
Key points
- Every filter parameter is a standing promise that a query shape stays fast at scale; the filter list is a list of query plans you agreed to fund.
- Allowlist fields and operators explicitly; validate types and enums; reject unknown parameters — a silently ignored typo returns the unfiltered collection as an all-2xx wrong answer.
- Combinations are the real contract surface: composite-index mechanics decide which combinations one index covers, so filter design is co-design with the schema owner.
- Have a stated combination policy — small-and-covered, required anchor filter, or honest tiers — instead of implied uniformity the database cannot deliver.
- Bind filters into cursor identity so traversals cannot continue under changed predicates.
- Genuinely open-ended querying is a separate product with a parser, cost control and usually a separate datastore — not a pass-through WHERE clause.
Follow the failure
How the contract fails or gets misused, hop by hop — and what it costs when it completes.
- 1Team → API: maps query params straight onto the ORM's
where()— every column filterable, zero docs needed, ships in a day. - 2Consumers → API: discover filtering works on anything; dashboards and scripts accrete filters on unindexed columns.
- 3Collection → scale: a partner automates
?refund_reason=…(no index) hourly; each call is a full scan competing with production writes. - 4Provider → database: p99 degrades API-wide; the slow-query log is a catalog of filters nobody designed.
- 5Team → cleanup: tries to restrict the filter set and discovers every accidental filter has a consumer — the allowlist now requires a deprecation program.
- One consumer's creative filter degrades every consumer's latency — the blast radius of a table scan is the whole database, not the one endpoint.
- Silently ignored parameters corrupt downstream data: clients process unfiltered collections believing they asked for subsets.
- Internal schema leaks through filterable columns: renaming a column becomes a breaking change to filters nobody knew existed.
Design, observe, evolve
A contract decision is incomplete until you know how you would notice it failing and how it changes later.
- • Enumerate filterable fields and operators in the contract, generate validation from that enumeration, and reject unknown parameters with a field-level error.
- • Back every documented filter combination with an index decision before it ships; make "which index serves this?" a required line in the endpoint review.
- • Anchor per-tenant APIs with a required filter that doubles as the authorization boundary and guarantees the index prefix.
- • Cap result windows and execution time on filtered queries regardless — limits are the backstop for the combination you missed (see [[unbounded-collections]]).
- • Log filter-shape usage (which fields, which combinations) per consumer: it is simultaneously your index worklist, your deprecation evidence, and your next-filter demand signal.
- • Alert on slow-query-log entries originating from filter endpoints — each one is a contract promise the database is failing to keep.
- • Track `unknown_parameter` rejections: a cluster on one name (`staus`, `state`) is a docs or SDK bug; a scatter of internal column names is someone probing your schema.
- • Adding filter fields and operators is additive and safe; do it on measured demand rather than speculation.
- • Removing or restricting a filter is a breaking change requiring usage telemetry and a deprecation window — accidental filters have consumers too (see [[consumer-driven-evolution]]).
- • When ad-hoc demand outgrows the allowlist, add the query product (tiered or separate endpoint, replica-backed) beside the fast path instead of loosening the fast path's promises.
- • Allowlists mean telling consumers "no" — repeatedly, with each "no" a small product negotiation the pass-through design never has (until it has a database incident instead).
- • Every supported combination costs a composite index: storage, write amplification, migration time; the filter surface is bounded by the write budget, not imagination.
- • A required anchor filter simplifies indexes and authorization but stiffens the contract — the legitimate cross-partition consumer needs a second, deliberately-designed path.
Misconceptions
?is_admin=true as an oracle), or the DoS of unindexed predicates. The allowlist is the control; escaping is table stakes.status=failed (typo'd) and got *everything*, 200 OK. A rejected typo costs one round trip; an ignored one can cost a partner a corrupted reconciliation run. Reject unknown filter parameters specifically, whatever your body-field policy is.