Foundation Models
A foundation model is pretrained once on broad data and reused across many tasks. "Pretrained" hides a training system you did not run, data you did not choose and an objective you did not pick — and what you inherit shows up as serving cost and as benchmark numbers that are not your task's numbers.
The problem, the obvious approach, and why it breaks
Every lesson starts where the work starts: someone has a problem, and the first model that comes to mind looks fine offline.
What does "pretrained on broad data" actually give you, what does it hide, and why is a benchmark score not an evaluation of your system?
A claims-processing team wants to extract structured fields from scanned correspondence. A vendor's foundation model scores near the top of every published document-understanding benchmark. The head of operations wants to know whether that means it will work on their correspondence, and what it will cost to run on four hundred thousand documents a month.
Benchmarks are standardised evaluations, the model tops them, so it is the best available. Call the API per document, parse the fields, and size the bill from the per-token price times the average document.
The benchmark is a task with its own field definitions, layouts and label conventions. On the team's correspondence, with its own conventions — "date of loss" written three ways, amounts with currency codes — the extraction accuracy is not the benchmark number, and there is no way to know what it is without labelling the team's own evaluation set (Evaluation Slices).
- The benchmark is a task with its own field definitions, layouts and label conventions. On the team's correspondence, with its own conventions — "date of loss" written three ways, amounts with currency codes — the extraction accuracy is not the benchmark number, and there is no way to know what it is without labelling the team's own evaluation set (Evaluation Slices).
- The bill is not the average. Scanned pages tokenise into far more tokens than clean text, long correspondence attends quadratically, and the output is generated per token. Four hundred thousand documents at the tail of the length distribution is a different number from four hundred thousand times the mean (Inference Cost).
- When the vendor updates the model, the extraction behaviour changes — a field the old version formatted one way now comes back another — and there is no diff to read, because the artifact is a name, not a file (The Model Supply Chain).
What is being predicted, and from what data
This domain leads with these two. A target nobody defined precisely is a label nobody can trust, and a dataset nobody can describe is a model nobody can debug.
- Extract a fixed set of fields — claim number, date of loss, claimant, amount — from each document; the label is the value an operator keyed, and the decision is whether the case proceeds without manual entry.
- The benchmark measured something related but different: general document question-answering on public datasets, scored by exact match on their fields.
- Four hundred thousand documents a month, scanned at variable quality, in the team's own layout conventions, with operator-keyed values for the last two years as labels.
- The foundation model's training data is undisclosed beyond "web-scale documents and text"; its objective was next-token prediction with instruction tuning on top; the training run is not reproducible by anyone outside the vendor.
- The benchmark datasets are public, which means their contents may be in the model's training data — a form of evaluation leakage nobody can rule out (Evaluation Leakage).
How it actually works
Precisely enough to predict its behaviour — not a framework API.
- A foundation model is a large transformer (Transformer Fundamentals) trained by self-supervision on a broad corpus: predict the masked or next token over enough text and the representations become generally useful (Self-Supervised Learning). The value is that the expensive part — learning language, layout and world structure — was paid once and can be amortised across downstream tasks.
- What "pretrained" abstracts is a training system: data collection and filtering at web scale with choices you cannot inspect, a training run on thousands of accelerators for weeks with a budget you did not authorise, and an objective — next-token likelihood, then preference tuning — that is not your extraction accuracy. The model is very good at its objective; the correlation between that objective and your task is what you are actually betting on (Don't Delegate Understanding).
- The serving consequence follows from the size: tens of billions of parameters means tens of gigabytes of weights, per-token generation bounded by memory bandwidth, and an accelerator or a hosted API for every call (CPU or GPU for Inference, Memory Bandwidth & VRAM). Quantisation and distillation trade some quality for a smaller, cheaper model (Quantization, Pruning & Distillation).
What "pretrained" hides
The word makes it sound like a property of the weights. It is a description of a system: a corpus assembled and filtered by people who were not thinking about your documents; a training run at a scale no team reproduces; an objective chosen because it scales, not because it is your metric; and an evaluation against public benchmarks that are, at best, a proxy. All of that is upstream of the file you download or the endpoint you call.
Inheriting it is the whole value proposition and the whole risk. The value is that the expensive generic learning is done. The risk is that everything you would normally check — what data, what split, what leakage, what objective — was decided elsewhere and is not available to check. The domain's rule about not delegating understanding applies here with the largest artifact you will ever ship (Don't Delegate Understanding).
- 1Corpus assembly
Web-scale collection, deduplication, filtering by heuristics and classifiers
fails by Your domain is under-represented or filtered out; public benchmarks are included
- 2Pretraining
Next-token or masked-token objective over the corpus on thousands of accelerators for weeks
fails by The objective rewards fluent likelihood, not your field accuracy
- 3Instruction / preference tuning
Fine-tuning on curated instructions and human preferences
fails by Preferences are for helpfulness in general, not for your format; behaviour shifts between versions
- 4Evaluation
Public benchmarks, reported as leaderboard scores
fails by Benchmarks may be contaminated; they are not your task
- 5Release
Weights or an endpoint, with a model card
fails by The card describes the vendor's evaluation; your evaluation does not exist yet
Every step is one this domain teaches you to inspect for your own models. For a foundation model you cannot, so the inspection moves to the only place it can happen: your own evaluation set, on your own data, re-run on every version.
Benchmarks against your task
The benchmark and the task share a name — document extraction — and differ in every particular that decides accuracy: field definitions, layout conventions, scan quality, label format. A benchmark rank orders models on the benchmark; it does not order them on the team's correspondence, and the ordering can flip on the slice that matters.
The only evaluation that predicts production is one built from production: the team's documents, the operators' keyed values, sliced by type and quality. It costs a few thousand labels. It is the cheapest thing in the project and the first thing that gets skipped.
Top-three on the public document-understanding leaderboards; vendor demo on a handful of the team's documents looked clean.
Exact-match on the operator-keyed fields was well below the benchmark figure in aggregate, and far below on handwritten annotations, which are a fifth of volume; the monthly bill came in well above the mean-based estimate.
- 1The benchmark measures different fields under different conventions on cleaner documents; the team's task was never measured before rollout.
- 2Handwritten annotations are a slice the benchmark barely contains and the aggregate number could not reveal.
- 3Scanned pages tokenise heavily and the long tail of multi-page correspondence attends quadratically, so cost tracked the tail, not the mean.
The foundation model's general capability translates to the team's field-extraction accuracy at the level measured, and continues to on the next model version.
holds when The model version is pinned; the evaluation set is re-run before any version change is adopted; the document mix is monitored for new sources.
breaks when The vendor updates in place; the preference tuning shifts output formats; a new correspondence source arrives with layouts the evaluation set did not cover.
respond Hold the previous version if possible; re-evaluate and adjust the parser or the prompt template; if the gap persists across versions, move the task to a model you control.
What it costs to serve, and when it is the wrong tool
Size decides serving. Tens of billions of parameters is tens of gigabytes of weights, which means a GPU with that much memory or a hosted endpoint, and per-token generation bounded by how fast those weights stream through the accelerator's memory (Memory Bandwidth & VRAM, CPU or GPU for Inference). Quantising to fewer bits per weight and distilling to a smaller student both shrink the bill and both move the accuracy, so they are re-evaluated on the same set (Quantization).
And the honest question is whether the foundation model is the tool at all. For a fixed set of fields on known layouts, at four hundred thousand a month, a fine-tuned encoder with a layout-aware head is cheaper by orders of magnitude, faster, and inspectable. The foundation model earns its cost when the task is open-ended, the layouts unbounded, or the labelled data unavailable — and that is a measurement, not a default.
| Option | Quality | Latency | Cost | Interpretability | Data needed | Operational | Note |
|---|---|---|---|---|---|---|---|
| Hosted foundation model, prompted | No labelled data needed to start; version and price controlled by the vendor. | ||||||
| Open foundation model, self-hosted and quantised | You own the version; you also own the GPUs and the serving stack. | ||||||
| Fine-tuned small encoder with a field head | Needs the labelled set — which the evaluation already required. | ||||||
| Layout rules + classifier fallback | Brittle across new layouts; the honest baseline the others must beat. |
caveat The quality scores are equal only because the team's evaluation set has not been run; the whole point is that the ordering on that column is unknown until it has been. The dataNeeded column hides that the evaluation set is required regardless, which narrows the foundation model's advantage.
How to build it
Most important first.
- Build your own evaluation before you compare models. A few thousand of the team's documents with operator-keyed fields, sliced by document type and scan quality, is the only benchmark that predicts production (Evaluation Slices, Golden Datasets on the Agentic side covers the same idea for agent tasks).
- Price from the token distribution, not the mean: log tokens in and out per document on a representative sample and multiply through the tail (Inference Cost).
- Pin the model version and treat a vendor update as a model change with a regression run against your evaluation set before it is adopted (Model Regression Tests, The Model Supply Chain).
- Ask whether a foundation model is the right tool at all. For a fixed set of fields on a known layout, a small fine-tuned encoder or even a layout-aware rules-plus-classifier system may match the accuracy at a fraction of the cost and with inspectable behaviour (When Not to Use ML).
What to measure
Which number actually maps to the decision — and which numbers look relevant and are not.
- Field-level exact-match rate on the team's own labelled documents, per field and per document type — the number that decides whether cases proceed without manual entry.
- Cost per document at the p50, p90 and p99 of the token distribution, and the monthly total that implies.
- Do not measure by the published benchmark rank. It measures a public task with possibly-leaked data and says nothing about the team's field conventions.
What must stay true after deployment
The field this whole domain exists for. A model is a set of assumptions with weights attached; these are the ones a monitor or a test should be checking.
- The correlation between the model's pretraining objective and the team's extraction task, as measured on the team's evaluation set, persists across model versions and across changes in the document mix.
- The model version serving production is the version that was evaluated; a vendor-side update is detected and re-evaluated before it reaches the queue.
- The token-length distribution of incoming documents stays within the range the cost estimate was built on; a new document source with longer scans moves the bill.
- Offline: run the candidate model over the team's labelled set and report per-field, per-slice exact match; compare against a small fine-tuned encoder baseline on the same set.
- Online: shadow the model against operator keying for a month, logging model version, tokens and disagreement per field (Shadow Deployment).
- Over time: re-run the evaluation set on every version change and on a monthly cadence, and track cost per document against the estimate; both are model-monitoring signals for a model you do not own (Model Monitoring).
What can go wrong
- The model is excellent on typed correspondence and poor on handwritten annotations, which are a fifth of the volume and most of the disputed claims; the aggregate number hid the slice that matters.
- A silent vendor update improves the benchmark and changes the date format the model emits; the parser downstream fails for a week before anyone connects the two.
- The team adopts the model, then the vendor raises the price or retires the version; the extraction system has no fallback and no in-house artifact.
- A foundation model gives broad capability with little labelled data at the price of size, cost, latency and a dependency on a training process you cannot inspect or reproduce.
- A hosted API removes the serving problem and adds a version you do not control, a price you do not set and data leaving your boundary (ML Privacy).
- A smaller task-specific model is cheaper, faster and inspectable, needs your labelled data, and will not generalise to the next task the way the foundation model would.
- "It tops the benchmark, so it is the best model for us." It is the best model on that benchmark's task, layouts and labels, possibly having seen the benchmark in training. The team's task has not been measured.
- "Pretrained means we skip the training problems." Pretrained means someone else had the training problems and you inherit their outcome — their data biases, their objective, their cut-off — without the ability to inspect any of it.
- "A bigger foundation model is always the safer choice." Bigger is more capable on average and more expensive per token, slower per call and harder to serve. For a fixed extraction task with a clear latency budget, the smaller fine-tuned model is often the safer choice, and a rules baseline the honest first one.
Where this applies
ML advice is stated as universal far more often than it is. These labels say what each claim is specific to — and where CONTESTED appears, the note gives the strongest form of the opposing view.
- GENERALThat a pretraining objective is not your task metric, and that a public benchmark is not an evaluation of your system, holds for every foundation model on every task; what differs is how far the two diverge, which only your own evaluation set can say.
- DOMAIN-SPECIFICThe advantage of a foundation model is largest where the task lives in natural language or images with broad structure to inherit; it is smallest on tabular business data, on tasks with strict latency budgets, and where a regulator needs an inspectable decision — there a small model or rules is usually the right tool.
- CONTESTEDA serious position holds that foundation models have made task-specific training obsolete for most language and vision work: the cost of labelling and training your own model exceeds the API bill, the vendor's model improves without your effort, and prompt-level adaptation covers most of what fine-tuning used to do. That is right for many teams and tasks with modest volume; it is wrong where volume makes per-token cost dominant, where latency is tight, or where the task must be auditable — and the only way to know which side you are on is to measure both.
Where the depth lives
This domain teaches the model and hands the rest off by name.