Choosing the Right Abstraction
Escalate plain code → LLM call → structured output → tool calling → RAG → workflow → agent → multi-agent one rung at a time, and stop at the first rung that solves the problem.
The escalation ladder
The central skill of this domain is refusing complexity you have not earned. The ladder below is ordered by how much control flow the model owns and, almost exactly, by cost, latency and debugging pain. The rule: start at the lowest rung that could plausibly work, measure, and climb only when measurement says the current rung fails.
Each rung is shown with the same running example — handling inbound support email — so the escalation is visible as added machinery, not a different product.
Rung by rung, with the same example
Inbound email: "My invoice #4821 shows 120 EUR but I was quoted 100." What is the least machinery that handles it well?
- Plain code: regex the invoice number, look it up, auto-reply with a link. Works for a surprising share of mail. Zero model cost, fully testable.
- LLM call:
llm("Classify this email: billing | technical | other"). One call, text out, still fixed flow. Use when the input is free text and the output is a label or a draft. - Structured output: same call but the model must return
{"category": "billing", "invoice_id": 4821, "claimed_amount": 100}validated against a JSON schema. Now downstream code can act on it safely — see Structured Outputs. - Tool calling: the model may call
get_invoice(4821)andget_quote(customer)to compare amounts before drafting. One or two calls in a fixed pattern; the runtime executes — see Tool Calling Basics. - RAG: retrieve the refund policy and the customer's contract terms so the draft cites the actual rule instead of a hallucinated one — see RAG Overview.
- Workflow: classify → fetch → compare → (if mismatch) create refund request → notify. A persisted graph with retries and a human approval node — see Workflow State Graph.
- Agent: the discrepancy could be a currency conversion, a partial shipment, a duplicate quote or a fraud pattern; the model investigates with tools until it can explain the difference, under a step cap — see Single Agent.
- Multi-agent: a supervisor routes to billing, fraud and logistics specialists, each with their own tools and context. Justified only when one agent's tool set or context is measurably too large — see When Not to Use Multi-Agent.
The DSA mirror: why binary search instead of a hash map?
Algorithm choice teaches the identical discipline. Asked "find whether x is in the array", a beginner reaches for the most general tool; a senior engineer asks what the data admits. Sorted and static? Binary Search costs O(log n) with no extra memory. Many lookups, mutable set? A Hash Map gives O(1) average at O(n) memory. Neither is "better"; each is right for a shape of problem, and the more general one costs more than the narrower one whenever the narrower one suffices.
"Why an agent instead of a workflow?" is the same question in the same spirit. A workflow is binary search: it exploits structure you already know (the steps and their order) to be cheap, predictable and provable. An agent is the general tool: it handles inputs whose structure you cannot pre-state, at the cost of memory (context), time (iterations) and predictability. Reaching for the agent when the process is known is like hashing a sorted array of ten elements — it works and it is wrong.
- Known order of steps ↔ sorted data: exploit it with a workflow / binary search.
- Unknown, data-dependent path ↔ arbitrary keys: pay for generality with an agent / hash map, but only then.
- Both domains have the same anti-pattern: choosing the powerful tool to avoid understanding the problem.
- And the same test: state the invariant. If you can write the steps down, you have a workflow.
Tradeoffs across the ladder
Ratings are 1–5 where 1 is low complexity, latency and cost and 5 is best reliability and debuggability. They are typical values; a badly built workflow can be less reliable than a well-built agent, but the ordering holds far more often than not.
- Plain code — complexity 1, latency 1, cost 1, reliability 5, debuggability 5. Fails only on inputs it cannot parse.
- LLM call — complexity 1, latency 2, cost 2, reliability 3, debuggability 4. Probabilistic text; evaluate with a golden set.
- Structured output — complexity 2, latency 2, cost 2, reliability 4, debuggability 5. Schema validation catches most nonsense.
- Tool calling — complexity 2, latency 2, cost 2, reliability 4, debuggability 4. Side effects appear; validate arguments.
- RAG — complexity 3, latency 3, cost 3, reliability 3, debuggability 3. Retrieval quality dominates; needs separate evals.
- Workflow — complexity 3, latency 3, cost 3, reliability 5, debuggability 5. Persisted state and visible runs; highest reliability with a model in the loop.
- Agent — complexity 4, latency 4, cost 4, reliability 3, debuggability 2. Variable step count; requires budgets, traces, repetition detection.
- Multi-agent — complexity 5, latency 5, cost 5, reliability 2, debuggability 1. Token multiplication and tree-shaped traces; last resort.
How to decide in practice
Write the process down as steps. Every step you can name and order goes into code or a workflow. Every step where you wrote "figure out…" or "depending on…" is a candidate for a model call; ask whether structured output or a tool call closes it. Only steps where the *number* of sub-steps is unknowable get an agent loop, with a cap. Multi-agent enters only when a single agent has been measured to fail on context size or tool selection.
Then measure at the chosen rung with a golden set before climbing. Climbing costs roughly a 2–5x increase in latency and cost per rung past tool calling, and each rung adds a failure class (retrieval misses, stuck loops, supervisor bottlenecks) that needs its own eval — see Evaluating Agents: Testing Probabilistic Systems and Architecture Tradeoffs.
- Ask "what would break if I removed this rung?" — if the answer is "nothing measurable", remove it.
- Prefer a workflow with one agentic node over a single free-running agent for anything with side effects.
- Document the rung and why in the design; the next engineer will otherwise assume you needed the agent.
Key points
- The ladder: plain code → LLM call → structured output → tool calling → RAG → workflow → agent → multi-agent.
- Start at the lowest rung that could work; climb only when measurement says the current rung fails.
- Each rung past tool calling adds a failure class that needs its own evaluation.
- Workflow vs agent mirrors binary search vs hash map: exploit known structure when you have it, pay for generality only when you do not.
- If you can write the steps down, you have a workflow; the agent belongs only in the steps whose sub-step count is unknowable.
- Multi-agent is justified by a measured single-agent failure, not by the shape of the org chart.
When to use — and when not to
- At the start of every design, before any framework is chosen.
- When reviewing a proposal that starts at "agent" or "multi-agent" without measurement.
- When a system is too slow or expensive: descend the ladder for the steps that do not need their rung.
- As a maturity model — higher is not better, it is more expensive.
- To rank teams or products by how "agentic" they are.
- When the problem is not text- or judgment-shaped at all; then rung zero is the whole answer.
Failure modes
- Starting at agent because the demo was impressive, then bolting on step limits, replanning and evals to tame it.
- Using RAG when the needed facts fit in the prompt or a lookup table.
- Skipping structured output and parsing free text with regexes that break on the second prompt change.
- Choosing multi-agent to mirror team boundaries rather than a measured context or tool-selection limit.
- Never descending: a system that once needed an agent keeps it after the process became well understood.