Context Engineering
The context window is the only input the model has; assembling it deliberately from instructions, request, history, retrieved knowledge, tool results, memory and state is the real program you are writing.
The model sees exactly one thing
An LLM has no access to your database, your previous conversation, or your intent. Every call receives a single sequence of tokens — the context — and produces a continuation. Everything the agent "knows" at the moment of a call is either in the weights or in that sequence.
That reframes the job. "Prompt engineering" suggests wordsmithing one string. Context engineering is the systems discipline of deciding, per call, which information enters the window, in what form, in what order, and at what size. A weak model with excellent context routinely beats a strong model with a sloppy one.
The rest of this module treats context as an artifact you build with code: selection (Context Selection & Compression), ordering (Context Ordering & Lost in the Middle), budgets (Token Budgets), and per-step assembly (Dynamic Context Assembly).
The seven sources
In practice, the context of a production agent call is assembled from a small, stable set of sources. Naming them lets you budget and debug each one separately.
- System instructions — role, rules, output format, tool descriptions. Stable across a session; ideal for prefix caching.
- User request — the current turn, verbatim. Never paraphrase it away.
- Conversation history — prior turns, usually summarised beyond a recent window.
- Retrieved knowledge — chunks from RAG (RAG Overview), with source ids for Citations.
- Tool results — the raw or truncated output of the last tool calls (Tool Calling Basics).
- Memory — facts about this user or task persisted across sessions (Memory Types).
- Current state — the plan, step counter, scratchpad, and any structured state the orchestrator tracks (Workflow State Graph).
Context is the program
A conventional program is code plus inputs. An agent call is weights plus context, and the weights are fixed. So the context is the only thing you actually author, and it deserves the same discipline as code: version it, test it, diff it between runs, and never let it grow by accident.
A useful mental test: if a run went wrong, could you reproduce it from the exact context that was sent? If your tracing (Tracing Agents) does not capture the final assembled context per call, you cannot debug the system — you can only guess.
1def build_context(turn, state, memory, retriever, tool_log, budget=12_000):2 parts = [3 ("system", SYSTEM_PROMPT), # stable prefix, cacheable4 ("memory", render_memory(memory.for_user(turn.user_id))),5 ("history", summarize(turn.history, keep_last=6)),6 ("knowledge", render_chunks(retriever.search(turn.text, k=5))),7 ("tools", truncate_each(tool_log.recent(3), max_tokens=800)),8 ("state", render_state(state)),9 ("request", turn.text), # always last, verbatim10 ]11 return fit_to_budget(parts, budget) # drops/shrinks by priority, never the requestWhat good looks like
A well-engineered context is small, labelled, and boring. Each section has a header the model can anchor on (## Retrieved documents), untrusted content is fenced and marked as data (Indirect Prompt Injection), and the user request sits where the model attends most.
It is also measured. Track tokens per section per call, and watch the numbers over a release: a context that grows from 4k to 30k tokens without a corresponding gain in task success is a bug — the context-overflow-degradation challenge makes it painfully concrete.
- Label every section; the model should never have to guess whether text is an instruction, a document, or a tool result.
- Keep the request verbatim and the instructions explicit; compress everything else.
- Log the final context; it is the only thing that explains the output.
Key points
- The model only sees the context window; assembling it is the program.
- Seven sources: system instructions, user request, history, retrieved knowledge, tool results, memory, current state.
- Budget and debug each source separately — they fail in different ways.
- Label sections, keep the request verbatim, mark untrusted content as data.
- Log the exact assembled context per call or you cannot reproduce failures.
- Bigger context is not better context; measure task success against tokens.
Context builder
- Over budget: the provider truncates or rejects — usually the newest, most relevant material is what gets lost.
- Retrieved evidence sits in the middle of a long context. Models attend less to the middle; move instructions and evidence toward the edges.
- Raw tool results are 3,100 tokens for ~3 useful fields. Extract facts instead of dumping JSON.
- 18 turns of history mostly restate the problem. Summarize to a few sentences.
## System instructions
You are a support agent for Acme. Answer from sources; never promise refunds above policy…
## User request
I was charged twice for order #4821. Can you refund the duplicate?
## Conversation history (18 turns)
user: hi… assistant: hello… user: my order… (16 more turns)
## Retrieved knowledge (5 chunks)
[1] Duplicate charges under 100 EUR are refunded automatically… [2] Shipping… [3]…
## Tool results (raw)
{"order": {"id": 4821, "line_items": [... 40 fields ...], "charges": [49.0, 49.0]}}
## Long-term memory (12 facts)
Prefers email. Lives in Berlin. Asked about shipping in March. Complained about packaging…
## Current state
{ verified: "duplicate charge", amount: 49, policy_limit: 100 }When to use — and when not to
- Any LLM call with more than one input source — which is every agent call.
- Diagnosing wrong answers: start from the assembled context, not the model.
- Before adding memory, RAG, or more tools — decide where their output lives in the window.
- A single-shot call with one short input needs a prompt, not an assembler.
- Do not build an elaborate assembler before you have traces showing which sections matter.
- Do not use context to patch a problem deterministic code should solve (e.g. validating an id).
Failure modes
- Unbounded history growth until every call hits the limit and quality collapses.
- Tool results dumped raw (a 40k-token JSON blob) crowd out the actual instructions.
- Unlabelled sections: the model treats a retrieved document as an instruction.
- No trace of the assembled context, so "it answered wrong" is unreproducible.
- Instructions buried in the middle of a long window and silently ignored.