Plain LLM vs RAG vs agent
“Given a new feature request, how do you decide between a plain LLM call, RAG, and an agent?”
What this tests
- Knowledge of the escalation ladder and what each rung solves
- Ability to derive the architecture from the problem, not from fashion
- Awareness of what each option costs to operate and test
- Whether they ask about the data and the failure cost before answering
Answers by level
Read the beginner answer first and notice what is missing.
Each rung solves a specific deficiency. A plain call is right when the model already has the knowledge and the task is a single transformation (classify, rewrite, extract). RAG is right when the answer depends on information the model does not have and that changes: your documents, your tickets, this week's prices. An agent is right when the sequence of actions is unknown ahead of time and depends on intermediate results. See Choosing the Right Abstraction and RAG Overview.
So I ask three questions. Does the answer require private or fresh data? If yes, retrieval of some form. Can I enumerate the steps? If yes, a workflow, even if it has several LLM calls. Does the task require deciding among actions based on results? Only then an agent. Many "agent" requests are actually a plain call plus structured output, or RAG with a good reranker.
I also weigh operating cost: a plain call is testable with fixtures; RAG adds an ingestion pipeline and retrieval evals; an agent adds loop limits, tool permissions, and tracing. Each rung roughly doubles the surface I have to maintain.
Green flags · Red flags
- Separates the knowledge axis (RAG) from the control-flow axis (agent)
- Asks whether the answer needs private or fresh data
- Asks whether the steps can be enumerated
- Mentions building a small eval set before deciding
- Includes "no LLM at all" as an option
- Considers operational surface area of each choice
- Maps complexity of the task directly to "use an agent"
- Treats RAG as a way to make the model smarter rather than better informed
- No questions asked about the data or failure cost
- Chooses by framework capability