What Is Agentic AI?
A system is agentic when a model decides which action to take next based on what it observed, inside a loop with a termination condition — everything else is a program with an LLM in it.
Five stages, one axis: who decides the control flow?
The word "agent" is used for everything from a chat box to a fleet of autonomous processes. The useful way to sort these systems is a single question: who chooses the next step — the programmer at design time, or the model at run time?
Traditional software has fully static control flow: every branch is an if written by a human. An LLM application inserts a model call into that fixed flow (classify a ticket, draft a reply). A RAG application adds a retrieval step before the call so the answer is grounded in your data. An AI agent hands the model the choice of which tool to call and whether to keep going. An agentic system composes several of these loops with routing, state, approvals and monitoring around them.
Each stage moves decision-making from code into the model. That buys flexibility and costs predictability, latency, money and debuggability. Choosing the stage is the core engineering decision of this domain — see Choosing the Right Abstraction.
Stage by stage
The same product need — "answer customer questions about invoices" — looks different at each stage. Notice that the input and output types barely change; what changes is where the decisions live.
- Traditional software: a search form over an invoice table. Every path is enumerable and testable with unit tests. Failure is a bug, not a probability.
- LLM application:
f(question) = llm(prompt + question). One call, no tools, no loop. The model can only use what is in its weights and your prompt, so it will confidently invent invoice numbers. - RAG application:
f(question) = llm(prompt + top_k(retrieve(question)) + question). Still one call and a fixed pipeline, but now grounded in real documents. Retrieval quality becomes the dominant failure source — see RAG Overview. - AI agent: the model is given
search_invoices,get_customerandissue_refundas tools and asked to resolve the request. It decides which to call, reads the result, and decides again. The number of steps is not known in advance. - Agentic system: a router sends billing questions to the invoice agent and account questions to another; refunds above 500 wait for a human; every step is traced; a budget kills runs after 20 iterations. See Supervisor Architecture and Human-in-the-Loop Overview.
A precise definition of "agentic"
A system is agentic when, at run time, the model decides which action to take next based on observations, and that decision happens inside a loop that continues until a termination condition is met. All four parts matter: model-made decisions, observation-conditioned decisions, a loop, and termination.
Drop any one and it stops being an agent. A pipeline where the model is called three times in a fixed order is a workflow, however clever the prompts. A single tool call with no follow-up is structured output with side effects. A loop with no termination condition is an incident waiting to happen — see Budgets, Limits and Termination.
This definition is deliberately narrow. It lets you say "this does not need to be an agent" with a straight face, which is the most valuable sentence in the field.
- Decision: the model outputs a choice among actions (call tool X with args Y, ask the user, finish), not just text.
- Observation: the choice is conditioned on prior results — the tool output from step 3 changes what happens in step 4.
- Loop: steps repeat; the count is data-dependent, not fixed at design time.
- Termination: an explicit stop — goal reached, step limit, token budget, or an unrecoverable error.
What this definition rules in and out
Testing candidates against the definition sharpens vocabulary quickly. A "coding agent" that reads files, runs tests, reads the failures and edits again is agentic: observation drives the next action and the loop stops when tests pass or a step cap is hit. A "summarization agent" that always does chunk → summarize → merge is a workflow with a marketing name.
Autonomy is a dial, not a binary. Many production systems are workflows with one agentic step inside them, e.g. a fixed ingestion pipeline whose "resolve ambiguous entity" step is a small tool-using loop. That is often the best design: agentic only where the decision is genuinely unknowable upfront.
- Chat with a system prompt: not agentic (no actions, no loop).
- Classify → route → template reply: not agentic (fixed graph; the model fills slots).
- Model calls
search, reads results, decides to callsearchagain with a refined query, then answers: agentic. - Cron job that calls an LLM once per row: not agentic, and correctly so.
Key points
- The progression traditional → LLM app → RAG → agent → agentic system is one axis: how much control flow the model owns at run time.
- Agentic = model decides the next action from observations, in a loop, with a termination condition. All four parts are required.
- A fixed sequence of model calls is a workflow, not an agent, regardless of how the prompts are phrased.
- Every step up the progression trades predictability, latency, cost and debuggability for flexibility.
- Most good systems are mostly deterministic with agentic behaviour confined to the steps that truly need it.
From software to agentic systems
Deterministic instructions: the same input always produces the same output.
The programmer decides every branch in advance.
Validate a form, compute an invoice, route an HTTP request.
Whenever the rules can be written down. This is most software — and the correct default.
When to use — and when not to
- The number and order of steps depends on intermediate results you cannot enumerate at design time.
- The task requires reading results and adapting (debugging, research, multi-step data gathering).
- A human doing the task would loop: try, look, adjust, try again.
- The steps are known in advance — write a workflow and call the model inside it.
- A single model call with structured output already produces the answer.
- Correctness must be provable or fully testable; probabilistic control flow makes that impossible.
- Latency or cost budgets are tight and iteration counts are unbounded.
Failure modes
- Calling a fixed pipeline an "agent" and then adding agent-style safeguards (step limits, replanning) it does not need.
- Building an agent where the loop never terminates because "goal complete" is judged by the same model that keeps finding more to do.
- Assuming that moving from RAG to agent fixes wrong answers when the real problem was retrieval quality.
- Shipping an agent without traces, so no one can say why step 7 called the wrong tool.
Tradeoffs
Ratings describe the "AI agent" stage; each stage to the left is lower complexity/cost and higher reliability/debuggability.