Agentic AI Engineering Interview Guide

Only have 5 minutes? Ask these 3 questions

Each card links to the full question with its green and red flags.

Q1 · Tier 1

“What agent or LLM-powered system have you shipped to production?”

Tests whether they have moved past tutorials.

Q5 · Tier 2

“How do you manage costs when your agents make hundreds of LLM calls per task?”

The most honest signal of production maturity.

Q7 · Tier 3

“Design a multi-agent system for [a domain you care about]. How do agents coordinate, and what happens when one fails?”

Reveals architectural depth and failure thinking.

Scoring framework

Use this to calibrate responses before you start. Most candidates land at Intermediate. Frontier is rare and worth paying a premium for.

Level	Description	Typical signals
Beginner	Has built demos	LangChain quickstart, single API wrapper, no eval pipeline, ChatGPT for everything
Intermediate	Production practitioner	Working agent in prod, error handling, basic monitoring, one framework, knows eval exists
Advanced	System owner	Multi-agent architecture, cost optimization, eval in CI/CD, security model, observability
Frontier	Platform builder	Agent infra for the org, provider abstraction, team-wide standards, cross-service orchestration

Beginner→Intermediate→Advanced→Frontier

Tier 1: Basics

Any interviewer can ask. No technical background needed. A recruiter or hiring manager can ask these and evaluate the responses with the flags below.

Q15-minute screen

“What agent or LLM-powered system have you shipped to production?”

Tests whether they have moved past tutorials.

Green flags

Names a specific system with real users. Can describe the architecture: which models, what tools the agent uses, how it handles failures.
Talks about what went wrong. Production experience always comes with war stories.
Explains why they chose their approach over alternatives they considered.
Distinguishes between a demo and a production system without being prompted.

Red flags

Only mentions personal projects or hackathon demos with no real users.
Describes a thin wrapper around a chat API as an "agent."
Can't explain how the system handles errors, bad outputs, or unexpected inputs.
No war stories. Everything worked perfectly the first time. (It didn't.)

Q2

“How do you know when your agent gives a wrong answer?”

Tests whether evaluation is systematic or anecdotal.

Green flags

Has a systematic evaluation approach. Mentions specific tools (Braintrust, Langfuse, RAGAS) or a custom pipeline they've built.
Builds eval datasets from real production traffic, not only hand-written test cases.
Knows the difference between testing retrieval quality and testing generation quality.
Mentions LLM-as-judge with explicit scoring rubrics ("score 0 if factually wrong, 1 if correct but vague, 2 if correct and concise").

Red flags

"We review the outputs manually." No systematic approach.
Can't describe what metrics they track or what a regression looks like.
Conflates "the model said it was confident" with actual evaluation.
Has never caught a regression after a prompt change or model update.

Q3

“When would you build an agent vs. just write regular code?”

Tests whether they treat agents as a tool with real costs, not a default.

Green flags

Has clear criteria. Agents earn their place when the task requires dynamic reasoning, tool selection, or adapting to highly variable inputs. Deterministic logic stays deterministic.
Names a specific case where they chose NOT to use an agent and explains why.
Can articulate the cost, latency, and reliability tradeoffs of adding an LLM to any workflow.
Treats agents as a tool with real costs, not a default approach.

Red flags

Thinks everything should be an agent. No concept of when the overhead isn't worth it.
Can't name the downsides: cost per request, latency, non-determinism, debugging complexity.
Has never decided against using an agent for a task.
Has one production experience from a company that went from $0/month to $47K/month in orchestration costs. (This actually happened. See Iterathon's 2026 multi-agent economics report.)

Tier 2: Intermediate

Tech-aware interviewer. These questions assume basic technical context. They probe production maturity and operational thinking: the difference between someone who built a demo and someone who keeps an agent running at scale.

Q4

“Walk me through how you test and evaluate an agent before deploying it.”

Tests whether prompts are treated with the same rigor as code.

Green flags

Has a repeatable eval pipeline: golden dataset, automated scoring, baseline comparison.
Tests agent trajectories (the full sequence of tool calls and reasoning), not just final outputs.
Runs regression tests before prompt changes go live. Treats prompts with the same rigor as code.
Can describe how they handle the fact that LLM outputs are non-deterministic. (Multiple runs, statistical thresholds, acceptable variance bands.)
Mentions CI/CD integration for eval. Prompt changes don't ship without passing the suite.

Red flags

"I test it manually before deploying." No automation.
Only evaluates the final answer, ignoring the reasoning path that got there.
No concept of regression testing for prompts. Ships changes and hopes.
Treats prompts as throwaway strings, not versioned assets with change history.

Q55-minute screen

“How do you manage costs when your agents make hundreds of LLM calls per task?”

The most honest signal of production maturity.

Green flags

Has specific tactics: model routing (cheap model for simple steps, expensive model for hard ones), prompt caching for shared prefixes, token budgets per task.
Can estimate cost per request or cost per user for a system they've built. Specific numbers, not hand-waving.
Knows when a smaller model is good enough. Not every step needs the largest model.
Monitors and alerts on spend. Catches cost spikes before the monthly bill arrives.

Red flags

"We just use GPT-4 for everything." No concept of model routing.
Can't estimate what a system costs to run. No awareness of token economics.
Has never set a cost budget, alert, or per-task token limit.
Treats LLM API calls like they're free. They're not.

Q6

“Your agent needs to call external APIs and tools. How do you build and secure that?”

Tests tool design end-to-end and whether a permission model exists.

Green flags

Describes tool integration end-to-end: schema design, input validation, error handling, retry logic, timeout budgets.
Thinks about permissions. Each tool scoped to least privilege. Write operations require explicit approval or deny-by-default.
Knows MCP (Model Context Protocol) and can explain when to use it vs. native function calling.
Validates tool call parameters before execution. Doesn't trust the model to always call tools correctly.
Has dealt with rate limits, auth token rotation, or partial failures in production.

Red flags

Only used pre-built tools from a marketplace. Never designed a tool schema.
No security model. The agent can do whatever the API key allows.
Can't explain how function calling works under the hood: what the model sees vs. what your code handles.
Passes user input directly into tool parameters without validation. (This is an injection vector.)

Tier 3: Advanced

Technical interviewer. These questions need someone who can evaluate the depth of the answers. They test architecture, debugging skills, and security thinking.

Q75-minute screen

“Design a multi-agent system for [a domain you care about]. How do agents coordinate, and what happens when one fails?”

Reveals architectural depth and failure thinking.

Green flags

Decomposes the problem into agents with clear responsibilities and boundaries. Not "just spawn more agents."
Addresses coordination: how agents share state, resolve conflicts, avoid infinite delegation loops.
Has a failure strategy. What happens when one agent returns garbage? How do you prevent cascading failures across the system?
Knows when multi-agent is overkill and a single agent would be simpler, cheaper, and more reliable.
Mentions observability: how you trace a single user request across multiple agents.

Red flags

No coordination model. Just throws agents at the problem and assumes they'll figure it out.
Happy-path only. No failure handling, no fallback, no circuit breakers.
Can't explain the cost and latency overhead of multi-agent vs. single-agent approaches.
Assumes multi-agent always beats single-agent. (It often doesn't.)

Q8

“Your RAG pipeline retrieves relevant documents but the agent still gives wrong answers. Debug this.”

Tests whether they can separate retrieval, context assembly and generation.

Green flags

Separates the problem into layers: retrieval quality, context assembly, generation faithfulness. Debugs each independently.
Checks chunk boundaries. The relevant information might be split across chunks, truncated, or sitting right at the edge of two segments.
Looks at context window utilization. Too many retrieved chunks can dilute the signal and push the model to ignore key passages.
Tests with different query formulations to determine if the issue is query-dependent or systemic.
Measures faithfulness directly: does the generated answer actually follow from the retrieved context, or is the model going off-script?

Red flags

"Just add more documents" or "increase top-k" without analyzing the actual failure.
Can't distinguish between a retrieval problem and a generation problem. Treats the whole pipeline as a black box.
Doesn't know what chunking strategy they're using or how it affects results.
No concept of faithfulness metrics. Can't explain how to check whether the answer is grounded in the retrieved context.

Q9

“How do you defend an agent that processes user-uploaded content against prompt injection?”

Tests whether security is architectural or a line in the system prompt.

Green flags

Knows the difference between direct injection (user tries to override system instructions) and indirect injection (malicious instructions embedded in documents the agent retrieves).
Has concrete defenses: input/output separation, treating retrieved content as untrusted data, canary tokens, allowlisted tool parameters.
Designs permissions in layers. Even if the prompt is compromised, the agent can't call tools it shouldn't have access to.
Mentions red-teaming before deployment. Has actually tested attack vectors against their own systems.

Red flags

"We tell the model to ignore bad instructions." That's not a security model.
Doesn't know what indirect prompt injection is. Only thinks about direct user input attacks.
No permission boundaries on agent actions. If the prompt is compromised, everything is compromised.
Treats security as something to add after the product launches.

What level should you expect?

Calibrate expectations before the interview. Frontier candidates are rare. Don't pass on an Intermediate because you wanted Advanced, and don't hire an Advanced person for a role that needs Frontier.

Role	Minimum level	Ideal level	Key signal
Junior	Beginner	Intermediate	Has built a working agent, learning fast, asks good questions about production concerns
Mid	Intermediate	Advanced	Production agents with monitoring, eval coverage, cost awareness
Senior / Lead	Advanced	Frontier	Multi-agent architecture, owns cost profile and failure modes end-to-end
Staff+	Frontier	Frontier	Agent platform infrastructure, org-wide standards, cross-team tooling

Beyond the questions: practical assessments

Three formats that reveal things verbal answers can't.

45 min

Agent debugging session

Provide a broken agent system: a RAG pipeline returning wrong answers, tool calls timing out, cost budget exceeded. Watch how they trace the issue through system layers. You're evaluating diagnostic process, not the fix itself.

4–6 hours, time-boxed

Take-home: build a tool-calling agent

Give them a spec and a couple of tool definitions. Evaluate: error handling, eval coverage, cost awareness, and documentation of their design decisions. Review in a follow-up session where they walk through their tradeoffs.

30 min

System design sketch

Present a product requirement that needs agent orchestration. No code. Evaluate their decomposition, coordination model, failure handling, and whether they reach for multi-agent when single-agent would do.

Skip LeetCode. It tests algorithmic puzzle-solving. You need someone who can debug a hallucinating agent at 2am.

Red flag quick reference

Dealbreakers

Claims production experience but can't describe a single failure mode they've encountered
No evaluation strategy beyond "I tested it manually and it looked right"
No concept of agent permissions, tool scoping, or any security model

Serious concerns

Only used one LLM provider and one framework. No basis for comparison.
No awareness of token costs. Can't estimate what their system costs to run.
Treats prompts as throwaway text rather than versioned, tested, reviewed code

Minor / contextual

Unfamiliar with a specific framework by name (LangGraph, CrewAI, Mastra). Frameworks change fast. Fundamentals transfer.
Hasn't built multi-agent systems. Single-agent is the right choice for most problems.
Uses different tools than your stack. Tooling transfers easily. System thinking doesn't.

A note on fairness

Agentic AI engineering as a distinct discipline barely existed before 2025. The frameworks are new. The patterns are still forming. Nobody has five years of experience building production agents, because five years ago the models couldn't use tools.

Focus on reasoning quality and system thinking over specific framework names. Ask "how would you approach this even if you haven't done it before?" to open questions to candidates who've worked on adjacent problems. A strong software engineer who understands distributed systems and can reason about non-deterministic behavior will often outperform someone with flashy agent demos but no production discipline.