Case Study: AI Agent Execution API
A platform API for running AI agents: submit a task, watch the agent think and use tools in real time, cancel it, and account for every token it burned.
An agent run violates every default assumption of request/response: it lasts seconds to hours, produces value continuously (tokens, tool calls) rather than at the end, costs real money per second, and can fail in a dozen partial ways — tool errors the agent recovers from, budget exhaustion mid-thought, a model provider dying between steps. The contract that survives this composes patterns you've seen separately: the async job pattern for lifecycle (The Async Job Pattern), SSE with resumable event ids for streaming (Server-Sent Events), mandatory idempotency because a duplicate run costs dollars (Idempotency Keys: The Mechanism), and cancellation as a state transition with honest cost semantics. The novel part is the event log as the spine: the run's truth is an append-only, replayable sequence of events, and *everything else* — streaming, polling, billing, audit, resume — is a view over it.
Consumers
Fire agent tasks from their own request handlers: submit fast, get an id, deliver results to *their* users later. Blind-retry safety is existential — a duplicate run is duplicate spend.
Token-by-token streaming, live tool-call visibility, a cancel button that visibly works, and seamless resume when the laptop lid closes mid-run.
Many concurrent runs, webhook completion (no polling fleets), per-run cost attribution to the cent, and full transcripts for audit and eval pipelines.
Requirements
- • Submit a run (agent config, input, tool allowlist, budget caps) and return immediately — execution is always asynchronous, and a network-level retry of the submit must never start a second (billed) run.
- • Observe a run in real time: model tokens, tool invocations and results, state changes — and catch up losslessly after a disconnect.
- • Cancel a running agent; cancellation is prompt, cost stops accruing quickly, and partial output remains readable.
- • Tool failures are in-run events the agent may recover from — distinguished cleanly from run-level failures.
- • Every run reports token usage and cost, live during execution and final at the end; budget caps are enforced mid-run.
- • Completed runs are retrievable in full (transcript, tool calls, cost) for 30 days.
Resources
The central resource: config, input, budget, and a state machine — `queued` → `running` → `succeeded` | `failed` | `canceled` | `expired`, with `cancelling` as a real intermediate state because distributed cancellation takes time and hiding that would make the contract lie.
The append-only spine: `run.state_changed`, `message.delta`, `tool_call.started`, `tool_call.completed`, `usage.updated`, each with a per-run monotonic `sequence`. The event log *is* the run; stream, poll, webhook, and audit are all projections of it — one truth, four delivery mechanisms.
Addressable per-call records (name, arguments, result or error, duration, attribution). Promoted from log lines to sub-resources because consumers query them directly: "which runs called `send_email`?" is a compliance question, not a debugging one.
Structured cost accounting — tokens by type, tool invocations, compute time, cost in minor currency units — attached to the run and updated via events. Modeled explicitly because money is the axis every enterprise consumer reconciles on.
Versioned, named agent configs. Runs reference `agent@version`, pinning behavior: the same submit next week runs the same agent, and behavior changes ship as versions, not silent mutations ([[versioning]] applied to prompts).
Operations
| Operation | Purpose | Design notes |
|---|---|---|
| POST /agent-runs | Submit a run. | Returns `202` with {id, status: "queued"} in under 100ms, unconditionally — even when a worker is free — because *one* response shape means clients can't skip building the async path. Idempotency-Key required: replay returns the original run; same key with different body is 409. This is the payment-API discipline, because a run *is* a payment (Idempotency Keys: The Mechanism). |
| GET /agent-runs/{id} | Current state, live usage, and — when finished — the result. | The recovery anchor: after any ambiguity (submit timeout, stream drop, webhook missed), polling this endpoint resolves the truth. Read-after-write documented, Retry-After hints while non-terminal. |
| GET /agent-runs/{id}/events | The event stream — SSE. | SSE over WebSocket deliberately: the flow is one-directional, works through proxies, and gets resume *from the protocol* — Last-Event-ID maps onto the event sequence, so reconnect replays exactly the gap (Server-Sent Events). The same endpoint without the Accept: text/event-stream header returns the log as a paginated collection: stream and history are one contract, not two. |
| POST /agent-runs/{id}/cancel | Request cancellation. | Returns `202` with status: "cancelling" — the agent may be mid-tool-call on a remote worker, and pretending cancel is instant would be the contract's first lie. The run settles to canceled (with partial output and final cost) or, if completion won the race, succeeded. Idempotent: cancelling twice, or cancelling a terminal run, returns the current state with 200, because the caller's goal is a terminal state, not a transition. |
| GET /agent-runs | List runs: by status, agent, time window, tag. | Cursor-paginated on (created_at, id); status=running is the operational dashboard query, tags carry the consumer's own correlation ids. |
| GET /agent-runs/{id}/tool-calls | Structured tool-call records for one run. | The audit view — arguments and results as data, not prose, with per-call error objects for the calls that failed and were retried or absorbed by the agent. |
| POST /webhook-endpoints | Register for terminal-state webhooks (`run.succeeded`, `run.failed`, `run.canceled`). | For fleets, polling doesn't scale; webhooks carry event_id + run id and consumers dedupe — and the docs repeat the platform rule: the webhook is a doorbell, GET /agent-runs/{id} is the truth (Consumer-Side Idempotency). |
| GET /agent-runs/{id}/usage | Cost breakdown: tokens by model and type, tool time, total cost. | Live during the run (the budget-watching view) and immutable once terminal — the reconciliation artifact finance actually ingests. |
Error contract
| Code | Status | When | Retryable |
|---|---|---|---|
| VALIDATION_FAILED | 400 | Bad submit: unknown agent version, malformed input, tool allowlist naming tools the agent doesn't have. | no |
| IDEMPOTENCY_CONFLICT | 409 | Idempotency key reused with a different body — the client's key generation is broken, and honoring either body silently would hide it. | no |
| INVALID_STATE | 409 | A transition the state machine forbids from the current state. Body carries `current_status`. Note what is *not* here: cancelling a terminal run is a `200` no-op, not this error. | no |
| BUDGET_EXCEEDED | 422 | At submit: the run's configured cap exceeds the account's remaining budget. Mid-run exhaustion is *not* an HTTP error — it terminates the run as `failed` with `reason: "budget_exhausted"`, partial output retained and billed. | no |
| CONCURRENCY_LIMIT | 429 | Too many simultaneously running runs for the account. `Retry-After` reflects expected queue drain — distinct from request-rate limiting, which has its own headers ([[quotas-vs-rate-limits]]). | after delay |
| MODEL_UNAVAILABLE | 503 | At submit: no capacity to *accept* work. Mid-run provider failures are absorbed by internal retries or surface as run-level `failed` events — the HTTP layer only ever reports on the operations the caller directly invoked. | after delay |
Decision log
Decision → reason → alternative → trade-off. The alternative is part of the record.
Last-Event-ID → sequence), billing is a fold over usage events, and audit is the log itself — no reconciliation between parallel truths (Streaming APIs: Partial Data as a Contract).cancelling + terminal settle + cost-until-stop makes the observable behavior match reality (Resources Have State Machines).204.succeeded (completion won the race) — one more case, but the honest one, and the events show exactly what happened.on_tool_error: "fail_run") — policy as configuration rather than a contract default that lobotomizes the agent.usage.updated events power budget kills and dashboards; the immutable terminal usage object is what invoices reconcile against — the payment API's reconciliation posture, applied to compute.How it evolves
- • Human-in-the-loop approval: a
waiting_for_inputstate plusPOST /agent-runs/{id}/input. Old clients were told from V1 to treat unknown non-terminal states as "still working, keep observing" — they see a paused run, not a crash; only clients that *offer* approval UIs need the new endpoint (Enum Evolution: The New Value That Broke Old Clients). - • Multi-agent runs: child runs carry a
parent_run_id, and the parent's event log gainschild_run.*event types. List-by-parent is additive; consumers ignoring unknown event types (the documented rule) keep working while orchestration-aware UIs light up. - • Structured output contracts: submits gain an optional
output_schema; conforming runs emitoutput.validatedevents and a typedresult. Schema violations become a newfailedreason — absorbed by clients because failure reasons were an open set from day one. - • Priority tiers and scheduling windows land as submit-time fields (
priority,not_before) plus queue-position data inGET /agent-runs/{id}— pure addition, because thequeuedstate existed from V1 even when queues were usually empty (Backward Compatibility: The Real Rules).