Home / Chapter 5 · Agents
Last edited · 9 min read
The agent loop
An agent is a model in a loop: it gets a goal and tools, picks a call, your code runs it and sends back the result, until the model answers without a call. It differs from a workflow in one thing: the model, not your code, chooses the next step.
In plain wordsA workflow is a checklist for an intern: do this, then that, and if something doesn’t fit, come to me. An agent is an assistant who gets a goal, a phone and a calendar, and works out the next steps alone. It copes with surprises, but you don’t know in advance how long it will take or what it will do along the way.
Choose workflow or agent and what is in the calendar on Friday, then press ▶. Watch who picks the next step and how the context grows with every turn
The bar shows what the last model call is made of (scale: 3,500 tokens). Token counts are illustrative, not measured. The calendar, tools and model responses are made up to show the flow.
The loop in code
- The loop is a dozen or so lines of ordinary code. Send the model the context and the tool definitions. If the response contains calls, append the model’s response to the context unchanged, run the calls, append their results and send the whole thing again. If it doesn’t, you are done. The model runs nothing; it only writes what it wants to call (see “Tools (function calling)”).
- In the Anthropic API such a response has
stop_reason: "tool_use"andtool_useblocks with an ID. The results go back in the next message, after the assistant’s turn, astool_resultblocks with the sametool_use_id, and you mark an error withis_error: true. In the OpenAI Responses API these arefunction_callandfunction_call_outputitems linked bycall_id. Several calls from one response are run in parallel, and all the results are sent back together. - The model also thinks between calls (interleaved thinking; in Claude Opus 5.5 thinking can’t be turned off, as of September 2026). The Anthropic API requires the thinking blocks of a tool-use turn to come back complete and unmodified, and rejects edited ones with a 400. In the Responses API you send reasoning items back with the results, or chain requests with
previous_response_id(see “Reasoning models”). - A response without a call is only the model’s judgement that it is done. Code adds hard stop conditions: a turn limit, a token and cost budget, a time limit, detection of repeated calls. It also handles the other stop reasons, none of which ends the task: a response cut off by the token limit (
max_tokens) or the context window (model_context_window_exceeded) is an error to handle,pause_turn(a provider-side tool loop hit its cap) means sending the response back to continue, andrefusalneeds its own path.
Workflow or agent
- In a workflow the programmer writes the steps, and the model does narrow jobs inside them, such as extracting data from a request or writing a message. In an agent, code provides the goal and the tools, and the model picks the steps. A workflow is cheaper, faster, repeatable and easy to test, but it handles only what someone anticipated. An agent works around surprises and pays in tokens, time and predictability. The order to try: a single model call, then a workflow, and an agent only when the steps cannot be listed up front.
- Patterns from Anthropic’s “Building effective agents” (2024): prompt chaining with checks in code between the calls, routing (classify, then a separate path or a cheaper model), parallelisation (independent parts at once, or several attempts and a vote), orchestrator–workers (a model splits the task into subtasks you can’t know in advance) and evaluator–optimizer (one model writes, another grades, until the result passes). In production, a workflow with one agentic step in the middle often wins.
- Other names for the same ideas. ReAct (Yao et al., 2022) is this loop: the thought is the model’s text or thinking, the action a tool call, the observation its result. Andrew Ng’s four agentic patterns (2024) appear in this guide as the evaluator–optimizer (reflection), “Tools (function calling)” (tool use), the task list in long tasks (planning) and “Multiple agents” (multi-agent collaboration).
Failures in long tasks
- Return a tool error to the model as an ordinary result, with a hint: not “error 409” but “conflict with Quarterly review, check free slots”. The model then changes its approach, as in the conflict example. An exception that kills the loop wastes all the work done so far. Errors the model can’t fix, such as missing permissions or an exhausted budget, are handled by code.
- In a long history the agent loses sight of its goal. What helps: a plan, i.e. a task list the agent ticks off and rewrites at the end of the context, notes in files, and history compaction (see “Context engineering and memory”).
- Agents get stuck: they retry the same call, bounce between two steps, announce a success that never happened. The answer is limits and repeat detection in code, not requests in the prompt. An interrupted agent leaves the world half-changed, e.g. the meeting moved but the message not sent. So state-changing tools are idempotent, you save the loop state after every call so that work can resume after a failure, and a human gets a report of what has already happened.
Cost, permissions and evaluation
- Every turn sends the whole context again, so total input tokens grow roughly with the square of the number of turns. Latencies add up: every turn is a full model call plus the tool’s run time. The arithmetic is in “The context window and agents”. As long as the history is only appended to, everything but the newest part of each turn is a stable prefix, so with “Prompt caching” it is read from the cache at about a tenth of the input price; the curve stays quadratic, only cheaper.
- An agent acts on your behalf, so it gets the least privilege it needs. Irreversible or outward-facing actions (sending, paying, deleting) are approved by a human, and code enforces that, not the prompt: in the example, code pauses the loop before the message is sent. Emails, web pages and tool results can contain instructions (see “Prompt injection”).
- Judge an agent by the end state, not by its summary: is the meeting really on Friday, and did the message go out? A trace, i.e. a record of every turn with its tools, arguments, results and tokens, shows where the agent goes astray and what it costs. The same case passes one time and fails the next, so you compute pass^k: does the agent complete the task on every one of k attempts (see “Evals”).
- A task that splits into independent parts can be handed out by a lead agent to sub-agents with separate contexts, at the cost of many times more tokens. See “Multiple agents”. How a coding agent puts these pieces together (permissions, sandbox, plan, compaction, subagents) is covered in “The coding-agent harness”.
Check yourself
When would you build an agent instead of a workflow, and how do you secure its loop in production?
An agent is a model in a loop: it gets a goal and tools, picks a call, your code executes it and appends the call and its result, until the model answers without a call. In a workflow you write the steps and the model does narrow jobs inside them, so it is cheaper, faster and repeatable. An agent pays off only when the steps cannot be listed up front. Then code enforces turn, budget and time limits, grants least privilege, holds irreversible actions for human approval and checkpoints state after every step. Quality is measured by the end state, with each task run several times.
Po polsku
Agent to model w pętli: dostaje cel i narzędzia, wybiera wywołanie, twój kod je wykonuje i dopisuje wywołanie z wynikiem, aż model odpowie bez wywołania. W workflow kroki zapisujesz ty, a model robi w nich wąskie zadania, więc jest taniej, szybciej i powtarzalnie. Agent opłaca się dopiero, gdy kroków nie da się wypisać z góry. Wtedy kod pilnuje limitu tur, budżetu i czasu, daje najmniejsze uprawnienia, wstrzymuje akcje nieodwracalne do zgody człowieka i zapisuje stan po każdym kroku. Jakość mierzy się stanem końcowym, a każde zadanie puszcza kilka razy.
Follow-up questions (6)
- How does an agent know it is done?
- It doesn’t know, it judges: it answers without calling a tool. Sometimes it announces a success that never happened. That is why code has hard limits (turns, tokens, time), and you check the result with an end-state test, not the model’s summary.
- The agent keeps calling the same tool. What do you do?
- As a quick fix, code detects a repeated call with the same arguments, adds a note for the model, and breaks the loop on the next repeat. The cause is usually in the tool: an unclear error message or a result that shows no progress. The run goes into the eval set as a new case.
- The agent stopped halfway through a task. How do you design for that?
- You assume it will happen. State-changing tools accept an idempotency key, and you save the loop state after every call, so work can resume without repeating side effects. A human gets a list of what has already been changed and, where possible, an action that undoes it.
- How do you judge whether an agent is ready for production?
- A set of tasks from real cases, each run several times and graded on the end state, plus cost and time per task. You compute pass^k, because users expect it to work every time: for a task the agent completes in 90% of attempts, with independent attempts, three successes in a row happen in about 73% of cases. Across a suite you compute pass^k per task and average it.
- How do you classify an agent’s failures?
- Following Chip Huyen (2025), into three groups. Planning: a wrong or non-existent tool, bad arguments, a missed goal, a false claim of success. Tool: the call is right, but the tool returned a wrong result, so each tool is tested on its own. Efficiency: the task is done, but with too many steps, too much cost or too much time against a baseline. Traces tagged this way show whether to fix the prompt and tool descriptions, the tool itself or the limits.
- Do you need a framework to build an agent?
- No. The loop is a dozen or so lines on a plain API, and that is the place to start. A framework helps with durable state, resuming and traces, but it hides the prompt and the context, which you have to understand anyway.