Chapter 4 — AI Agents and Tool Calling

Fourth post of the chapter-by-chapter walkthrough of LLM Primer V: Building Real-World LLM Applications. The chapter that treats an agent as a language model looped against tools whose schemas, memory layers, and multi-agent wiring have to be engineered before the loop can be trusted with anything real.

Why this chapter exists

A stateless model that only takes text in and emits text out is a calculator. Production work requires turning it into an actor that pursues goals across multiple steps — call this tool, then that one, then decide which of the results to summarize for the user, and stop when the task is done. The move looks simple in a diagram and is not simple in reality. Agents wander, loop, call the wrong tool, invent arguments the tool cannot accept, forget what they already fetched, and act outside their authority. Chapter 4 is about the engineering that keeps the loop legible and bounded — the schemas, the memory discipline, and the multi-agent structure that turn a plausible-looking demo into a production actor.

4.1 Agent architectures as composition, not selection

The base loop is ReAct: reason about the situation, act by calling a tool, observe the result, and repeat until the goal is met or the step budget is exhausted. Production systems then compose three augmentations on top of the base rather than choosing between them. Native function calling — the provider constrains tool arguments against a JSON Schema at decode time — hardens the I/O contract so the loop no longer has to recover from malformed calls. Explicit planning, in the Plan-and-Execute style, front-loads a plan for long-horizon tasks and reruns the planner only when the world changes. Workflow orchestration encodes the known transitions of the task as a graph and leaves only the within-phase decisions to the model, which is the shape most enterprise deployments converge on because it separates what is knowable from what needs judgment. Reflection loops, multi-model routing across a cost-tiered fleet, and specialist sub-agents sit alongside these as further augmentations.

4.2 Tool calling mechanics — the schema is the contract

Every tool has a schema, and the schema is the contract the loop enforces on the model. Property-level descriptions are documentation for the model, not for the human reader; enums close the argument space where the domain permits; idempotency keys let the loop retry a tool call without doubling its effect; structured retryable errors let the model recover cleanly rather than guess. Tools should be minimal — one clear job per tool — because a fat tool with ten optional flags is a tool the model will get wrong. An ask_user tool belongs in the catalog explicitly, so the agent has a legitimate way to escalate ambiguity rather than inventing arguments. Concurrency is safe only when tools declare independence; the loop treats declared-independent tools as parallelizable and everything else as strictly sequential. Every real production incident traced to "the agent did the wrong thing" resolves to a tool whose schema did not say what its arguments meant.

4.3 Three layers of memory: short-term, long-term, semantic

Agents need memory because a task rarely fits in one turn. Short-term memory is a sliding window over recent conversation with pinned messages that survive the slide — the system prompt, the current goal, the running plan — and periodic summarization of the runs that fall out of the window. Long-term memory is a vector store of curated facts written on confirmation, not on every observation, and retrieved at multiple points in the loop rather than only at the start. Semantic memory is a knowledge graph of triples for the queries that need structured composition rather than similarity — "who reports to whom," "which products belong to which category," relationships the vector store flattens. The three are stored differently because they are used differently, and the discipline is to route writes and reads to the right layer instead of collapsing everything into one embedding index.

What Chapter 4 sets up

Agents and RAG both produce stochastic traces. The user complains, the log shows a hundred spans across three tools and eight model calls, and the team has to decide whether the failure is a retrieval regression, a prompt drift, a tool schema issue, or the agent choosing correctly against a broken downstream system. That question cannot be answered without a discipline for turning traces into measurable pass/fail signals. Chapter 5 is that discipline — LLM-as-judge, the RAG Triad, trajectory tests for agents, and the continuous loop where production traces feed the eval set that gates the next release.

Next — Chapter 5: Evaluating LLM Applications. The evaluation discipline that turns stochastic traces into pass/fail signals a team can ship against.