When a model needs tools, and when it does not

There's a strong pull toward building the most capable-sounding thing: an agent, with tools, that loops. It demos well. It is also, for most tasks, a reliability downgrade over something much simpler.

Each capability you add buys something specific and costs something specific. Know which you're buying.

The four architectures

Plain prompting. Everything needed is in the prompt. Deterministic-ish, one call, cheap, fast, easy to evaluate. Fails when the answer depends on information you don't have at prompt-writing time.

Retrieval. Fetch relevant content, put it in the prompt, answer over it. Handles corpora too large for context and content that changes. Introduces retrieval failure: if the fetch misses, you get a confident answer over the wrong passages, and nothing signals it. Your quality ceiling becomes your retrieval quality, not your model quality, which is where most disappointing "RAG" systems actually go wrong.

Tools (single-step). The model calls a function, gets a result, answers. Gets you live data, real computation, and actions. Each call is a place things can fail, and the model can call the wrong tool with the wrong arguments.

Agent loop. The model calls tools repeatedly, deciding for itself when it's done. Handles tasks whose steps aren't known in advance. Costs you predictability: variable cost, variable latency, and a much larger space of failure modes, and errors compound. A wrong step at iteration two poisons everything after it.

The decision, in four questions

Does the answer depend on information that changes? If yes, you need retrieval or tools. Static knowledge that changes yearly can live in the prompt. Policy that changes monthly cannot.

How large is the source of truth? Fits comfortably in context and is stable → paste it. Thousands of documents → retrieve.

Does the task require an action or a real computation? Sending an email, querying a database, doing arithmetic you need to be exactly right → tools. Models should not do arithmetic they could delegate.

Are the steps knowable in advance? If you can write the sequence, write it in code and call the model at each step. That's a workflow, and it's more reliable and far easier to debug than an agent. Reserve the agent loop for tasks where the path genuinely cannot be determined until you're partway through.

That last question is the one that saves the most grief. Most things people build as agents are workflows with an agent's unpredictability bolted on.

Worked example: employee policy questions

Walk the questions.

Information changes? Yes. Policies get revised, and a stale answer about parental leave is worse than no answer.

Size? A few hundred documents. Too large to paste, and pasting everything for every question would be slow and expensive even if it fit.

Actions or computation? No. It's a question-answering task.

Steps knowable? Yes: find relevant policy, answer from it, cite it. Same three steps every time.

Conclusion: retrieval over a policy corpus, no agent loop. Fixed pipeline, one model call per question, citations to the source document mandatory.

The strongest argument against: retrieval will sometimes fetch the wrong passage, and the system will then answer confidently from an outdated or irrelevant policy. An employee acts on it, and the answer was wrong in a domain where wrong answers have legal consequences. An agent loop could notice its retrieval looked thin and search again with different terms. Genuinely better recall on hard queries.

Why accept it anyway: the agent's extra recall is bought with unpredictability, and unpredictability is worse than a known limitation in a compliance-adjacent domain. Instead, address the failure directly: mandatory citations so the employee sees which policy the answer came from and its date; an explicit "I could not find a policy covering this. Ask HR" path so absence of evidence doesn't get papered over; and logging of low-confidence retrievals for human review. That's a bounded, inspectable system with a known weakness, versus an unbounded one with an unknown weakness.

Note the structure of that argument: pick, state the real objection, accept the cost explicitly, and mitigate the specific failure rather than escalating architecture to make the discomfort go away.

The reliability arithmetic

If each step in a chain is 95% reliable, a five-step agent loop is about 77% reliable end to end, and the failures aren't independent. A bad early step makes later steps worse. Adding steps to fix a quality problem often makes the system worse overall.

Before adding a step, ask what its individual failure rate is and what happens downstream when it fails. If you can't answer, you can't predict the system's behaviour, which means you can't operate it.

Where this is weak

It assumes you know your task. If you're exploring. You don't yet know what questions people will ask, or what "good" looks like. Build the simplest version, put it in front of real users, and let the failures tell you which capability you actually need. Architectural reasoning about a task you don't yet understand produces confident designs for the wrong problem.

The move

Before building: answer the four questions in writing. Then pick the simplest architecture that answers them, not the most capable one. Write down the strongest objection to your choice and what you'll do about that specific failure. If you can't state the objection, you haven't understood the design well enough to build it yet.