- In short
- A rising rate of wrong-tool selections or unexplained cost growth is a symptom of capability bloat, an oversized tool set, rather than a model-quality problem. Diagnosis starts by mapping each connected tool back to the task requirement that justifies it. The fix is removing the unjustified tools, not adding more prompt instructions asking the model to choose carefully or upgrading to a higher model tier.
Reading the symptoms of an oversized tool set
This is the evaluate-level capstone of the capability-bloat task statement: given a deployed agent that is misbehaving, correctly attribute the misbehaviour to its tool set rather than to the model. Two symptoms point here. The first is a rising rate of wrong-tool selections, the agent reaching for the wrong tool at a step more and more often. The second is unexplained cost growth, spend creeping up without a matching change in traffic or task. Both are classic signatures of capability bloat, and both are routinely misdiagnosed as the model getting worse.
The misattribution matters because it sends teams down the wrong fix. If you believe the model is the problem, you write sterner prompts or upgrade the tier. Neither touches the actual cause, which is that the tool set has grown to the point where the model has too many candidates to disambiguate and too much definition text riding in every request. Diagnosing correctly means recognising these symptoms as tool-set symptoms.
- Capability-bloat diagnosis
- Attributing a rising wrong-tool-selection rate or unexplained cost growth in a deployed agent to an oversized tool set rather than to model quality, by mapping each connected tool back to the task requirement that justifies it and removing the tools that no requirement supports.
Diagnosis: map every tool to a requirement
The diagnostic procedure is concrete. Take the agent's full tool set and, for each connected tool, trace it back to the specific task requirement that justifies it, the same justification the necessity test demanded at inclusion and the audit workflow re-checks over time. Tools that map cleanly to a current requirement stay. Tools that map to nothing, or to a requirement that no longer exists, are the bloat. That mapping exercise is the diagnosis: it turns a vague "the agent is misbehaving" into a specific list of tools that have no business being connected.
The mapping also explains the symptoms mechanically. Every unjustified tool is an extra candidate at selection time, which is where wrong-tool errors come from, and its definition consumes tokens on every request, which is where the quiet cost growth comes from. Remove the unjustified tools and both symptoms recede, because you have removed their cause.
The fix is removal, not more instructions or a bigger model
The trap-laden part of this knowledge point is the fix. Faced with wrong-tool selections, the instinct is to add prompt instructions: "think carefully about which tool to use," "only use tool X when condition Y holds." This is treating a structural problem as a wording problem. The model is not selecting wrong because it was not told to be careful; it is selecting wrong because it has too many similar candidates. More instructions do not shrink the candidate set. The fix is removing the unjustified tools so there are fewer, more distinct candidates to choose among.
The other tempting non-fix is upgrading the model tier. A more capable model may mask the problem for a while, but capability bloat is a design defect, not a capability ceiling, so a bigger model still carries the wider attack surface, still pays the token cost of the extra definitions, and still faces an inflated candidate set. Tool-set discipline is required at every tier. The credited answer removes tools; it does not out-spend the problem.
What the exam trips candidates on
The two traps are the two wrong fixes. The first is attempting to fix tool-selection errors purely through prompt engineering when the real issue is an oversized tool set; instructions do not reduce the candidate count that causes the errors. The second is assuming a higher-capability model tier eliminates the need for tool-set discipline; it does not, because bloat is structural. The credited answer diagnoses by mapping tools to requirements and fixes by removing the unjustified ones.
Worked example
Over three months, a deployed agent's wrong-tool-selection rate has climbed and its monthly cost has grown 40% with no increase in request volume. An engineer proposes adding a detailed prompt section instructing the model to reason carefully about tool choice, plus upgrading from the mid tier to the top tier. Diagnose and prescribe.
Start with the symptoms and what they point to. A rising wrong-tool rate together with cost growth that traffic does not explain is the textbook signature of capability bloat, so the first move is to suspect the tool set, not the model. The correct diagnosis is to map every connected tool back to the task requirement that justifies it. In an agent that has accreted tools over three months, that mapping typically surfaces a batch of tools tied to no current requirement, and those are the source of both symptoms: extra candidates inflating selection errors and extra definitions inflating per-request token cost.
Now weigh the engineer's proposal. The prompt section telling the model to reason carefully does not shrink the candidate set, so it leaves the structural cause of the wrong-tool selections untouched; it is the first trap exactly. The tier upgrade may paper over the selection errors temporarily, but it does nothing about the token cost of the bloated definitions, so it will not reverse the cost growth, and it leaves the design defect in place, which is the second trap. The prescription is to remove the unjustified tools identified by the mapping. That directly reduces the candidate set the model disambiguates over, lowering the wrong-tool rate, and it removes the definition tokens driving the cost growth, all without changing the prompt or paying for a higher tier.
Common misreadings to avoid
Misconception
Rising wrong-tool selections mean the model needs clearer instructions about when to use each tool.
What's actually true
Misconception
Upgrading to a higher model tier will resolve tool-selection errors and cost growth caused by a large tool set.
What's actually true
How this shows up on the exam
Expect a scenario that presents symptoms, a climbing wrong-tool rate or cost growth without a traffic change, and offers prompt engineering or a tier upgrade as the fix. The reliable reading is that these are capability-bloat symptoms, that diagnosis means mapping tools to requirements, and that the fix is removing unjustified tools. This knowledge point is the evaluate-level payoff of the audit workflow and scoped subagent access, and it shares its root-cause discipline with the production failure taxonomy, which similarly warns against fixing a structural failure with stronger wording.
A deployed agent shows a rising wrong-tool-selection rate and 40% cost growth with flat request volume. Which response best addresses the actual cause?
People also ask
What causes an agent to keep selecting the wrong tool?
Is rising tool-selection error a model-quality problem?
Does a better model tier fix tool-selection errors?
Watch and learn
Official Anthropic Academy lessons first, then hand-picked walkthroughs. Videos load only when you press play.
No videos curated for this concept yet
We are still curating the best official and community videos for this topic.
Official prep for this domain
Anthropic's own free prep module for this part of the syllabus, on the official prep course. Free with an Anthropic Academy sign-in.
References & primary sources
Master this concept with Archie
Practice it inside an adaptive study session. Archie, your Socratic AI tutor, tracks your mastery with Bayesian Knowledge Tracing and schedules the perfect next review.