Wrong tool use
The agent skips, ignores, fabricates or over-uses a tool, or picks the wrong one.
ToolFailBench labels tool-use failures Tool-Skip, Result-Ignore, Output-Fabrication and Unnecessary-Tool-Use; its authors report the best of 19 models at 86.33% clean tool use (ToolFailBench, from its abstract). Tool choice also follows how a tool is described: BiasBusters finds semantic match between query and tool description is the strongest interpretable driver across seven models (BiasBusters), and ToolTweak raised a tool's selection rate from about 20% to as high as 81% by editing only its name and description (ToolTweak). Models also follow social cues such as authority or consensus more than formal role (Who is In Charge?).
Our own reading: a fabricated tool result is a false input the later checks may not see, so this can cause gaming the check or a wrong goal downstream. Guard: descriptive tool names.
Terms on this page
- Tool call: A request from an AI model to run one action, such as a search or a file write, which the surrounding program carries out.
Is a
- Failure kind: A property of how a coding agent fails, so any future failure that fails the same way belongs to it.
Guarded by
- Descriptive tool names: Name and describe each tool for its purpose.
Can cause
- Gaming the check: The agent passes the test without doing the task.
- Wrong goal: The agent optimises what was stated, not what was meant.