Wrong tool use

The agent skips, ignores, fabricates or over-uses a tool, or picks the wrong one.

By Mackenzie Bowes · mackenzie@mackenziebowes.com · @_mackenziebowes on X

Is a
Failure kind
Guarded by
Descriptive tool names
Can cause
Gaming the check, Wrong goal

ToolFailBench labels tool-use failures Tool-Skip, Result-Ignore, Output-Fabrication and Unnecessary-Tool-Use; its authors report the best of 19 models at 86.33% clean tool use (ToolFailBench, from its abstract). Tool choice also follows how a tool is described: BiasBusters finds semantic match between query and tool description is the strongest interpretable driver across seven models (BiasBusters), and ToolTweak raised a tool's selection rate from about 20% to as high as 81% by editing only its name and description (ToolTweak). Models also follow social cues such as authority or consensus more than formal role (Who is In Charge?).

Our own reading: a fabricated tool result is a false input the later checks may not see, so this can cause gaming the check or a wrong goal downstream. Guard: descriptive tool names.