Failure kind

A property of how a coding agent fails, so any future failure that fails the same way belongs to it.

By Mackenzie Bowes · mackenzie@mackenziebowes.com · @_mackenziebowes on X

A failure kind is defined by how the failure happens, not by one instance of it. The kinds in this map are compounding failure, gaming the check, wrong tool use, wrong goal, drift over time and padding.

Terms on this page

  • Tool call: A request from an AI model to run one action, such as a search or a file write, which the surrounding program carries out.

Kinds of this

  • Compounding failure: The chance of finishing falls as the task gets longer.
  • Drift over time: Long runs lose the thread.
  • Padding: Extra instructions that cost and do not help.
  • Wrong goal: The agent optimises what was stated, not what was meant.
  • Wrong tool use: The agent skips, ignores, fabricates or over-uses a tool, or picks the wrong one.