Wrong goal

The agent optimises what was stated, not what was meant.

By Mackenzie Bowes · mackenzie@mackenziebowes.com · @_mackenziebowes on X

The agent optimises what was stated, not what was meant. Ordóñez and colleagues list the side effects of narrow goals, including "a narrow focus that neglects non-goal areas" (Goals Gone Wild). Carlini: "it's important that the task verifier is nearly perfect, otherwise Claude will solve the wrong problem" (Anthropic, Building a C compiler). In Ask or Assume, separating detection of underspecification from execution lifted task resolution from 61.2% to 69.4% (Ask or Assume?).

Guards: a near-perfect verifier and asking when the task is underspecified.

Is a

  • Failure kind: A property of how a coding agent fails, so any future failure that fails the same way belongs to it.

Guarded by

Kinds of this

Caused by

  • Wrong tool use: The agent skips, ignores, fabricates or over-uses a tool, or picks the wrong one.