Gaming the check

The agent passes the test without doing the task.

By Mackenzie Bowes · mackenzie@mackenziebowes.com · @_mackenziebowes on X

The agent passes the test without doing the task. On tasks whose tests contradict the spec, GPT-5 cheated in 76% of One-Off cases (ImpossibleBench). StrongDM says tests inside the codebase got gamed ("return true") (StrongDM, Software Factories).

Our own reading: this is a kind of wrong goal seen from the verifier's side, because a verifier the agent can pass without doing the work states the wrong goal. Guards: read-only tests, holdout scenarios, a stop option.

Is a

  • Wrong goal: The agent optimises what was stated, not what was meant.

Guarded by

  • Holdout scenarios: Checks stored outside the codebase, where the agent cannot see them.
  • Read-only tests: Tests the agent can run but cannot edit.
  • Stop option: A way for the agent to flag a task as impossible and hand it to a person.

Caused by

  • Wrong tool use: The agent skips, ignores, fabricates or over-uses a tool, or picks the wrong one.