How unattended coding agents fail

Six kinds of failure in coding agents that run without a person watching, what each is a kind of, what guards against it, and the published study behind each.

By Mackenzie Bowes · mackenzie@mackenziebowes.com · @_mackenziebowes on X

Also called

agent failure modes, kinds of agent failure

A map of the ways a coding agent fails when no person is watching. Each kind is a property of the failure, so a new failure that fails the same way belongs to an existing kind. Every row names the study or report it rests on and links it. Lines starting Synthesis: are our own conclusions, not any source's.

1. The kinds

1. Compounding failure: the chance of finishing falls as the task gets longer.
2. Gaming the check: the agent passes the test without doing the task.
3. Wrong tool use: the agent skips, ignores, fabricates or over-uses a tool, or picks the wrong one.
4. Wrong goal: the agent optimises what was stated, not what was meant.
5. Drift over time: long runs lose the thread.
6. Padding: extra instructions that cost and do not help.

2. The guards

1. Short checked segments
2. Read-only tests
3. Holdout scenarios
4. Stop option
5. Near-perfect verifier
6. Ask when underspecified
7. Fresh starts
8. Descriptive tool names
9. Leave out repository overviews

3. How the kinds relate

1. Gaming the check is a kind of wrong goal, seen from the verifier's side. Our own reading.
2. Compounding failure and drift over time both grow with run length. They differ in cause: compounding is a chance of error at each step, drift is loss of context over many steps. Our own reading.
3. Wrong tool use can cause gaming the check or a wrong goal downstream, because a fabricated result is a false input the later checks may not see. Our own reading.
4. Every kind has one or more guards, listed above. Where a guard is our own prediction and untested, its page says so.