Compounding failure

The chance of finishing falls as the task gets longer.

By Mackenzie Bowes · mackenzie@mackenziebowes.com · @_mackenziebowes on X

The chance of finishing falls as the task gets longer. Ord argues that agent success on METR's tasks fits a constant failure rate per unit of task time (Ord, half-life of AI agent success). METR's 2026 report puts the public frontier 50% horizon at about 12 hours and the 80% horizon at about 1.5 hours, measured in the time a skilled human takes (METR frontier risk report, 2026). On Sierra's τ-bench, GPT-4o's retail score falls about 60% from one attempt to eight consistent ones (Sierra, τ-bench).

Guard: short checked segments. It grows with drift over time; both worsen with run length. Our own reading: compounding is a chance of error at each step, drift is loss of context over many steps.

Terms on this page

  • AI agent: A system where a language model decides for itself how to do a task and which tools to use.

Is a

  • Failure kind: A property of how a coding agent fails, so any future failure that fails the same way belongs to it.

Guarded by

Grows with

Grows with this