Compounding failure
The chance of finishing falls as the task gets longer.
The chance of finishing falls as the task gets longer. Ord argues that agent success on METR's tasks fits a constant failure rate per unit of task time (Ord, half-life of AI agent success). METR's 2026 report puts the public frontier 50% horizon at about 12 hours and the 80% horizon at about 1.5 hours, measured in the time a skilled human takes (METR frontier risk report, 2026). On Sierra's τ-bench, GPT-4o's retail score falls about 60% from one attempt to eight consistent ones (Sierra, τ-bench).
Guard: short checked segments. It grows with drift over time; both worsen with run length. Our own reading: compounding is a chance of error at each step, drift is loss of context over many steps.
Terms on this page
- AI agent: A system where a language model decides for itself how to do a task and which tools to use.
Is a
- Failure kind: A property of how a coding agent fails, so any future failure that fails the same way belongs to it.
Guarded by
- Short checked segments: Cut a long task into short segments and check each one.
Grows with
- Drift over time: Long runs lose the thread.
Grows with this
- Drift over time: Long runs lose the thread.