Near-perfect verifier

A check that is close to right, so the agent cannot solve the wrong problem.

By Mackenzie Bowes · mackenzie@mackenziebowes.com · @_mackenziebowes on X

Carlini: "it's important that the task verifier is nearly perfect, otherwise Claude will solve the wrong problem". A known-good compiler served as the oracle, which let 16 agents work at once (Anthropic, Building a C compiler).

Is a

  • Guard: Something that stops a kind of failure before it reaches the end of an unattended run.

Guards

  • Wrong goal: The agent optimises what was stated, not what was meant.