Read-only tests
Tests the agent can run but cannot edit.
On ImpossibleBench, hiding the tests cut cheating to near zero but also cut honest performance; read-only tests restored it (ImpossibleBench).
Is a
- Guard: Something that stops a kind of failure before it reaches the end of an unattended run.
Guards
- Gaming the check: The agent passes the test without doing the task.