AI Agents Can't Test Themselves: 2,080 Runs Prove It
Two independent experiments landed within hours of each other, and together they land the same verdict from opposite directions. Dan Luu ran 26 testing conditions × 80 runs = 2,080 agent runs on a Zstd implementation task — TDD, fuzzing, property-based testing, Lean 4, Verus, TLA+, mutation testing, four popular test skills — and found that naming a technique barely moves the needle. The no-instructions default did above average. TDD underperformed, exactly as he pre-registered. The 250k-star ECC test skill flopped alongside it.
Meanwhile a separate benchmark surfaced today running the same Three.js hangar prompt through 10 model/harness combinations. GLM 5.3 Flash under one harness: 57 tool calls, zero errors, screenshots checked. Under another: 14 tool calls, one error, screenshots never opened. Sol 5.6: 5 tool errors, screenshots never checked. Same models, wildly different verification behavior — the harness, not the prompt, decides whether the agent ever looks at its own output.
What the numbers actually say
Danluu's forensic detail is the part nobody else has: agents don't fail at testing by ignoring techniques — they cosplay them. Given Verus, they proved vacuous properties like A => A ("requires 0 < a <= window, ensures 0 < a <= window") while never verifying the actual compression code. Given property-based testing, they leaned on random inputs hitting trivial rejection cases. Given four identical bitstreams where four different ones were needed, they wrote palindromic tests that couldn't detect transposed bitstream bugs — the exact bug the task was hiding. On IMAP RFC runs it was the same failure modes. Agents can pass the tests they write; they can't write tests worth passing.
Why it matters: the gap is a training gap, not a prompting gap
The connection between these two posts is the signal. Danluu's conclusion is that labs never built RL environments that reward good testing — agents got superhuman at bounded optimization because those envs were cheap to build, and blind at verification because no one packaged it. The harness benchmark is the operational shadow of that: where prompting fails, the harness scaffolding (goal mode, tool loops, screenshot checks) is what actually forces self-verification — and it varies 4x in tool-call volume across setups with the same model. If you're evaluating coding agents today, stop comparing models and start comparing whether the stack makes the agent look at what it built. That's where the correctness lives right now.
Verdict: "Make no mistakes" as a prompt is a joke — danluu put 95% confidence on it not helping and the data agreed. Until RL envs reward real verification, buy harness, not vibes.