AI That Fixes AI — Anthropic's Automated Alignment Researcher

Anthropic just published a paper that quietly shifts the Overton window on AI safety research. The setup: give Claude a research loop — search literature, propose methods, train models, test — and ask it to fix alignment failures. The result: Claude closed the safety gap across all 10 categories of alignment failure, including deception, sycophancy, privacy violations, and jailbreaks.

This isn't fine-tuning. Claude autonomously trained student models, constrained by a monitoring agent that vetoed any method that would degrade general capabilities or distill Claude's own alignment directly. The fixes stuck: they generalized to withheld benchmarks Claude never saw during research, and transferred to models 4.7× larger than the ones it optimized for.

Then the kicker: Claude outscored 28 human safety researchers who were given up to eight hours to devise methods for the same problems. Let that land. A model autonomously generated better alignment research than domain experts with time to think.

TechCrunch has the framing right — this is a peek at self-improving AI. Not the recursive self-improvement of sci-fi, but the real, measurable version: an AI that can audit its own safety, propose fixes, validate them, and deploy them without human in the loop. Anthropic calls it "automated alignment researchers," and the paper is careful about constraints (the monitoring agent, the capability-gating), but the trajectory is clear.

The implications are double-edged. On one hand, this is exactly the kind of safety research that needs to scale — alignment is a game of speed, and if AI can outrun human researchers on finding and fixing failures, that's a net win. On the other hand: who monitors the monitors? If the automated researcher gets better at proposing methods, and the monitoring agent gets better at approving them, the loop closes in ways that become opaque.

Worth watching how the open-source community reacts. The paper names Petri (Anthropic's open-source alignment simulator) as a key evaluation tool — expect forks and extensions within weeks.