I Tried to Watch AI Improve Itself — It Mostly Just Twiddled Knobs

Recursive self-improvement is the endgame everyone talks about but nobody has seen. An AI that improves the process that creates AI systems so the next one is better, which improves the process further — the compounding loop that breaks the scaling ceiling.

A new benchmark from Einsia.AI and Tsinghua tries to measure whether current models can do the first step: design a better training algorithm. The results are uncomfortable reading for anyone betting on near-term RSI.

AI4AI-Bench (arXiv:2608.20318) freezes 10 real research repositories spanning 10 training algorithm families — supervised fine-tuning, agentic RL, reward modelling, preference optimization, diffusion RL, machine unlearning, graph diffusion, weight averaging, pruning. An agent gets 4 hours on a single B300 GPU to rewrite the training code, then the modified code is rerun from scratch for up to 12 hours and scored by a fixed evaluator the agent never sees.

The scoring ladder is the most thoughtful part of the design. On a scale where 0.1 = the algorithm the repo ships, 1.0 = the task optimum, and 0 = an uninformative model, every submission maps to the same coordinate. It means you can compare improving a perplexity from 53.4 to 13.0 with lifting an accuracy by 0.015 — apples to alien fruit, normalised.

Let me show you what 290 cells across 6 systems (GPT-5.6 variants, Claude Opus 5, Sonnet 5, Kimi K3) actually produced.

---

The Number That Matters

The study-wide mean is 0.166. The best system — Claude Opus 5 — averages 0.250. The single best configuration, Opus 5 at medium effort, hits 0.288.

That means: the strongest agent in the study, on its best day, closed under a fifth of the gap between the algorithm a researcher already shipped and the theoretical optimum for that task.

And 124 of 290 cells (43%) scored below 0.1. They made things worse than the baseline. The agent opened the repo, spent 4 hours and up to $434 of API calls, and the code it left behind produced a worse model than what was already there.

"Systems are ordered, and the ordering is compressed. The best system's average is closer to the weakest system's than it is to the optimum it was asked to approach." — AI4AI-Bench

The Real Finding: Nobody Touches the Algorithm

The score is just the headline. The paper's real contribution is in what it classifies — and this is where it gets uncomfortable.

Every submission was read by a separate LLM and tagged into eight families on two sides of a line:

graph LR
    subgraph "Run Side (changes how the run goes)"
        A[Training duration / checkpointing]
        B[Hyperparameters: LR, batch size]
        C[Which checkpoint to keep]
        D[How much capacity / where attached]
    end

    subgraph "Learning Side (changes how the model learns)"
        E[The loss / objective]
        F[The supervision signal]
        G[The update rule itself]
        H[The training data]
    end

Of 263 submissions that changed something, 141 never touched the learning side — they stayed entirely on the run side. That's 53.6% of all attempts. Four hours with a complete research repo, a clear instruction to improve the training algorithm, and more than half the submissions changed how long it trains, the batch size, or which checkpoint to save.

The 122 that did reach the learning side averaged 0.226 vs 0.126 for those that didn't. This holds when you drop any single task, any single model. The algorithmic layer is where the distance gets closed — and most submissions never go there.

What Reasoning Effort Actually Buys

The paper isolates one controllable variable: reasoning effort (the budget you give the model for chain-of-thought before acting). Going from the lowest effort setting to the highest:

• Share touching the learning algorithm: 8% → 64% (8x increase)
• Mean score: 0.094 → 0.196 (roughly 2x)
• Empty/zero-score submissions: concentrated at low effort
• Cost: median exploration cost rose from $1.69 to $34.60

The paper's framing is precise: "Reasoning effort buys nerve, and nerve is what pays." More thinking doesn't make the attempt better — it makes the agent willing to attempt the thing that matters. At low effort the submissions tweak logging and batch sizes. At high effort they replace loss functions and add new supervision signals.

But even at max effort, 0.196 is still only a tenth of the way from the shipped algorithm to the optimum. Nerve gets you to the right layer. It doesn't guarantee you can operate there.

The Three Exceptions Prove the Rule

The paper highlights three submissions that genuinely redesigned their task. They're worth reading because they show what's missing everywhere else:

1. One-shot pruning turned into a 3-stage pipeline — The repo scores each weight once, removes it, stops. One submission replaced this with a pipeline: a different selection rule, then layerwise distillation, then masked fine-tuning. Perplexity went from 53.4 to ~13. It found and fixed a bug along the way (activations overwriting in place → first attempt scored 572).
2. Weight averaging became an optimization problem — The repo averages 72 model checkpoints. One submission built a measurement rig 500x faster (0.38s per evaluation vs 190s), ranked 5 methods on it, and selected the best.
3. RL replaced with imitation learning — The agentic RL task (RAGEN) was solved perfectly by one configuration that abandoned GRPO entirely: generate boards, label with optimal moves, fine-tune on the supervision.

The common thread: each built measurement infrastructure first. A solver to find the task's ceiling. An evaluation rig 500x faster. A debugger to localise a memory corruption bug. They diagnosed before they prescribed.

"What separates a submission that reaches that layer from one that does not is not effort spent but a step taken first: reading the training dynamics as a specific failure mechanism, and then addressing that mechanism." — AI4AI-Bench

What This Means

I've been running autonomous research agents for months on Dispatch. This paper confirms something I've felt but couldn't quantify: current agents are instrument-builders in theory, knob-twiddlers in practice.

Give them a harness and 4 hours, and they'll tune learning rates, adjust batch sizes, increase checkpoint frequency. They'll do what the framework exposes. What they won't do — unless you spend heavily on reasoning — is open the training loop, read the dynamics as a causal story, and rewrite the learning rule.

The cost data is the most damning. Opus 5 leads the study at a median of $181 per configuration. Luna, the weakest GPT-5.6 variant, spends $48 but scores 0.117. Sol, the strongest, spends $434 and scores 0.191. Money doesn't buy algorithm design. The systems that go to the algorithmic layer do so because of something closer to scientific intuition — the willingness to stop running experiments and instead build an instrument that tells you what's failing.

The paper's own conclusion is the right one: "The algorithmic design level is where the distance to a better algorithm is closed and the one these agents reach least often, and more reasoning effort mostly buys the willingness to reach it."

Recursive self-improvement isn't dead. But the first step — an agent redesigning its own training algorithm — is harder than any headline about "agentic coding" suggests. Current systems don't lack compute or budget. They lack the reflex to read training dynamics as a mechanism that can be changed.

And until that reflex exists in model weights, not just in prompt engineering, RSI stays theoretical.

Bottom Line

AI4AI-Bench is the first benchmark that asks whether an agent can improve its own training algorithm — and the answer, across 290 attempts by 6 top systems, is "barely, and only when you spend enough reasoning to make it brave enough to try." The real gap isn't performance. It's that 53% of submissions never touched the algorithm at all. They didn't fail at algorithm design — they didn't attempt it.

The three submissions that did succeed show the way forward: build measurement first, diagnose before you prescribe, and treat the training loop as a mechanism you can replace, not a set of knobs you can turn. Everything else is just expensive hyperparameter search.