No One Has Figured Out Recursive Self-Improvement

Recursive self-improvement is the closest thing AI has to a philosophical singularity. The idea: an AI system designs a better version of itself, which designs a better version, and the curve steepens until you're no longer in control of the trajectory.

For all the hand-waving, nobody has measured whether current systems can even take the first step — redesigning the algorithm that trains the next model.

"AI4AI-Bench: Benchmarking LLM Agents in Algorithmic Design for Recursive Self-Improvement" by Chi, Li, Hong et al. is the first benchmark that isolates this ability and puts real numbers behind the question. The numbers are sobering.

---

The Setup

RSI (recursive self-improvement) has a concrete meaning in this paper: can an agent modify a training algorithm so the resulting model learns better? Not tune hyperparameters. Not collect more data. Change how the model learns — the loss function, the update rule, the architecture of the optimizer itself.

AI4AI-Bench gives agents 10 frozen research repositories, each implementing a different training algorithm family. The agent gets 4 hours on one B300 GPU to understand the codebase and rewrite the training algorithm. Its submission is then run from scratch (up to 12 hours of training) and scored against a fixed evaluator the agent never sees.

The scoring scale is clever: 0 = an uninformative model, 0.1 = the algorithm the repository already ships, and 1.0 = the task-optimum performance. So any score above 0.1 means the agent improved the existing algorithm. Below 0.1 means it made things worse.

---

The Results: A Flat Ceiling

Across 29 configurations of 6 different agent systems, on all 10 tasks, the mean score is 0.166. The best system reaches 0.250.

Let that sink in. Even the strongest system in the benchmark closes under a fifth of the gap between the algorithm that was already there and the theoretical optimum. The median system barely outperforms doing nothing.

graph LR
    A["Baseline (0.10)"] --> B["Mean Score (0.166)"]
    A --> C["Best System (0.250)"]
    A --> D["Optimum (1.00)"]
    B -- "+0.066" --> E["~7% of optimum gap"]
    C -- "+0.150" --> F["~17% of optimum gap"]
    style A fill:#27272a,stroke:#52525b
    style D fill:#a78bfa,stroke:#a78bfa

The paper slices the data further. The most important finding: most agents never change how the model learns at all. They tinker with data processing, evaluation scripts, logging — anything except the training objective or update rule. The minority that do modify the learning algorithm average 0.226 versus 0.126 for those that don't. That's a real gap, but it still sits far below the 0.1-to-1.0 range.

More reasoning effort (measured by thinking tokens or search budget) mostly buys willingness to modify the algorithm, not skill at doing it. The proportion of submissions that touch the training algorithm jumps from 8% to 64% as reasoning budget increases, and the mean score climbs from 0.094 to 0.196. But the delta-per-reasoning-token drops fast: the first bit of reasoning gets you to try; the next 10x barely makes you better at it.

---

Why This Matters

The threshold for "recursive self-improvement" isn't a single agent writing a slightly better loss function. It's the threshold where the next system, trained with the improved algorithm, is more capable than the system that designed the improvement — and can therefore design an even better one. AI4AI-Bench shows we're nowhere close.

A 0.166 mean score means the current crop of coding agents — GPT-4o, Claude, DeepSeek-V4, whatever scaffold you're running — can barely improve on existing research-level training code. They can find bugs, clean up implementations, and sometimes make incremental gains, but they cannot invent a better way to learn.

For builders: this is a concrete benchmark you can point to when someone claims their agent architecture "will recursively self-improve." Run it. The paper releases the task suite, evaluators, and every scored submission. If you think your system is different, prove it.

For researchers: the gap between "willing to modify the algorithm" (8→64% with more reasoning) and "good at modifying it" (0.094→0.196) is the real bottleneck. We don't need more compute. We need better representations of what a training algorithm is and how its pieces interact causally.

---

Limitations

The benchmark is limited to 10 training algorithm families with 4-hour time budgets on one B300. Real algorithm design often requires longer iteration cycles and access to scaling runs. The evaluation also assumes a fixed, hidden evaluator — in practice, the fitness function for training algorithms evolves as the field advances. And the agent is judged on a single rewrite attempt; real algorithm research involves weeks of iterative refinement.

The 10 repositories, while spanning diverse families (regularization, normalization, optimization, etc.), don't cover every approach. There's no meta-learning, no evolutionary search, no learned optimizers — these are intentionally excluded to keep the benchmark tractable, but they represent important adjacent capabilities.

Finally, the paper spans 10 authors and the task suite is well-constructed, but the findings are a single snapshot. The real trajectory — whether these scores improve as agent capabilities grow — is the question the benchmark is built to track.