Phantom Gains: When Self-Improvement Is Just Measurement Noise

A new paper from Cheng Xu, Nan Yan, Liming Chen, and M-Tahar Kechadi dropped on arXiv yesterday — "Phantom Gains: Auditing Self-Improvement Against a Measured Null" — and it should worry anyone running self-training loops on LLMs.

The claim is surgical: most of what gets reported as "self-improvement" in language models is an artifact of inadequate measurement. They audited three rounds of rank-32 LoRA self-training on Qwen3-8B against a frozen control that went through the identical pipeline. What they found: seven distinct measurement failures, each of which inverts a reported finding when its control is absent.

---

The Problem Nobody Accounts For

When you track whether a model improves on individual problems — which ones it gains and which ones it loses — you're differencing two noisy estimates. Baseline evaluation has noise. Post-training evaluation has noise. The difference doubles it. But virtually nobody measures the noise floor.

The paper's key insight: you need a measured null. A frozen copy of the base model that goes through the exact same evaluation pipeline, giving you a distribution of what "change" looks like when nothing actually changed. Without it, you can't tell real improvement from regression to the mean or batching artifacts.

graph LR
    subgraph "Standard Practice"
        A[Base Model] -->|Evaluate| B[Score A]
        A -->|Train| C[Fine-tuned Model]
        C -->|Evaluate| D[Score B]
        D -->|Compare| E["'Improvement'"]
    end
    subgraph "What the Paper Does"
        F[Base Model] -->|Train| G[Fine-tuned Model]
        F -->|"Frozen control
identical pipeline"| H[Null Model]
        G -->|Evaluate| I[Score B]
        H -->|Evaluate| J[Null Distribution]
        I -->|Compare against null| K["Real or Artifact?"]
        J -->|Sets significance threshold| K
    end

Seven Failures, All Standard Practice

The catalog of failures is damning because none of them are exotic. Several are standard practice:

• Greedy decode ledger: Building capability change metrics on a single greedy decode manufactured "capability changes" on an untrained model — largely an artifact of inference batching.
• Expansion statistic: The metric used to separate "acquisition" from "sharpening" assigned an untrained control model a rate of 0.280 — meaning it "acquired" capabilities it never trained for.
• Natural threshold repair: The common fix of choosing a natural threshold (e.g., "the model had to get it right twice") doesn't survive replication. Across the frozen comparisons such a design already contains, its null stays non-zero.

They replaced ad-hoc thresholds with a per-problem exact test against a pooled baseline under false-discovery-rate control, which detects nothing on any held-out replicate. The method is stable regardless of the multiple-testing rule, error rate, or pool size.

The Asymmetry That Matters

The most consequential finding comes from comparing different training strategies:

External distillation (learning from a teacher model's outputs) demonstrably improves problems the base model rarely reaches. Self-training (LoRA on the model's own generations) does not. A regression test rejects this asymmetry as a byproduct of distillation's larger overall gain with p < 10⁻⁸ — meaning the difference is real, not just variance.

Worse: self-training actively corrupts problems the model already solved at baseline, at rates well above the measured noise floor. You're not going nowhere — you're going backward on the things you could already do.

---

What This Means for Practitioners

If you're running self-improvement loops — and there are entire companies built around this now — you need to hear this:

• You need a measured null. A frozen baseline through the identical pipeline costs no new experiments if you're already running multi-arm studies. Don't skip it.
• Per-problem transition tracking is fragile. Tiny changes in batch size, sampling temperature, or evaluation order can manufacture "improvement."
• Distillation works. Self-training on your own outputs? Not so much. The paper suggests the mechanism is that self-training amplifies existing biases; distillation injects signal from outside the model's distribution.

The paper's companion (which also dropped yesterday) — AI4AI-Bench by Chi et al. — benchmarks recursive self-improvement from a different angle. Across 29 configurations of 6 systems, the mean score is 0.166 out of 1.0. Even the best system only reaches 0.250: closing under a fifth of the distance between the existing algorithm and the optimum. The two papers together paint an uncomfortable picture: self-improvement is either an artifact of measurement, or it's barely measurable at all.