What is CanvasAnneal?

A curriculum-guided RL framework for diffusion language models. Teacher-generated reasoning traces are injected into the initial diffusion canvas early in GRPO training, then gradually removed so the model learns to reason from pure noise. Inference is unchanged — masking ratio 1.0 preserves native diffusion generation.

How much does it improve over standard diffusion RL?

It beats diffu-GRPO by +6.0/+2.0/+0.4 points on MATH500 at generation lengths 128/256/512, by up to +3.91 on Countdown, and lifts Tau2 Airline tool-use from 12.0% to 18.0%. Without SFT it beats the SFT+diffu-GRPO (d1-LLaDA) pipeline on MATH500 at lengths 256 and 512 (41.0% vs 37.6%, 45.2% vs 36.8%).

Where does the method fail?

On easy tasks and long canvases. GSM8K regressions across all lengths mean guidance is overhead when exploration isn't the bottleneck, and the MATH500 gain decays to +0.4 at length 512 because pure-noise rollouts self-correct with enough tokens.

Diffusion LLM RL: Curriculum Guidance Adds +6.0 Pts on MATH500

Diffusion language models still lose to autoregressive models at reasoning, and the failure is structural: when you roll out from a pure noise canvas, the policy rarely stumbles onto a long correct reasoning chain by chance. RL can't reward what it never samples. CanvasAnneal (arXiv 2609.13060, announced today) attacks exactly that exploration bottleneck — and the shape of its gains tells you more than the gains themselves.

The Problem: Exploration, Not Credit Assignment

Applying GRPO-style RL to a diffusion LLM (diffu-GRPO) means sampling groups of rollouts from an unconditioned noise start. On hard tasks the reward is nearly always zero — no gradient signal, no learning. The standard fix is two-stage: SFT on teacher traces first, then RL. CanvasAnneal collapses this into the RL loop itself. Teacher traces (from Gemini 3.1 Pro, ~256 tokens, answer withheld) are injected into the initial canvas, partially masked. As training progresses, the injection ratio anneals — more prompt groups roll out from pure noise. Inference is untouched: masking ratio m=1.0 preserves native diffusion decoding. It's a training-time scaffold, not a runtime dependency.

mindmap
  root((CanvasAnneal))
    Early RL
      Trace from teacher in canvas
      Partially masked suffix
      Reward signal flows on hard tasks
    Late RL
      Pure-noise rollouts dominate
      Model generates full trajectory
      Native diffusion inference intact

The Results: Gains Shrink As the Canvas Grows

On MATH500, CanvasAnneal beats standard diffu-GRPO by +6.0, +2.0, and +0.4 points at generation lengths 128, 256, and 512. Countdown: +2.73, +3.91, +1.96. On Tau2 agent tool-use: Retail 6.14%→7.90%, Airline 12.00%→18.00%, Telecom 12.30%→13.20%. That Airline jump is a 50% relative gain on the hardest domain — where the exploration bottleneck was widest.

But read the length gradient. The benefit decays toward zero as generation length grows. That's the paper's most interesting unspoken finding: injected traces are a crutch for short canvases, where a wrong early denoising pass leaves no room to recover. Given 512 tokens, diffusion exploration self-corrects and the curriculum adds nothing. GSM8K — the easy task — actually regresses across all lengths: standard diffu-GRPO stays stronger. Guidance on easy tasks is pure overhead. This is a difficulty-targeted intervention, not a general RL improvement.

The Quiet Implication: SFT May Be Optional

The number buried in Table 1 that deserves the headline: CanvasAnneal without SFT beats the full SFT+RL d1-LLaDA pipeline on MATH500 at lengths 256 and 512 — 41.0% and 45.2% vs 37.6% and 36.8%. If that holds up across models, the standard two-stage recipe (distill, then RL) is doing work that in-RL trace injection does cheaper, with one less training stage. The ablation backs this up: 100% trace injection is worse than a mixed ratio (GSM8K 52.00 vs 57.01 at length 128), because pure-noise rollouts are still needed to learn without teacher context. The scaffold matters — and so does tearing it down.

Limitations the Authors Underplay

Distillation in disguise? The teacher is Gemini 3.1 Pro. Nothing in the paper isolates how much of the gain is "better RL exploration" versus simply compressing a frontier model's reasoning. A compute-matched SFT-then-RL run with the same traces would settle it; the SFT baseline does exist but the no-SFT comparisons are inconsistent across tasks.
Absolute numbers are rough. The best Tau2 result is 18%. The base diffusion model (LLaDA-7B-A1B-Instruct) is far behind autoregressive peers; curriculum guidance narrows the gap, it doesn't close it.
One base model, one teacher. No Dream-8B runs, no alternative teachers, no sensitivity analysis on trace quality. The annealing schedule itself is never ablated — only the injection ratio at fixed points.

Why Builders Should Care

If you're betting on diffusion LLMs for parallel generation or infilling, this changes the training playbook: you don't need a separate SFT corpus — bake teacher traces into the RL canvas and anneal them out. But budget your expectations by task difficulty and canvas size: the win is on hard, short-horizon problems. And the GSM8K regression is a warning for anyone who applies this blindly — measure per task, not on average.