AL Alfred 2026-09-04 4 min read

Uno Diffusion LLM 3× Speedup: 8B Beats 26B DiffusionGemma

Every token of an LLM answer is written one at a time. Autoregressive decoding — next-token prediction, one forward pass per token — is why frontier models feel slow, and why faster chips buy less than they should. Speculative decoding parallelizes the process but needs a second "draft" model that half the time guesses wrong. Discrete-diffusion LLMs (d-LLMs) generate many tokens per step but train a different distribution, so you lose the quality of the autoregressive model you already trust.

A new class of models called diffusion-augmented LLMs — released as Uno — claims the missing middle: parallel decoding that preserves the exact AR distribution, needs no draft model, and delivers up to 3× throughput. The paper also lands a hard-to-ignore milestone: an 8B model beats the 26B DiffusionGemma and the proprietary Mercury 2 across agentic, coding, and long-context benchmarks.

The Problem: Autoregressive Decoding Is Serial by Design

Next-token prediction is why LLM quality is so good and why generation is so slow: token N+1 can't start until token N is done. Speculative decoding breaks the serial chain with a small draft model proposing several tokens that a big model verifies in one pass — but the draft must be trained, maintained, and matched to the target, and gains evaporate when the two disagree. d-LLMs like DiffusionGemma sample many tokens in parallel, but they define their own distribution: accelerate the decoding and you've quietly changed the model. Uno's goal is to keep the same AR distribution and draw from it in parallel.

Method: Diffusion-Augmented LLMs and Ψ-Spec Samplers

The trick is decoupling parameters. AR weights are trained with the standard next-token objective — the model stays the model you already have. A second, lightweight set of diffusion weights is learned in a "Diffusion Distillation" phase to draw multiple tokens at once from the AR model's own distribution. That phase adds negligible overhead to an existing training pipeline, and because the two parameter sets are decoupled, you can augment an existing open-weight AR LLM instead of training from scratch. At inference, a family of samplers called Ψ-Spec executes the parallel draws, delivering lossless acceleration — output quality is the underlying AR model's by construction — plus inference-time scaling at a fixed context length.

flowchart LR
    A[Next-token prediction] --> B["1 token / step
sequential"]
    C["Small draft model"] --> D["Big model verifies
mixed gains"]
    E["AR weights (NTP)"] --> G
    F["Diffusion weights
(distillation)"] --> G["Ψ-Spec sampler
draws K tokens in parallel"]
    G --> H["Up to 3× throughput
no draft model, lossless"]

Results: 3× Speedup — and an 8B Model Beats the 26B Competitor

Uno delivers up to 3× speedups over the base AR model — including at the largest batch size the device supports, the regime where autoregressive decoding is usually hardest to beat. It posts higher throughput than leading speculative-decoding methods at every evaluated batch size, with no draft model in the pipeline. The cross-model headline: the 8B Uno outperforms the 26B DiffusionGemma (the leading open d-LLM) and the proprietary Mercury 2 across all evaluated benchmarks in agentic tool use, coding, and long-context reasoning. Because Ψ-Spec enables inference-time scaling at a fixed context length, users get a serve-time knob: spend extra compute per request instead of pre-scaling context. Code and checkpoints are public.

Limitations: What "Lossless" Does and Doesn't Cover

"Up to 3×" is device- and batch-size-dependent — the headline is throughput, not latency on a single stream, and hitting the top of the range likely needs Ψ-Spec tuned to your hardware. The diffusion weights are an extra parameter set at inference and the distillation phase is a new stage in your training pipeline; the authors call it negligible, but it's still a new stage. Augmenting an existing model means running distillation on your weights — you can't bolt prebuilt diffusion weights onto any AR model. "Lossless" is demonstrated on the evaluated benchmark suite (agentic tool use, coding, long-context reasoning), not proven for every workload. And the 8B-versus-26B win is on benchmarks, not a universal ranking.

Why Builders Should Care

Decoding speed is the new training cost. For anyone serving open-weight models, Uno sketches an upgrade path with no draft model to maintain and no quality trade-off to argue about: same AR weights you already trust, an added module, and gains that hold at the batch sizes where serving density actually pays the bills. The 8B-beats-26B result is a cost signal — smaller model, same quality class, a fraction of the memory footprint and per-token cost. Inference-time scaling at fixed context length is the sleeper feature for agent workloads: spend extra compute at request time exactly when the task is hard. Checkpoints are out; this is testable today, which is exactly why it matters.

Related Posts