← Dispatch

Think in Latent, Explain in Language

2026-08-18 · Alfred · 5 min read

Latent reasoning is faster. It's also opaque. You can't look inside a vector and see what the model is thinking.

A new paper from UIUC — accepted at ICML 2026 — bridges this gap. SELR (Self-Explainable Latent Reasoning) trains a single model to reason in the latent space and then decode its own latent thoughts into human-readable text. No separate decoder. No architectural overhead. One model, two jobs.

The authors (Zhao, Cao, Wang, and Gui) demonstrate this on both LLMs and VLMs, achieving simultaneous gains in accuracy and token efficiency while making the black box transparent. The paper lands at an interesting moment: latent reasoning models like Coconut and DeepSeek's R1-style internalization are gaining traction, but the field has mostly accepted the interpretability trade-off as a cost of doing business. SELR says it doesn't have to be.

What Problem Does This Solve?

Chain-of-Thought reasoning works in the language space. The model generates text tokens that represent reasoning steps. This is interpretable by construction — you can read the trace. But it's verbose. Most tokens serve textual coherence, not reasoning. The discrete vocabulary constrains the information density of each token.

Latent reasoning sidesteps this by feeding the final hidden state (a "continuous thought") back as the next input embedding instead of projecting it through the vocabulary. The model thinks in vectors. This is dramatically more token-efficient: a few latent tokens can encode what takes 150+ text tokens to express. But you cannot read a vector. The thinking becomes opaque.

Existing approaches make a choice: either accept the black box (Coconut) or train a separate, larger decoder model to explain the latent thoughts (Heima). Heima introduces significant parameter overhead — its decoders are based on LLaMA3.1-8B, more than twice the size of the backbone it's explaining — and the explanation is decoupled from the actual reasoning process.

SELR rejects the trade-off entirely.

What's the Method?

The core insight is a multi-task training objective that optimizes for two goals simultaneously:

Answer Loss — standard cross-entropy on the final answer tokens, conditioned on the latent reasoning trajectory. This ensures the latent thoughts are actually useful for solving the task.

CoT Loss — cross-entropy that trains the same model to decode its own latent representations back into human-readable reasoning steps. The model learns to translate its vectors into language.

graph LR
    subgraph Training
        A[Image + Question] --> B[Latent Thoughts]
        B --> C{Answer Loss}
        B --> D{CoT Loss}
        C --> E[Correct Answer]
        D --> F[Human-Readable Reasoning]
    end
    subgraph Inference
        G[New Input] --> H[Latent Reasoning]
        H --> I[Answer]
        H --> J[Decode Latents → Text]
    end
    style E fill:#1a3a1a,stroke:#4ade80
    style F fill:#1a1a3a,stroke:#818cf8
    style I fill:#1a3a1a,stroke:#4ade80
    style J fill:#1a1a3a,stroke:#818cf8

The CoT Loss does double duty. It provides a rich text-based supervisory signal that guides the formation of the latent thoughts (solving the supervision problem — how do you supervise a vector?). And it forces the model to learn latent representations that are inherently aligned with human logic (solving the interpretability problem). The model generates token-efficient continuous thoughts that it can later translate on demand.

The authors explore two training strategies: a single-stage approach where both losses are applied simultaneously, and a multi-stage curriculum where the model gradually shifts from language-space to latent-space reasoning. The multi-stage variant, using either uniform or exponential sampling of CoT steps, performs best overall.

What Are the Results?

The numbers tell a clear story. On VLMs (Qwen2.5-VL-3B fine-tuned on LLaVA-CoT-100k):

On LLMs (tested across GSM8k, SVAMP, GSM-Hard, and other benchmarks):

The 6-token-versus-150-token comparison is where SELR makes its strongest case. The gap between SELR and CoT-SFT is not a reasoning quality gap — it's a reasoning budget gap. SELR packs more reasoning per token because it operates in a continuous space unconstrained by discrete vocabulary.

What Are the Limitations?

Three worth flagging:

MMVet degrades. SELR's multi-stage variants score lower on MMVet, driven by compromised multi-step reasoning (e.g., math sub-problems) and long-form generation (e.g., free-form writing). This is inherent to latent compression — you're trading verbose exploration for compact encoding. Tasks that genuinely need extended generation suffer. Heima degrades more severely on MMVet (34.26 vs 36.10), so SELR is less bad, but the degradation is real.

Training data dependency. SELR relies on ground-truth CoT annotations. The LLaVA-CoT-100k dataset's rigid three-step format (summary → caption → reasoning) introduces a format shift that actually hurts the base model's zero-shot performance. The paper acknowledges this — the SFT baseline performs worse than the original model because the structured format forces multi-stage reasoning even for simple questions.

Scale validation is thin. The authors validate VLM generalization at 7B in the appendix, but the core experiments are at 3B. The LLM experiments use a 1.5B student model. Larger-scale behavior is asserted but not deeply explored.

Why Should Someone Building Things Care?

If you're building reasoning agents, you face a choice today: use explicit CoT (interpretable, expensive, slow) or latent reasoning (fast, efficient, opaque). SELR is the first method that lets you have both from a single model. No separate decoder infrastructure. No post-hoc explanation that might not reflect actual reasoning.

The practical implications:

Debugging gets real. When your agent produces a wrong answer, you can now decode its latent thoughts and see what it was "thinking" — in the same vector space where the reasoning actually happened. This is fundamentally different from post-hoc explainability, which generates plausible-sounding but potentially misleading narratives.

Latency budgets get tighter. 8.7× speedup with no accuracy loss (and sometimes a gain) means you can run more reasoning steps in the same time budget, or deploy on smaller hardware. For production systems, this is the difference between a model that's too slow and one that ships.

The architecture is simpler. One model, one training run, no separate decoder to version, deploy, and monitor. The SELR approach means interpretability is a property of the model, not an add-on.

The paper closes with an honest admission: the MMVet degradation shows that latent compression has limits. Tasks that genuinely need long-form generation still need explicit CoT. But for the vast majority of reasoning tasks — classification, QA, math, visual reasoning — SELR's approach is a clear win. The model thinks faster, and you can read its mind.

Related reading

Source: