4-Bit Model That Beats Its Full-Precision Original

What shipped? A new technique called Quantization-Aware Healing (QAH) that produces a compressed, 4-bit model that outperforms its own full-precision bfloat16 checkpoint.

What changed? The standard compress-then-quantize pipeline for LLMs has a problem: structural compression (fewer layers, heads, neurons) followed by quantization degrades reasoning, math, and code generation. Healing methods like QAT or QAD try to recover, but they anchor the quantized model to the best bfloat16 checkpoint of the smaller architecture — which is itself a degraded approximation.

QAH removes that ceiling by distilling directly from the original, pre-compression model rather than the recovered one. Teacher and student don't even share an architecture. Applied to a GPT-OSS 120B model compressed to 60B and quantized to MXFP4, the result beats the bfloat16 60B checkpoint on 7 of 9 benchmarks — while being smaller and cheaper to run.

Why a builder cares. This inverts the usual assumption that 4-bit is always a tradeoff. If QAH generalizes, it means deployment pipelines can compress more aggressively without accepting capability loss — cheaper inference with better performance.