K2 Horizon 0.9B Hands-On: IFM's Open Mini-Model, Tested

IFM (the LLM360 people) dropped K2 Horizon today: a fleet of six Apache-2.0 models claiming "state of the art at every scale" — including a 0.9B that allegedly scores 48.5 on AIME 2026. That number is absurd on its face. Sub-1B models don't do competition math. So instead of reading the press post twice, I cloned their repos, read the actual modeling code, downloaded the official GGUF, discovered it doesn't load in any stock tool, patched llama.cpp by hand until it ran, and put the 0.9B through real problems on a 2-core CPU box. Here's what the open release actually is — and where the hype starts leaking.

What K2 Horizon Actually Ships

Six models, one family: 375B-A23B, 36B-A4B, 32B, 7B, 3.7B, 0.9B. They share a vocabulary (64,256 tokens), architecture, and training pipeline. The flagship claim: the 0.9B, 3.7B, and 7B are each "state of the art at their scale." The 3.7B card lists SWE-bench Verified 68.6 — a number that was frontier-class for 70B models eighteen months ago. The 7B has a 524K context. If any of this is true, it's because the small models are distillation concentrate: the appendix says the 0.9B is a multi-teacher on-policy distillation (mOPD) run trained against math/code, STEM, and instruction-following teachers, with ~10 trillion synthetic tokens in pretraining. Small model, big teachers.

The Release Is Rougher Than the Press Post

The blog calls it "the most comprehensive open release to date" — full training lifecycle, intermediate checkpoints, training code, fine-grained logs. On day one, that's partially a promise, not a delivery. The ifm-ai/horizon-post-train repository — the thing that's supposed to contain the post-training code — is an empty stub: LICENSE, .gitignore, and a README that reads "Stay tuned! Something is on the Horizon." Weights are up, recipes are not.

Then there's the GGUF. The official 0.9B GGUF repo ships one file: K2-Horizon-1B-BF16.gguf (2.16GB, BF16). I built llama.cpp from master and pointed it at the file:

$ llama-cli -m K2-Horizon-1B-BF16.gguf -p "hi"
llama_model_quantize: failed to quantize: unknown model architecture: 'k2-horizon'

llama.cpp has 149 registered architectures. k2-horizon isn't one of them, there's no support PR open, and the GGUF's kv keys are namespaced k2-horizon.*, so even a naive rename fails. Every "quantization support" claim in the announcement routes through tooling that doesn't exist yet. The models work — if you use vLLM or SGLang from unreleased PR branches, or wait.

How I Got It Running Anyway

The 0.9B's tensor layout is a dead-on match for a dense Qwen2-style transformer: attn_norm, attn_q/k/v, attn_output, ffn_norm, ffn_gate/up/down per layer, 28 layers, 32 heads / 8 KV heads, head_dim=64. The only real novelties live in the 36B MoVA model (more below). So I patched llama.cpp locally: registered LLM_ARCH_K2HORIZON as an alias of the qwen2 implementation, mapped the k2-horizon pre-tokenizer to the Qwen2 digit-splitting one, and fixed the Q/K/V width math for the non-standard head dim. Rebuilt, requantized:

$ llama-quantize K2-Horizon-1B-BF16.gguf k2h-1b-q4.gguf Q4_K_M 2
model size  = 2056.83 MiB (16.00 BPW)
quant size  =  632.77 MiB (4.92 BPW)

Then it ran, on two Xeon cores, at ~14–16 t/s. No GPU, no vLLM, no unpublished branch. A day-one open release should not require a source patch to execute — but the model itself is a legitimately well-behaved citizen once you bridge the gap.

The 0.9B on Real Problems

I gave it three problems. The model emits [Start thinking] … [End thinking] reasoning segments, then the answer.

Test 1 — algebra word problem (30 animals, 74 legs, how many chickens): solved it cleanly — sets up C + K = 30, 2C + 4K = 74, derives 23, then checks its own answer: 46 + 28 = 74. Correct, and the self-verification is the sign of real distillation quality.

Test 2 — Fibonacci digit sum (a₁=a₂=1, sum of digits of a₂₅): correctly identifies F₂₅ = 75025, digit sum 19. Full arithmetic shown, no errors across 25 sequence steps.

Test 3 — fizzbuzz (write the function): this is where the smallness leaks. The Qwen2 pre-tokenizer I mapped splits digits, so my prompt's fizzbuzz(15) arrived as fizzbuzz(1 5). Instead of recovering, the model spent 400+ tokens of meta-deliberation arguing about whether "1 5" meant 1, 5, or 15 — and never emitted the code. A 70B would have laughed at a typo. This one got lost inside its own head.

That's the honest picture: real math competence, fragile robustness. The reasoning pipeline is genuinely good; the tokenizer boundary handling and ambiguity recovery are 0.9B-budget. On clean inputs it's remarkable. On dirty inputs it spirals.

Reading the AIME 48.5 With Skepticism

The 48.5 AIME 2026 number is the headline. Before you believe it, read IFM's own appendix — which is refreshingly honest: "AIME results have high sampling uncertainty because each benchmark contains only 30 problems." AIME 2026 is 30 problems. A handful of correct samples moves the score by a point each. And the appendix notes scores "depend on prompt templates, reasoning effort, and sampling" — the eval used the training rollout path with huge output budgets (they recommend up to 32,768 output tokens). So: 48.5 is real in their harness, and meaningless as a headline without the harness. My two local math tests passed, which is consistent with a strong small model — but one algebra problem is not AIME, and 30 problems is barely statistics.

The 36B-A4B is the genuinely interesting piece: MoVA (Mixture-of-Value-Attention) routes attention values through 64 experts (top-4, sigmoid router, softplus post-gate) instead of the usual MoE-on-FFN pattern — 36B total, ~4B active. I read the implementation in modeling_k2_horizon.py. It's real code, cleanly derived from the Qwen3MoE modular, with FlashAttention-compatible sparsity. That's a legit research contribution buried under a marketing release.

Bottom Line

K2 Horizon 0.9B is the first sub-1B open model I've run that does multi-step math with visible, self-checking reasoning — that's an achievement distillation deserves credit for. But the release overpromises: the post-training code repo is a stub, the official GGUF doesn't load in any shipped tool, and the AIME headline carries error bars IFM's own appendix admits to. Download the weights, patch nothing, and plan on the vLLM PR branch. If you want the 48.5, trust the harness it was measured in — not the tweet-sized version of it.