ARC-AGI-1: 44% in 67 Cents With a Small Transformer

Mithil Vakde trained a small transformer from scratch on a single RTX 5090 — 1.5 hours, 67 cents of compute — and scored 44% on the ARC-AGI-1 public evaluation. Same ballpark as dedicated ARC systems like TRM/HRM, ahead of most LLMs, and fully open source (github.com/mvakde/mdlARC). The same model also scores 7% on ARC-2.

What shipped with the 44% ARC-AGI-1 run?

This is the third post in Vakde's series and an upgrade from his previous 40% result. The gains come from plain engineering: SwiGLU + RMSNorm, 8 layers instead of 4, the NorMuon optimizer, and flash attention with varlen training plus flex attention kernels at inference. The notable change is dropping input-token training entirely — the model now trains supervised, on outputs only. Counterintuitively, test loss got worse while scores improved. The ablations say the real load-bearers are the representation choices: 3D RoPE and per-task embeddings. Remove either and the score collapses to ~24%. The union of solved tasks across runs reaches 55%, so he argues 65% is reachable inside the transformer framework.

Why does a 67-cent ARC-AGI-1 score matter?

Dispatch has been tracking the other end of this benchmark family: ARC-AGI-3, where harnesses and scaffolding now determine scores more than models do. This is the opposite coin. No harness, no test-time-compute arms race, no frontier model. One consumer GPU and pocket change beat most LLMs on the original benchmark that carried a million-dollar prize. The signal is that sample efficiency — not scale — is where the interesting headroom is. If a 67-cent transformer can match systems that cost orders of magnitude more, the metric that matters on any benchmark is score per dollar, not score alone.

What are the caveats to the 44% ARC-AGI result?

The approach is transductive: the model trains on eval-puzzle inputs at test time, though never the hidden labels. Vakde argues that is legitimate metalearning — the benchmark is designed so solvers learn from eval puzzles — and the debate has drawn in senior names (Chollet, Howard, Beyer, Rohan Anil). Whether you buy the framing or not, the cost floor is the story. 44% is not AGI; it is a data point on what a plain transformer can do when the representation is right.