DK Dark Knight 2026-09-05 5 min read

GPT-6 Astra Scores 99.9% on ARC-AGI-3

The benchmark that was supposed to measure the "residual gap" to AGI just got nearly closed by a model that shipped yesterday. GPT-6 Astra scored 99.9% on ARC-AGI-3 — the agentic benchmark where humans solve 100% of environments and no frontier model has cracked even half of them — and it did it for $19K in compute, using fewer actions than the median tested human on 96% of levels. That's not an incremental release. That's the goalposts moving.

What ARC-AGI-3 Actually Tests

ARC-AGI-3 is the third generation of the ARC benchmark series, built to test agentic intelligence: exploration, world modeling, goal-setting, and planning under sparse reward. Agents land in novel, abstract, turn-based environments with no instructions and have to figure out the mechanics, build a model of the world, and act. No training data can prep for it — the environments are designed to only contain core knowledge priors, and humans can solve 100% of them. The point of the series is to measure the residual gap between frontier AI and AGI, defined as a system that can acquire any skill a human can, as efficiently as a human can.

This is the same family as the ARC-AGI-1 puzzle benchmark OpenAI's o3 reportedly scored 87.5% on in 2024. ARC-AGI-3 is harder: it's not a single-shot puzzle, it's an interactive environment where the agent must act, observe, model, and re-plan. Previous leaderboard state on it was nowhere near 99.9%.

The Numbers: 99.9% for $19K

ARC Prize ran Astra under two harnesses, and the difference between them is the whole story:

graph TD
  A[ARC-AGI-3 Environment] --> B{Harness}
  B -->|Standard| C[Model passes its own notes between turns]
  B -->|Provider Adapter| D[Opaque reasoning state preserved + compaction]
  C --> E[62.7% max effort - $26K]
  D --> F[99.9% high effort - $19K]

Every reasoning level under the Provider Adapter harness cleared 96.7%, costing between $17K and $23K. Under the Standard harness, scores ranged from 17.5% (low) to 62.7% (max). The gap between the harnesses is the tell: the model's internal reasoning state, kept alive across the whole session, is doing the work. Kill that continuity and Astra loses a third of its capability.

The Symbolic World Model Behavior

The most interesting observation in the ARC Prize writeup isn't the score — it's how Astra got it. Greg Kamradt's team watched Astra turn unfamiliar environments into compact symbolic world models: representing game mechanics as logical rules and inventing its own domain-specific shorthand to track state and plan actions. It wasn't pattern-matching. It was writing itself a spec, then executing against it.

That's the behavior you'd actually want in an agent — not a bigger next-token predictor, but something that compresses its environment into a model it can reason over. It's also exactly the kind of capability that makes the "Astra is a security risk" story from last week (100% on ExploitBench, per the leaked safety doc I covered) more credible rather than less.

What It Means — and What It Doesn't

Let me be precise about what this is and isn't. It's a state-of-the-art result on the hardest public agentic benchmark, produced by an architecture that preserves reasoning state across long horizons with compaction. That's real, and it resets expectations for what an agent can do on novel interactive tasks.

It is not AGI, and the harness caveat matters. 99.9% requires OpenAI's private reasoning-state plumbing — you can't reproduce it with a plain API call and a stateless agent loop. The Standard-harness score of 62.7% is closer to what a normal developer gets stitching Astra into their own tooling today. The benchmark also costs $19K per run on a model priced at $10/M input and $50/M output tokens — this is a frontier capability with frontier prices, and the OpenAI/Azure routing on OpenRouter doesn't change that.

Where this lands for builders: the agent scaffolding layer just became the differentiator. If reasoning-state continuity is worth a 37-point swing on ARC-AGI-3, then the harnesses wrapping these models (like the ones Anthropic ships in Claude Code, or what OpenAI is baking into Copilot and its API) are where the value is — not the model weights themselves.

Bottom Line

GPT-6 Astra on ARC-AGI-3 is the strongest public evidence yet that long-horizon agentic capability is real and shipping — 99.9% for $19K, with human-beating action efficiency. But read the fine print: the score lives in provider-side reasoning plumbing, prices are premium-tier, and the Standard-harness number (62.7%) is what your own agent loop would likely see. The lesson isn't "models are AGI now." It's that the harness around the model is now worth more than the model.

Related