Entity Tracking Emerges at 410M Parameters — Smaller Models Are Smarter Than We Thought

There's a narrative in AI research that linguistic capabilities scale with parameters — that small models are good at pattern matching but bad at understanding. A new paper from Drożdż and Heilbron (University of Amsterdam) published on arXiv yesterday (2608.18083) puts a crack in that story. Using naturalistic narratives and direct human comparison, they show that entity tracking — a core component of language understanding — emerges at just 410 million parameters and exceeds human performance at scale.

---

What Is Entity Tracking?

Entity tracking is what lets you read a paragraph and know that "she" refers to the doctor, not the patient, and that the suitcase is now in the car, not the bedroom. It's keeping a mental map of who did what to whom, where things are, and how they change — even when those changes aren't explicitly restated.

This is trivial for humans but famously hard for models. Prior work claimed that serious entity tracking required multi-billion parameter, code-specialised models. Drożdż and Heilbron tested this with a cleaner experimental design: naturalistic narratives (not artificial probe sentences), multiple levels of narrative complexity, and direct comparison with 48 human participants.

graph TD
    subgraph "Entity Tracking Task"
        N[Narrative Input] --> H[Human N=48]
        N --> M1[Model 410M params]
        N --> M2[Model 7B params]
        N --> M3[Model 70B+ params]
    end
    subgraph "Key Finding"
        H -->|Performance degrades with complexity| HR[Human ceiling]
        M1 -->|Matches human at 410M| E1[Emergence threshold]
        M2 -->|Exceeds human| E2
        M3 -->|Far exceeds human| E3
    end
    subgraph "Surprise"
        E1 --> S[S: Prior work said need billions + code training]
    end
    style S fill:#7f1d1d,stroke:#ef4444
    style E1 fill:#1e3a5f,stroke:#3b82f6
    

---

The Results

Two findings stand out:

Human entity tracking degrades with narrative complexity, not length. Humans can track entities through a long but simple story as well as through a short one. But add more characters, more location changes, more objects in play — and performance drops. This is intuitive: complexity makes demands on working memory that length alone doesn't.

Models hit human-level entity tracking at 410M parameters. Not 7B. Not 70B. A model with fewer parameters than a typical smartphone assistant matches human performance on this task. And as scale increases, models far exceed human performance — they track entities through narratives that would overwhelm a person.

---

Limitations

The paper tests narrative entity tracking in English, and the naturalistic narratives are controlled for length and complexity but not for genre or domain variation. The human comparison (N=48) is moderate — larger studies might reveal more nuance in the human performance curve. The paper also doesn't probe how models achieve this: is it a specific attention pattern, a property of the training data distribution, or something else? That's left for future work. Finally, entity tracking in narratives is one capability — it doesn't imply broader understanding or reasoning at the same parameter scale.

---

Why Builders Should Care

This finding has practical consequences:

Smaller models are more capable than benchmarks suggest. If entity tracking — a linguistically demanding task — emerges at 410M parameters, then evaluation suites that test primarily on this kind of understanding may be undercounting small model capability. For deployment decisions, this means you might not need the 70B parameter model for tasks that require narrative coherence or long-range reference resolution. A quantized 7B or even a well-tuned sub-billion model could handle it.

Human comparison changes the frame. Most model evaluations are model-vs-model (LLM-as-judge, benchmark scores, reference metrics). This paper includes humans as a baseline, and the result is striking: humans are not the ceiling. For tasks like entity tracking, models are already operating in a regime where they're more reliable than people. If you're building a system that depends on tracking entities across documents or conversations, the constraint is no longer model capability — it's prompt design and context management.

The "scale is everything" narrative gets a useful counterexample. Not everything requires billions of parameters. Some fundamental linguistic capabilities emerge early and improve gracefully with scale. The architecture of a 410M parameter model already contains the mechanisms needed for this kind of understanding — more parameters just sharpen them. That's good news for on-device deployment, edge inference, and anyone who's been told they need a frontier model to handle basic linguistic structure.