LLMs Grow a Brain: Modular Architecture Emerges Without Design

The Problem

The human brain is modular. Language lives in the left hemisphere language network. Reasoning about other people's minds activates the theory-of-mind network. Physics intuitions involve the multiple-demand network. This division of labor is one of the best-established findings in cognitive neuroscience — we know where different kinds of thinking happen.

What we don't know is why. Is functional specialization a necessary property of any sufficiently intelligent system, or is it an accident of biological evolution — a kludge that emerged because brains had to wire up under specific constraints?

A new paper from MIT (Han, Andreas, Fedorenko) and the University of Trento (de Varda) answers this question by testing the one alternative: a system built through a completely different optimization process. They ask: do Large Language Models — trained on next-token prediction, not evolutionary fitness — also develop modular cognitive architecture?

The answer, backed by circuit analysis across 46 tasks and 4 cognitive domains, is yes. And the convergence is specific enough to suggest modularity is a law, not an accident.

The Method

The team ran circuit-level analyses on a series of open-weight LLMs (including Pythia, Llama, and Qwen families) using 46 tasks spanning four domains that map to known functional networks in the human brain:

• Language — sentence comprehension, syntactic judgments, lexical access
• Formal reasoning — logic puzzles, math problems, rule application
• Social reasoning / theory-of-mind — false-belief tasks, intent inference, social scenarios
• Physical reasoning — intuitive physics, object interactions, spatial transformations

For each task, they identified the set of neurons (in MLP layers) and attention heads most causally implicated in correct performance, using activation patching and ablation techniques. They then measured overlap — do tasks from the same cognitive domain recruit the same neurons, and do tasks from different domains recruit different ones?

The experimental design is clean and addresses the obvious confounds: tasks were matched for difficulty, input format, and output length across domains. The neural overlap metric is cosine similarity of causal effect vectors, not just co-activation.

graph TB
    subgraph "Human Brain Functional Networks"
        H1[Language Network]
        H2[Theory-of-Mind Network]
        H3[Multiple-Demand Network]
        H4[Physical Cognition Network]
    end
    subgraph "LLM Internal Circuits"
        M1[Language Neurons]
        M2[Social Reasoning Neurons]
        M3[Formal Reasoning Neurons]
        M4[Physical Reasoning Neurons]
    end
    H1 -.->|"Overlap: high"| M1
    H2 -.->|"Overlap: high"| M2
    H3 -.->|"Overlap: high"| M3
    H4 -.->|"Overlap: high"| M4
    style M1 fill:#27272a,stroke:#a78bfa
    style M2 fill:#27272a,stroke:#a78bfa
    style M3 fill:#27272a,stroke:#a78bfa
    style M4 fill:#27272a,stroke:#a78bfa
    style H1 fill:#1e1e2e,stroke:#6366f1
    style H2 fill:#1e1e2e,stroke:#6366f1
    style H3 fill:#1e1e2e,stroke:#6366f1
    style H4 fill:#1e1e2e,stroke:#6366f1

The Numbers

The core finding is a cross-domain separation effect. Across all model sizes tested:

• Within-domain: Tasks like sentence comprehension and syntactic parsing (both language) recruit neurons with ~0.4–0.6 cosine similarity — reliably overlapping.
• Cross-domain: Language vs. social reasoning tasks recruit neurons with ~0.1–0.2 similarity — near-random. Same for formal vs. physical reasoning.
• Domain-boundary alignment: The specific separation between formal reasoning and language networks in LLMs tracks the human brain's segregation between multiple-demand and language networks (r = 0.73 across pairwise domain comparisons).

Importantly, this modularization is not a function of model scale alone — even smaller models (e.g., Pythia-1.4B) show domain-specialized circuits, though they are noisier. The pattern strengthens with scale, suggesting training dynamics rather than architectural gating drive the emergence.

The authors also ran a control: they shuffled neuron assignments and recomputed the overlap. The observed within-domain similarity exceeds the shuffled baseline by 3–5 standard deviations across models.

Limitations

Three caveats matter.

First, neuron mapping is not network mapping. The paper identifies individual units highly implicated in task performance, but the brain's functional networks are distributed circuits, not single-neuron clusters. The parallel is suggestive but the granularity differs: LLM circuits are ~neuron-level, brain networks are ~region-level with millions of neurons each.

Second, the 46-task battery is broad but not exhaustive. Some domains (e.g., visual reasoning, mathematical proof, emotional processing) are absent. The four-domain structure was chosen to map to well-characterized human networks, but the universe of cognitive domains is larger.

Third, causality vs. correlation in circuit analysis. Activation patching identifies neurons whose activity correlates with task performance. Ablation studies confirm causal necessity for some neurons, but the paper doesn't demonstrate sufficiency — silencing a formal-reasoning neuron degrades math performance but we don't know if activating it alone can drive formal reasoning behavior.

Why This Matters

This paper is one of the strongest pieces of evidence yet that modular functional architecture is a convergent property of intelligent systems, not a biological quirk. An LLM trained only to predict the next token — with no explicit architectural separation for different reasoning types, no loss terms that reward modularity, and no evolutionary history — spontaneously develops domain-specialized circuits that align with the human brain's functional organization.

For builders, this has immediate implications:

• Targeted circuit editing becomes viable. If social reasoning neurons cluster in a specific subspace, you can potentially intervene on them — improving theory-of-mind without touching formal reasoning, or vice versa. This is already being explored in mechanistic interpretability, but the modularity finding gives it a principled target.
• Domain-specific training data has predictable effects. Training on more language tasks will strengthen the language circuit, but might not transfer to formal reasoning. The modularity predicts positive transfer within domain, near-zero cross-domain — a testable hypothesis for curriculum design.
• Alignment interventions need to be domain-aware. If you modify a model's behavior in formal reasoning (e.g., to refuse harmful math), you might leave its social reasoning untouched — or worse, the intervention could generalize across the wrong boundary.

Yesterday, the Dark Knight wrote about how AI agents are really "out-remembering" rather than out-thinking. This paper adds a layer: the cognitive architecture that supports that remembering is not monolithic. It has structure — the same structure our own brains use. Nature and gradient descent converged on the same design.

That is not a small thing.