The Artificial Experimentalist

Science has a blind spot. When we study complex systems — cellular automata, biological morphogenesis, ecological networks — we mostly run them open-loop: set initial conditions, let them simulate, observe the outcome. No intervention during execution. No real-time steering.

A gardener doesn't just plant seeds and wait. They prune, water, reshape. The system and the gardener co-evolve. A new paper from Inria Bordeaux, Tufts, and Inria Lyon introduces CARL (Closed-loop Autotelic Reinforcement Learner), an agent that treats complex systems the way a gardener treats a garden — with continuous observation and minimal, targeted intervention.

What problem does this solve?

Existing methods for exploring cellular automata and complex systems are fundamentally passive. They search initial condition space, run simulations, and page through results. This works for small parameter sweeps but breaks down when the system has high-dimensional state, nonlinear feedback, or emergent dynamics that depend on the intervention history itself. You can't discover what only exists under continuous perturbation by running open-loop simulations.

CARL frames the problem as closed-loop autotelic reinforcement learning: an agent that autonomously samples diverse goals and learns a goal-conditioned policy to steer a complex system through minimal local perturbations.

What's the method?

The framework is instantiated on Lenia, a continuous cellular automaton famous for producing life-like self-organizing patterns (solitons that move, split, and persist). CARL uses a Double DQN with a U-Net architecture — fully convolutional, producing dense Q-value maps over the grid for each action type (add material, remove material, no-op). The agent doesn't know what a "soliton" is. Instead it's trained on a proxy task: maintain a target mass. When actions are costly, the agent faces a tradeoff between constant intervention and finding a self-sustaining configuration. Action costs tip the balance toward discovery.

            mindmap
              CARL Framework
                Perception
                  Grid state observation
                  Rule parameters
                  Action cost
                Autotelic Goals
                  Self-sampled target masses
                  Diverse update rules
                  Randomized initial states
                Policy (DDQN + U-Net)
                  Dense Q-value maps
                  Per-cell action selection
                  Local perturbations only
                Outcomes
                  Soliton discovery
                  Direction control
                  Human-in-the-loop steering
            

What are the results?

Three demonstrations, each significant:

1. Soliton creation. Trained across 85 Lenia update rules, CARL discovers stable solitons at a higher rate than heuristic baselines. The agent generalizes zero-shot to unseen kernel radii (ρ=6 → ρ=16, a 4× grid size difference), modified action parameters, and entirely novel update rule families. This means a single trained policy can explore new dynamical regimes it has never seen.

2. Soliton direction control. CARL learns to steer the movement direction of existing solitons with few interventions. This is distinctly harder than creation — the agent must apply precisely timed nudges to a system already in motion, compensating for nonlinear dynamics.

3. Human-in-the-loop. Trained agents can be deployed in real time as intermediaries. A human specifies high-level directional commands ("move left", "go right") and the agent translates them into low-level perturbations — effectively giving a human operator the ability to guide solitons through maze environments without needing to understand Lenia's update rules.

Why this matters

The paper's broader bet: that autotelic RL can produce artificial experimentalists — agents that autonomously discover and control emergent phenomena in any complex system. The authors explicitly connect this to biomedical goals: controlling morphogenetic patterns in bioelectric and mechanical fields, or steering collective cell behavior. Michael Levin's presence on the author list is a strong signal — his lab has long argued that the body's multi-scale architecture comprises persistent dynamical patterns (bioelectric, mechanical, metabolic) that move, resist erasure, and reshape their surroundings. A computational pipeline that infers effective stimuli to achieve target outcomes in complex emergent systems is exactly the tool that approach demands.

Limitations

Lenia is a sandbox, not biology. The action space (add/remove material in a radius) is simple and privileged — real biological systems don't expose clean intervention primitives. The soliton detection heuristic (mass < 10% of grid capacity after 5000 steps) is crude and would miss many interesting non-soliton patterns. The U-Net policy architecture, while effective, may not transfer to systems without natural spatial structure. And the proxy task (mass maintenance) works because solitons happen to have lower mass than diffuse patterns — it's indirect discovery, not direct targeting.

Still, as a proof of concept for closed-loop discovery in complex systems, this is the most interesting thing I've seen from the Lenia ecosystem. The combination of autotelic RL, Michael Levin's theoretical biology framing, and the generalizable architecture makes this worth watching — especially if they extend it beyond cellular automata.