Ecdysis: Agent Harness Evolution 1.84x Faster, 18.56% Better

The fastest-moving frontier in agent engineering isn't model weights — it's the harness: the scaffolding of prompts, tools, retry logic, and control flow wrapped around an LLM. The new paradigm is harness evolution — let the harness rewrite itself against task feedback. The problem, per a paper out yesterday (Ecdysis, arXiv:2609.11677), is that every existing harness-evolution loop pays for its improvement in the most expensive currency available: repeated full agent executions, and it quietly overfits the harness to the tasks it saw.

The problem: per-failure optimization breeds model-specific hacks

Iterative harness search works like genetic algorithm over code: run tasks, watch failures, patch the harness, repeat. Each failure becomes a prompt patch. But a failed run has two possible parents — a model-specific deficiency (this model just can't do long division in-context) and a systematic harness deficiency (no agent with this tool schema could recover from the auth failure). Naive evolution can't tell them apart, so it patches the harness to accommodate the model's weakness. Each patch looks like progress on the training tasks and quietly narrows the harness's applicability to unseen ones. That's textbook overfitting, wearing a scaffolding costume.

The method: diagnose failures in aggregate, not one at a time

Ecdysis's core move is batch-level cross-instance failure aggregation: collect failure evidence from many task instances at once, then look for failure patterns that recur across tasks. Recurring across tasks = systematic harness deficiency → repair the harness. One-off = model-specific → deliberately do not accommodate it. A second mechanism, Failure-Driven Collaborative Refinement, runs multi-role diagnosis (separating cause-finding from patch-specification) and iterates on the modification spec rather than the raw patch. The effect: fewer agent executions per unit of harness improvement, and patches biased toward the failures that actually generalize.

mindmap
  root((Ecdysis))
    Old loop
      Run tasks, patch per failure
      Can't tell model weakness from harness flaw
      Overfits to observed tasks
    Ecdysis loop
      Aggregate failures across tasks
      Recurring pattern ⇒ harness repair
      One-off failure ⇒ no accommodation
      Multi-role diagnosis + spec refinement
    Payoff
      1.84x faster harness training
      +18.56% accuracy on resulting harness
            

The results: receipts

• 1.84x speedup in harness training versus existing harness-evolution methods — fewer repeated agent executions, which is where all the wall-clock and token cost lives.
• +18.56% reasoning accuracy for the resulting harnesses — so the speedup isn't bought by being sloppy; the harnesses are simultaneously faster to produce and better.
• The paper frames this as the same quantity improving in opposite directions: prior methods burn more compute to reach worse generalization. Both ends of that tradeoff get fixed at once here.

Limitations the abstract glosses over

Three things to hold. First, the whole approach rests on a classifier of failure causes — recurring-vs-one-off — and the paper doesn't fully resolve what happens when a failure is genuinely mixed (a systematic harness gap that only the weakest models fall into gets patched as model-specific, and vice versa). Second, cross-task aggregation needs a batch of similar tasks; in low-task-diversity regimes the signal for "systematic" gets thin. Third, the headline accuracy number is on the benchmark suite used for evolution; the abstract claims better generalization to unseen tasks, but the magnitude of that unseen-task gain relative to the 18.56% isn't the headline — read the tables, not the title. And like all harness-evolution work, it inherits a moving floor: harness patches that help today's model can actively hurt the next checkpoint.

Why builders should care

If you're running any self-improving agent pipeline — DSPy-style optimizers, harness search, agent-memory loops — this paper's diagnosis is directly transplantable: separate your failure taxonomy before your patch taxonomy. The single most common bug in hand-rolled agent loops is a system prompt accreted from patches against one model's failures; Ecdysis is a formalization of why that rots. The cheaper alternative it implies for small teams: keep a per-failure ledger tagged with the model version, and only modify the harness when a failure pattern crosses task boundaries. You get maybe a third of the benefit for none of the infrastructure. The full loop matters once you're running enough executions that the 1.84x speedup compounds.