WikiSkill: The Architecture That Lets Agents Actually Get Smarter Over Time

Every agentic system today faces the same wall: each new task starts from the same blank slate. Skills can be extracted from past trajectories, but the insights that guided those skills — what worked, what didn't, why — remain scattered across log files and optimization histories. Nothing accumulates. Nothing compounds.

A paper from Microsoft Research by Liyan Tang, Cyrus Rashtchian, Chun-Sung Ferng, Andrew Tomkins, Da-Cheng Juan, and Tu Vu proposes a structural fix. Their framework, WikiSkill, introduces a persistent knowledge base that co-evolves with agent skills. Raw execution experience gets distilled into structured wiki entries, which in turn inform skill creation and updates. Three tiers, each feeding the next.

What problem does this solve?

Current skill evolution methods extract reusable patterns from agent trajectories, but they discard the context that made those patterns effective. A skill becomes a black box: "when X happens, do Y." The wiki in WikiSkill preserves the why — the comparative reasoning that led to choosing Y over Z. That context becomes the substrate for future skill development. Without it, each evolution cycle is effectively blind: you're optimizing from scratch every time, with no institutional memory.

This is the difference between a junior developer who keeps a notebook and one who doesn't. The one with the notebook compounds knowledge. The other keeps rediscovering the same lessons.

What's the method?

flowchart LR
    subgraph Experience
        Agent[Agent Execution]
        Trajectories[Raw Trajectories]
    end
    subgraph Knowledge
        Wiki[Persistent Wiki]
    end
    subgraph Skills
        Evolved[Executable Skills]
    end
    Agent --> Trajectories
    Trajectories -->|Consolidation| Wiki
    Wiki -->|Informs| Evolved
    Evolved -->|Guides| Agent

WikiSkill separates three concerns:

1. Raw execution experience — full trajectories from agent runs, captured without curation
2. A persistent wiki — structured entries capturing insights, trade-offs, and the rationale behind skill choices. This is the key innovation: knowledge persists across tasks, sessions, and model versions
3. Executable skills — the function-callable resources agents actually invoke at inference time

The loop is continuous. Experience gets consolidated into wiki entries. Wiki entries inform skill creation and updates. New skill executions produce fresh experience, which feeds back into the wiki. The wiki is never reset.

What are the results?

The paper evaluates across diverse benchmarks and model scales. The headline results:

• WikiSkill consistently outperforms state-of-the-art skill-evolution methods and improves over no-skill baselines in most model-benchmark settings
• Skill evolution complements model scaling: larger models benefit more from evolved skills — the smarter the base model, the more it extracts from accumulated knowledge
• Smaller models with WikiSkill-evolved skills outperform substantially larger models without them — a 1.5B parameter model with the right wiki-driven skills beats a 7B model starting fresh
• Evolved skills transfer across models and model families: skills evolved by one model can be loaded into another architecture with minimal loss
• Skills evolved by other models can outperform self-evolved skills — cross-model evolution sometimes beats in-family optimization
• Ablation studies confirm the wiki is the critical component: remove the persistent knowledge store and performance collapses back to baseline

What are the limitations?

The paper doesn't specify the cost of maintaining the wiki. Consolidation requires LLM calls per experience → wiki entry, which adds latency and API cost compared to direct skill extraction. The evaluation focuses on task-specific benchmarks rather than open-ended agentic loops. And the wiki structure — what counts as a good entry, how entries get pruned, when to consolidate — still involves human design decisions that may not generalize across domains.

The cross-model transfer findings are promising but early. Transfer between close architectures (Qwen2 → Qwen2.5) is cleaner than cross-family (Qwen → Llama), and the paper doesn't probe the lower bound of transfer quality.

Why should someone building things care?

Because the most expensive failure mode in agentic systems isn't a wrong answer — it's starting over. Every time you spin up a new agent task, the system has forgotten everything it learned in the last one. WikiSkill is a concrete architecture pattern for compounding knowledge.

The architecture is framework-agnostic. The wiki is just structured text. The skills are just functions. The design pattern — separate experience, knowledge, and execution — is implementable today in any agent framework. If you're building agents that operate over weeks or months, this is the closest thing to a memory system that actually works across iterations.

The result that small models + skills beat big models cold is the one worth remembering. It means the teams investing in infrastructure for knowledge persistence get a force multiplier that compounds over time, while teams throwing bigger models at the same problems get linear scaling at best. The wiki is the leverage.