SWE-Prime: When Less Training Data Beats More

The problem

Fine-tuning LLMs on agent trajectory data has become the standard recipe for improving code-fixing ability. The logic seems sound: collect thousands of successful trajectories from SWE-agent runs on real GitHub issues, then supervised fine-tune (SFT) on them. More trajectories → more signal → better model.

But "successful" doesn't mean "good." A trajectory that lands on the right patch might still wander through irrelevant files, make unnecessary edits, or embed risky patterns. Train on enough of these and the model learns not just the destination but the detours.

mindmap
  root((SWE-Prime
Data Selection))
    Trajectory-Level
      Process Quality
      Result Quality
      Representativeness
    Segment-Level
      Contribution Score
      Learnability
      Risk Assessment
    Training
      Full context preserved
      Masked loss on segments
    Outcome
      10% trajectories
      Up to 24% improvement
            

The method

SWE-Prime (Zheng et al., 2026) introduces a multi-granularity, two-stage filtering pipeline that treats trajectory quality as something to measure, not assume.

Stage 1 — Trajectory-level screening. Three filters run over every successful trajectory in the dataset:

• Process quality: Did the agent explore efficiently, or did it flail? Measured by number of files touched, number of edits attempted, and similarity to a reference minimal trajectory.
• Result quality: Beyond binary pass/fail — does the patch introduce dead code, reduce test coverage, or depend on fragile hacks? Evaluated via static analysis.
• Representativeness: Is this issue-type already well-covered? Redundant trajectories get demoted.

The output: ~10% of original trajectories survive into the curated set — and these alone beat the full dataset.

Stage 2 — Segment-level selection. Even within a good trajectory, not every step is valuable. SWE-Prime groups consecutive actions into semantic segments and scores each one on three axes: contribution (how much does this step move the needle?), learnability (is the pattern generalizable or issue-specific?), and risk (could this introduce regressions if imitated?).

During training, all segments remain in the sequence to preserve context, but only selected segments contribute to the loss. The model can attend to the full trajectory while only being reinforced on the high-signal parts.

The numbers

Results are unambiguous:

• SWE-Bench Pro: Training on SWE-Prime's 10% subset yields 12.2% relative improvement over training on the full resolved dataset.
• SWE-Bench Verified: The gap widens to 24.2% — suggesting the filtering is even more impactful on cleaner, more realistic benchmarks.
• Training on all successful trajectories actually hurts performance compared to a no-SFT baseline in some configurations. Noise isn't neutral — it's destructive.

These gains come without additional compute at inference time. This is pure data-quality leverage: better curation, no architectural changes, no larger models.

Limitations

The filtering pipeline itself adds overhead. Computing process quality and risk scores requires static analysis tools and reference trajectories, which may not exist for every codebase. The representativeness filter depends on having a diverse-enough initial pool — if your dataset is already small and homogeneous, cutting 90% leaves nothing useful.

SWE-Prime also operates entirely on the trajectory level. It doesn't address issues in the reward model or environment — a trajectory can be perfectly polished and still solve the wrong problem if the task definition was ambiguous.

Finally, the paper tests on SWE-Bench only. How well these findings transfer to other agentic domains (browsing, data analysis, tool use) is open.

Why it matters

The prevailing instinct in the SWE-agent world is to scale data: collect more trajectories, train on everything that succeeded, trust that quantity washes out noise. SWE-Prime shows this assumption is wrong in practice. More data can make things worse.

For anyone building agent fine-tuning pipelines, the takeaway is practical and immediate: invest in data selection infrastructure, not just data collection. A well-curated 1,000 trajectories beats a noisy 10,000.

This also raises a deeper question about the current SFT paradigm for agents. If successful trajectories contain so much latent noise, what else are we accidentally teaching models to imitate?