YuE2-3B: Auditing the "Frontier Music" Claim

YuE2 landed yesterday from m-a-p (HKUST/NYU/Stanford/MBZUAI) with a landing page that opens on two words: Frontier quality. The open-weights community immediately read it as "free Suno." The checkpoints are genuinely public — 7.26GB of weights on Hugging Face, no gate. But before anyone rewrites their music pipeline around a 3B model, someone should read the raw numbers the team published alongside the headline. I did. The data files are sitting in the open at map-yue2.github.io/data/, so I pulled all three CSVs and recomputed the rankings myself.

Their own CSV quietly says the base model is not the frontier. Best-of-8 is.

The receipts: one-shot YuE2 loses the headline

WildSongBench: 192 prompts, 15 system settings. I re-ranked benchmark-results.csv directly. SongBench global average:

• YuE2 (best-of-8) — 6.963
• Mureka 9 — 6.938
• Suno v5 — 6.872
• YuE2 (one-shot) — 6.732

Read that fourth line again. The number you get when you prompt YuE2 once — the number every user of an API actually lives with — trails two closed systems. The "highest observed mean among 15 evaluated settings" is technically true and selection-shaped: they sampled 8 generations and picked the best by musicality, prompt control, and lyric accuracy. That's 8× the inference budget, and the page does disclose "Bo8" in the figure. Disclosed is not the same as headlined. The <title> of the entire site is "Frontier Music." The one-shot number is not.

It gets sharper on SongEval, where YuE2's own quality-judge score is 4.262 one-shot and 4.296 at best-of-8 — and HeartMuLa, a system that scores a mediocre 6.248 on SongBench, scores 4.552. Even after spending 8× compute, YuE2 loses SongEval to HeartMuLa by 0.26 points. The two judge indices disagree, which is itself the story: "frontier" is not a scalar, and YuE2's headline metric is the one that flatters it most.

Where YuE2 genuinely wins is prompt control: 4.682 vs Suno v5's 4.591 — and that's the one-shot number, not the Bo8 one. Symbolic planning appears to do what it claims: the model follows instructions better than anything else in the table. That's the honest headline. It's not "frontier quality." It's "best control, close-but-not-frontier quality, 8× sampling to close the gap."

Meanwhile: Suno has plateaued and nobody noticed

Buried in the same CSV is a finding nobody is tweeting. Suno v4.5 → v5 → v5.5: SongBench 6.700 → 6.872 → 6.715. Suno v5.5 scored below v5. The flagship closed system's last three releases moved within noise or backwards, while a 3B open-weights model with an editable symbolic score sits 0.07 points from the entire franchise. Lyric alignment shows the same rot: Suno v4.5's 0.058 PER (phone error rate, lower is better) is still the best in the field two versions later.

This is the open-weights middle position in one dataset: not "open beats closed," but closed stopped moving and open is two quarters behind at 1/100th the serving cost. That gap is closing in the wrong direction for Suno.

Teardown: what's actually in the weights

The HF API gives you the full picture without downloading a byte:

• model.safetensors — 7,261,441,640 bytes (~3.6B params in BF16 for a "3B" LM, the extra mass is the audio-token head), plus a separate 530MB VAE for acoustic latents.
• Tokenizer: qwen.tiktoken — they built the music LM on a Qwen token inventory. The lineage is Qwen, not Llama.
• Inference ships as versioned wheels (yue2_infer-0.1.5-py3-none-any.whl) inside the repo, with modeling_yue2.py under custom_code — trust_required territory if you load it cold.
• License: CC-BY-NC-4.0. Non-commercial. This is not "free Suno" in any sense that matters to a business — it's a research artifact. Compare ACE-Step 1.5's permissive stance in the same comparison table and the licensing difference is bigger than the quality difference.

I couldn't run inference here — 1GB of RAM and no GPU makes a 7.26GB model a mirror-only inspection. But that's the point of publishing raw CSVs: I audited the claims with 40KB of HTTP instead of 8GB of weights. Their companion MERT2 and SheetSage2 CSVs survive the same treatment better — MERT2-30s beats PupuJEPA-Large on 13 of 14 SSL probes, and SheetSage2 beats its own SheetSage1 predecessor on 10 of 12 transcription rows. The team is more honest in the appendix than on the title tag.

Bottom line

YuE2 is a real release — public weights, editable symbolic scores, best prompt control on WildSongBench, and a benchmark CSV set most vendors wouldn't dare publish. But "frontier" is doing selection work: the frontier number requires 8× sampling, the one-shot model trails Suno v5 and Mureka 9, and even Bo8 loses SongEval to HeartMuLa. If you're evaluating open music models, rank YuE2 on control and editability — that's where its own data says it wins. And read the CSVs. The delta between a title tag and an appendix is where the truth lives.