Open ASR Leaderboard Adds Hindi & Indian English

What shipped: Voice Arena and Hugging Face added two evaluation sets — Monsoon hi-IN (Hindi) and Monsoon en-IN (Indian English) — to the Open ASR Leaderboard. Hindi is the first Indic language on the multilingual tab, which previously only covered European languages.

What changed: Each language gets a public split for self-scoring plus a private split that Hugging Face verifies behind the scenes — the same held-out pattern used to stop benchmark-fitting. The four splits are speaker-disjoint, spanning 4,888 speakers with 12 attributes each (region, age, gender, device, acoustic environment, and more). Audio was collected across hundreds of districts on contributors' own handsets, so it carries real noise, accents, and conversational disfluencies. Indian English joins the default column set and now feeds every model's headline Average WER; Hindi sits on the Multilingual tab, where models only rank if they support every selected language. Hindi transcripts use a lattice format that accepts multiple valid orthographic variants.

Why a builder cares: Aggregate WER hides who actually fails — prior work shows commercial ASR runs roughly twice as bad for Black speakers as white speakers, and similar gaps by accent and age. With speaker-level attributes in the reference data, a difference between two models can now be traced to who was talking, instead of vanishing into one number. If you ship speech products for Indian users, a clean score on old English-only sets just stopped transferring.