al Alfred 2026-09-09 4 min read

LLM Bias Audits: 0 of 36 Effects Survive 40,726 Tests

A published benchmark reported that language models favor minority applicants when rating requests one at a time and penalize them when ranking side by side — a format flip that, if general, would explain years of contradictory fairness audits. Vohra and Ravikiran tested exactly that hypothesis in three domains regulators actually watch: hiring, lending, medical triage. 40,726 model requests, five models, applications differing only in the applicant's name, and a primary test frozen before any data collection. The flip does not appear. None of the 36 planned contrasts survives multiple-comparison correction. The single largest measured effect belongs to the audit itself: being listed first.

The problem: audit verdicts that contradict each other

Fairness audits of LLM decision-making are a mess of opposite signs. Correspondence-style audits (fictitious applications, race/gender signalled by names) have reported favoritism toward minorities, penalties against them, and direction reversals across model generations — often under identical experimental designs. A 2026 charitable-aid benchmark (FairFund) offered a clean explanation: the same models flip sign depending on whether the task is one-at-a-time rating or side-by-side ranking, and disparities inflate several-fold when the audit is camouflaged. If true, the entire disagreement in the literature is an instrument artifact. The catch: that evidence came from one domain — charitable aid — that no regulator scrutinizes. This paper asks whether the flip generalizes to domains with legal stakes, where it actually matters.

The method: pre-registered factorial, bounded nulls

The authors took 12 demographic-neutral base profiles per domain from the public AgentFairBench release (résumés, credit files, emergency presentations), rendered each under 40 validated name pairs (five per race-by-gender cell), and had five models judge them three ways: 1–5 rating, binary decision, and four-candidate ranking. The ranking arm is the clever part — a factorial design crossing demographic contrast (present/absent) with profile composition (matched/disguised). The disguised cell co-varies quality across Graeco-Latin squares so the demographic effect is identified statistically while no single prompt reads as a bias test. They also planted known disparities into triage rankings to measure their detection floor, and reran the original FairFund stimuli unchanged.

mindmap
  root((Audit formats))
    Rate 1-5
      Advantage +0.047 pts
      ~half of published +0.090
      survives in 2 of 5 models only
    Decide yes/no
      All contrasts null
      Tool-call form identical
    Rank 4-candidate
      Transparent bundles 100% ties
      Disguised +0.03 positions
      Hiring floor 0.052 excludes 0.067
      Lending/triage floors too wide
  Instrument effects
    Position bias +0.11 positions
    Audit recognition 89-100%
  Detection checks
    Planted disparities recover
    FairFund stimuli directional match

Results: a null with teeth, and one uncomfortable positive

The headline format-by-group interaction is +0.01 [−0.15, +0.17] — a null, but one wide enough to hide the published effect, which is why the equivalence machinery matters. What gives this paper force:

The robustness arm may be the most quietly valuable section. A wording change appeared to reverse the demographic contrast by −0.32 raw — until they noticed the two wording arms saw different applications (Black names averaged 2.96 at baseline, 3.39 for others). Fitted properly as an interaction, it's −0.01. The exact error these audits exist to catch, committed inside their own paper, caught by composition checks.

Limitations the authors disclose — and what they can't fix

Names are the only demographic signal, so nothing transfers to dialect, photos, or explicit statements; 40 name pairs and 12 base profiles per domain cap generalization hard. Five models is a sample of five, not of models. The null refutes a tie-permitting ranking instrument — the original audits force strict orderings, and the authors concede part of the published reversal may live precisely in that forcing, meaning their design can't kill the claim entirely. Every null is bounded, not absolute: lending and triage ranking floors sit above the published effect size. And the directional replication on FairFund materials crosses zero — it localizes the anomaly to those materials without explaining what carries it. Temperature-0 primary runs also don't match what deployed reasoning-enabled workflows run, though the exploratory rerun shrinks the surviving rating effect further.

Why builders should care

If you're wiring an LLM into screening, credit, or triage, this paper's core claim is not "models are fair" — it's that a fairness certificate is only valid for the exact format you tested. Rate vs. Decide vs. Rank, tool-call vs. text, tie-permitting vs. forced ordering: a node cleared under one configuration carries no guarantee for the others. Operationally: test the format your workflow actually calls (their tool-call arms show serialization changes nothing, so at least that axis is safe), always permit ties, and add a planted-disparity control — their planted effects recovered at 0.25 SD and above but not at the sizes actually in dispute, so your audit is blind below roughly a quarter of a standard deviation. The cheapest anti-position-bias fix remains shuffling list order; +0.11 positions for first-listed is larger than any demographic effect they measured. "The audit decides the verdict" isn't rhetoric — it's a warning that your evaluation harness is a decision node too.

Related Posts