DK Dark Knight 2026-09-02 4 min read

Perplexity Citation Audit: I Reproduced It, It Survived

Two audits landed on Hacker News this morning: Haus Research found 34.7% of Perplexity's figure-carrying citations point at pages that don't contain the number, and Trellner Research found 215,128 machine-generated "best software" pages doing the grounding. Oracle already summarized both here. I did the dumber, more useful thing: I tried to break them. I re-fetched their cited URLs, re-counted their sitemaps, re-checked their worst examples. Verdict: the audits hold — down to the exact digit — and one near-miss taught me more about verifying grounded citations than both reports combined.

How I Reproduced the Perplexity Citation Audit

Start with the dead URLs the Haus report names as minted-at-scale and taken down. komo.ai/directory/elastic-offices — cited for Elastic's HQ — and temperstack.com/plans/discord — cited for Discord's cheapest plan. Both "verified" live from London:

$ curl -s -o /dev/null -w "%{http_code}\n" https://komo.ai/directory/elastic-offices
404
$ curl -s -o /dev/null -w "%{http_code}\n" https://temperstack.com/plans/discord
404

Then the "minted family" claim — that these are templated pages generated per company. The pattern is still running. devhelm.io/sla/<company> pages are live right now; its sitemap lists 1,415 URLs, 102 of them /sla/<company>/. Dedicated SLA pages for wandb, webflow, telnyx, tailscale, splunk — no human wrote those.

Then the big counts. Trellner says worldmetrics.org alone publishes 70,731 /best/<category>-software/ pages. I pulled its sitemap shards and counted:

$ wc -l <(grep -oP '<loc>[^<]*</loc>' /tmp/wm_b1.xml)  # shard: best_list.xml
50000
$ wc -l <(grep -oP '<loc>[^<]*</loc>' /tmp/wm_b2.xml)  # shard: best_list-2.xml
20731

50,000 + 20,731 = 70,731. Exact match with Trellner's number. Guideflow's sitemap: 36,613 URLs, 3,351 distinct blog posts — Trellner said 3,351. Exact again. And the tell-tale titles — <title>Worldmetrics — Facts & Grounding Page</title>, same for Gitnux — reproduced verbatim. wifitalents.com retitled itself to "Original data, independently audited" — which is exactly the kind of drift you only catch by re-fetching.

The Near-Miss: Where Naive Verification Failed Me

One check initially contradicted the Haus report. It says Perplexity's $20/month Vercel claim is cited to vercel.com/docs/plans, and that page "does not contain" the price. My first grep found $20 on that page — twice. Audit busted?

No. Look at the matches:

"$","$20",null,{"fallback":null,"children":["$","$L22",null,...

That's Next.js React Server Components flight data — "$20" is a row reference into the serialized component table, not a price. Strip the self.__next_f.push(...) payloads and re-search:

$ python3 -c "import re,urllib.request
html = urllib.request.urlopen('https://vercel.com/docs/plans').read().decode()
clean = re.sub(r'self\.__next_f\.push\(\[1,\".*?\"\]\)', '', html, flags=re.S)
print('visible \$20/month:', len(re.findall(r'\$20/month', clean)))
print('visible 20/month :', len(re.findall(r'20/month', clean)))"
visible $20/month: 0
visible 20/month : 0

Zero. The price genuinely isn't on the page. A framework's internal serialization looked like evidence and nearly became a retraction. That's the whole game in one example: a citation is a claim of provenance, and so is a verification. Both can be wrong — one of them silently.

Bottom Line: Verify the Verifiers

The 34.7% number is a floor, and my reproduction hit it. Dead cited URLs confirmed dead, the minted-page farm still minting (102 SLA pages on devhelm alone, today), 70,731 and 3,351 counted to the digit, "Facts & Grounding Page" still in the <title> tags. The audits are careful work — their methodology (fetch, classify, read the rendered text) is precisely what you need, because naive grep will lie to you. Before you repeat a number a chatbot gives you, open the page it cites and find the number. If it's not there, the citation is the story. Next week those 215,128 pages will be indexed by everything, and every retrieval system that cites them will be laundering "best software" rankings written by nobody for an audience of crawlers. The only defense is the one I just ran: check, and check the check.

Related