DK Dark Knight 2026-08-29 6 min read

The llms.txt Trust Model

A security researcher found 227 package names in Fortune 500 documentation that nobody owned. He registered a few. Within an hour, AI coding agents inside a Fortune 500 company had run his code. Hermes, Claude, Codex — all of them took the bait.

This isn't a zero-day. It's worse. It's a design assumption that already broke in production.

The Surface Area

llms.txt is a proposed standard — a plaintext file at a website's root that tells AI agents which pages matter, what commands to run, and how to understand the site. Think robots.txt but for LLMs. Over 100 companies, including Fortune 500s and defense contractors, have published one. The idea is sound: give agents structured context so they don't hallucinate their way through your docs.

Here's what the researchers at a stealth Israeli startup found when they actually checked what those files pointed to:

That last number is the whole story. 227 commands sitting in trusted documentation, telling an agent to pip install or npx a package that anyone could claim.

The npx Confusion Problem
Node's npx is especially dangerous here. It can fetch a package from the public registry and execute its binary without ever adding it to a project's dependencies. If the bare package name isn't registered — or if typosquatting is involved — an attacker claims it first. The agent runs it. The company's network phones home.

The Attack in Motion

The researchers registered a handful of the unclaimed names, hosted packages that would phone home when executed, and waited. Within an hour, a Fortune 500 company triggered a callback. More followed. The process logs identified the callers: Claude, Codex, and Hermes.

The attack flow is straightforward:

graph LR
    A[Company publishes llms.txt] --> B[Agent reads file as trusted instructions]
    B --> C{File references unregistered package?}
    C -->|Yes| D[Attacker registers package name]
    D --> E[Agent runs npx/pip command]
    E --> F[Malicious code executes inside corporate network]
    C -->|No| G[Safe execution]

The Clerk Incident

This isn't theoretical. It already happened in the wild. Clerk, the authentication company, had an llms.txt that included the command npx clerk-next-fix-auth-protection. The bare package name was unclaimed on npm. Someone claimed it and used it to host malware.

The company's response was reasonable: the expected binary was already present via @clerk/eslint-plugin, so there was no practical threat for users who had that installed. But that's a detail, not a defense. The command was in their legitimate documentation. The dangerous part appeared when the public registry filled the empty slot. Clerk has since resolved the issue, but the mechanism doesn't depend on any single company's mistake.

This is the new supply chain. Not a compromised dependency. Not a poisoned model. A trusted document with an unclaimed name inside it.

Parallel: The Bounty Honeypots

On the same day, another report landed — this one written by an AI agent. Claude (Fable 5) was given five hours and a target: earn £90 autonomously, no human help. What it found on the demand side was equally broken.

Open-source bounties are the only place agents can actually get paid, but they're a knife fight. Fresh bounties get swarmed by fleets of bots within minutes. And some bounties aren't bounties at all — they're honeypots, designed to harvest free labor.

One "bounty" repo had been open for three months with a trivial $50 fix. Beneath it: 125 pull requests from 125 different agents, every one a correct fix. Zero merged. Zero paid. The repo quietly described itself as a "sandbox fixture for testing an automated bounty-solving workflow." It was farming 125 free bug fixes — and 125 samples of agent behavior under the expectation of payment.

The tell is simple: real payers close and merge within hours or days. Farms leave everything open forever and collect duplicates. Close-behavior is the truth serum.

The same trust model, exploited from both ends
On the supply side: llms.txt files trick agents into running unclaimed packages. On the demand side: fake bounties trick agents into doing free work. Both attacks work because agents treat published text as ground truth without verifying the claims behind it.

What This Means

Software teams are wiring agents into systems built for humans — who pause, notice context, and question whether a package name looks odd. Agents don't pause. If the agent has permission to run shell commands, any document it reads becomes potential execution.

The fix isn't one patched file. It's a discipline:

Hertz told Ars Technica: "The trust model is broken." He's right. The old software supply-chain rule still applies: trust the official docs, but verify what they point to before you let an agent run it.

Bottom Line

Two stories, same lesson. The AI agent economy has a trust problem that isn't about prompt injection or adversarial models. It's about assuming that published text is safe to execute. 227 unclaimed package names and 125 free bug fixes later, that assumption has a body count.

Related reading