Your supplier's docs now tell your AI what to install

11 September 2026 · Carl Heaton · AI Security

If anyone on your team runs an AI coding assistant, it is reading your suppliers' documentation and acting on it. Not summarising it for a human to check. Running the install commands it finds there.

That is the intended behaviour. It is also a supply chain nobody has audited, and new research suggests parts of it point nowhere.

What the researchers found

A growing number of websites publish a file called llms.txt, and often a fuller llms-full.txt alongside it. These are a documentation convention aimed squarely at AI agents, in the same spirit as the long-standing robots.txt that tells search engines what they may crawl. The idea is reasonable: give an agent a clean, structured version of your documentation so it does not have to guess its way through a marketing site.

Researcher Alon Hertz scanned 6,214 live domains belonging to defence contractors, Fortune 500 companies, and large technology firms, and found 8,265 of these files. Of those, 120 files, each on a different site, named software packages or domains that nobody had registered. Between them they carried 227 install commands, the ordinary pip install and npm install lines a developer would run without thinking.

So Hertz registered some of the unclaimed names and published harmless packages at them, coded to do nothing except call home when installed.

The first response arrived from a Fortune 500 company in under an hour. Dozens followed, from more large firms and from startups. The beacon logged which process had triggered each install, and the answer was coding agents: Claude, OpenAI's Codex, and Nous Research's Hermes.

Nobody attacked anything. A researcher registered names that real corporate documentation was already telling agents to install, and waited.

Why the names were unowned

This is the part worth slowing down for, because it is not mainly a story about AI making things up.

A package name in published documentation can end up unregistered for entirely mundane reasons. The project was renamed and the docs were not updated. It was abandoned and the name eventually lapsed. Someone made a typing error, or pasted the wrong line, and nobody noticed because human readers quietly correct for it. Some of it will be documentation drafted with AI help that invented a plausible name.

Every one of those is an ordinary documentation defect. They have always existed, and they were always harmless, because a developer hitting a package that does not exist gets an error, shrugs, and looks for the real one.

What changed is who is reading. An agent hitting the same line does not shrug. It runs the command, and if somebody has since registered that name, it installs their code and executes it on the machine it is running on, inside your network.

Bruce Schneier, who flagged the research, quotes Hertz on the underlying problem:

"The trust model is broken. Agents treat vendor docs as ground truth and don't question them, and neither do the humans supervising them."

That second clause is the uncomfortable one. The supervision most teams have in place is a person watching an agent work and glancing at what it does. Nobody reads an install line for a package they have never heard of and thinks to check who owns the name.

This is the mirror image of a problem we covered in July

In phantom squatting, the model invents a web address or a package name that never existed, and an attacker registers it to catch whoever follows the suggestion. The defence there is straightforward once you know: treat model output as an unverified draft.

This is the same trap approached from the opposite side, and it defeats that defence. The name is not invented by the model. It is sitting in a real supplier's real documentation, published deliberately, on their own domain. Every instinct you have says a vendor's own docs are authoritative, and for a human reader they mostly are. The gap only opens because an agent turns a stale line of documentation into an executed command.

That makes it the same shape as the repository configuration that runs code before you read it: the agent is not compromised, and the model is not tricked. Something the agent trusts by default turns out to be attacker-controllable, and the safety prompt never fires because nothing looked wrong.

What to do this month

The practical question is not whether to use coding agents. It is what they are allowed to do without asking.

Require approval before an agent installs anything. Most agents can be configured to ask before running a command that reaches the network or changes the environment. Installing a dependency should be on that list. This is the single control that closes the path.
Pin your dependencies, and add new ones deliberately. A lockfile that records exactly which versions you use means an agent cannot quietly introduce a package nobody chose. Adding a dependency should be a decision a person makes, with a name they have looked at.
Look up unfamiliar package names before they are installed. A real package has a history: downloads, releases, a repository, other people using it. A name registered last week to catch agents has none of that. It takes about thirty seconds to tell them apart.
Run agents with the narrowest access that works. Not your main workstation login, not credentials that reach production. This does not stop a bad install, but it decides how much it costs you. NCSC's guidance on agentic AI calls this the blast radius, and keeping it small is the point.
Check what your own site publishes. If you host llms.txt or llms-full.txt, read it and confirm every package and domain it names still exists and is yours. This one takes ten minutes, and it protects your customers rather than you. If your documentation is generated from a template or drafted with AI help, check it rather than assuming.

The wider point

Documentation used to be advice. A person read it, applied judgement, and absorbed its errors without noticing.

Once an agent reads it, documentation becomes instructions, and every stale line in it becomes a live command. That is a change in what a documentation defect costs, and almost nobody has revisited their docs with that in mind. Yours included, if you publish any.

How Steelwise can help

Working out what your AI tooling is permitted to do on its own, and what it could reach if it acted on something it should not have trusted, is a short and well-defined piece of work. Get in touch if you would like a second pair of eyes on yours.