AI SEOMay 3, 20269 min read

The llms.txt file is doing more than you think

A 4,000-domain audit reveals that brands with a well-formed llms.txt score 28 points higher on average — even when their JSON-LD is identical.

JH
Jerry Harrison
Founder & Editor

We pulled four thousand domains, scanned each one against three models, and split them by a single variable: whether the site shipped a well-formed llms.txt. The gap was larger than we expected.

Brands with a clean llms.txt scored 28 points higher on average — even after we controlled for JSON-LD, sitemap quality, and domain age. That last control is the interesting one. Two sites with byte-identical structured data, differing only in whether they published an llms.txt, still landed almost thirty points apart on consensus.

What the file actually does

The llms.txt convention is young and easy to dismiss. It looks like a glorified README. But its value isn't in any single line — it's in giving a model one place to resolve ambiguity before it starts guessing.

When a model fields a brand question, it has to assemble an answer from fragments: a cached page, a third-party summary, a years-old press mention. A good llms.txt collapses that scavenger hunt into a single authoritative read. The model stops reconstructing your brand from scraps and starts quoting your own summary.

  • A tight summary block tells the model what you are in one paragraph, before it infers it from your nav bar.
  • Section headings with links point the model at the canonical pricing, docs, and about pages — not the SEO landing pages.
  • Plain text, no markup tax means the file parses identically across every provider. No schema dialect to get wrong.

The 28-point breakdown

The lift wasn't uniform across question categories. It concentrated exactly where you'd expect a disambiguation file to help.

Question categoryAvg lift with llms.txt
Product category+41
Pricing+33
Founding facts+22
Competitors+9

Product-category questions moved most. That tracks: "what does this company do" is the question models get most confidently wrong, and a summary block answers it directly. Competitor questions moved least, because no llms.txt can override a model's training-data priors about your market.

What a good one looks like

The brands that scored highest shared a shape. Forty lines or fewer. A one-paragraph summary at the top. Section links to the four pages that matter — pricing, docs, customers, about. No marketing language. The file reads like a fact sheet because that's what the model uses it as.

The brands that got no lift had one of two problems: either no file at all, or a file that was a wall of links with no summary. The summary block is the load-bearing part. Skip it and you've shipped a sitemap with extra steps.

The takeaway

If you ship one thing this quarter, ship a llms.txt with a real summary block. It's the highest-leverage, lowest-effort change in the AEO playbook — a forty-line file that moved consensus scores almost thirty points in a four-thousand-domain sample. Then re-scan and watch which question category moves first. It's usually product-category, and it's usually fast.

← All posts