A convention has emerged of publishing a file at /llms.txt that tells AI systems what a site contains. It is straightforward, cheap to produce, and surrounded by more certainty than the evidence currently supports.

Here is a realistic account.

What it is

A markdown file at the root of your domain, like robots.txt but with a different job.

robots.txt says what crawlers may access. It is permissions.

llms.txt says what the site is about and where the important things are. It is a map.

The intention is to save a language model from inferring your site's structure by crawling navigation and guessing at hierarchy. Instead it reads a concise summary with links and descriptions.

What goes in it

A loose convention has formed:

# Company Name

> One paragraph describing what the organisation does.

## About

- Founded: 2024
- Location: City, Country
- Services: the main categories

## Services

- [Service name](https://example.com/services/name/): what it is, in one sentence.
- [Another service](https://example.com/services/other/): what it is.

## Key pages

- [Contact](https://example.com/contact/): how to get in touch.
- [Blog](https://example.com/blog/): guides on X, Y and Z.

The pattern throughout: a link, then a plain description of what is behind it.

Some sites also publish llms-full.txt containing the complete text of key pages, so a model can read the content without fetching each URL. Reasonable for documentation; excessive for most business sites.

How to write it

Factually. This is the main thing. A model reading "we deliver innovative solutions tailored to your needs" learns nothing. One reading "builds AI voice agents that answer calls and book appointments, for clients in five countries" learns something quotable.

Resist the instinct to make it persuasive. It is reference material, not a brochure.

Specifically. Founding year, location, what you actually do, which services, what you do not do.

Honestly. If you have one office, say one office. If you do not publish client metrics, say that. Contradictions between this file and your other content reduce confidence rather than increasing it.

Concisely. It is a map. A model wanting detail will follow a link.

Generate it, do not write it

The practical advice that matters most.

A hand-written llms.txt goes stale within months. Nobody remembers to update it when a service is added, and a file listing services you no longer offer is worse than no file.

Generate it from the same data that generates your site. If your service pages come from a data file, the llms.txt should read that data file. Then adding a service updates both automatically and the file cannot be wrong unless the site is.

This site does exactly that — the file is produced at build time from the service, industry, location and blog data.

Does it work?

The honest answer: partially, and nobody can tell you precisely how much.

What is true:

  • Several AI tools and documentation platforms read it
  • Adoption by the major assistant providers is inconsistent and not well publicised
  • There is no public mechanism to verify whether a given provider used yours
  • No controlled evidence exists that it changes citation rates

What this means practically: treat it as a cheap bet rather than a strategy. It costs an hour to implement properly and effectively nothing to maintain if generated. The downside is zero and the upside is uncertain but real, which is a reasonable trade.

Anyone describing it as essential is ahead of the evidence.

What it does not do

It does not grant or deny access. That is robots.txt. Publishing an llms.txt while blocking AI crawlers achieves nothing.

It does not improve rankings. Google has not indicated it uses the file, and it is not a search signal.

It does not replace good content. A model finding your llms.txt still needs the pages it points at to actually answer things.

It does not stop training on your content. Separate question, governed by robots.txt and by provider policy.

Where the effort is better spent

If you have limited time for answer-engine work, the order that matters:

  1. Make sure AI crawlers are not blocked. Check robots.txt first. This is the one that is occasionally catastrophic.
  2. Write extractable content — question-shaped headings with direct answers underneath.
  3. Implement FAQ and Article schema. Structured, machine-readable, and used by systems that definitely read it.
  4. Keep entity data consistent across your site, schema and third-party listings.
  5. Earn third-party mentions, which is what gets you into the candidate set at all.
  6. Then add llms.txt.

It belongs on the list. It belongs near the bottom.

A reasonable implementation

If you are doing it:

  • Generate it at build time from your real content data
  • Keep it factual and under a few hundred lines
  • Include company facts, services with descriptions, key pages, and your articles
  • Add a short notes section covering anything a model should know — for example, that you do not publish unverified client statistics
  • Serve it as text/plain at /llms.txt

Then check it occasionally to confirm it still reflects the site, which it will if generated.

If you want it set up as part of a broader answer-engine review, get in touch.