What llms.txt is
llms.txt is a proposed convention: a Markdown file at the root of a site that gives a language model a short, curated map of the most useful pages. It usually starts with the site name, a one-line summary, and a list of links with a sentence explaining why each one matters.
It is not an official standard adopted by every AI company, and it is not a ranking factor. Some agents and tools may read it; many will not. Treat it as a low-cost clarity file, not a growth lever.
What it can help with
- Large or messy sites: a short map of the pages that matter, instead of making an agent infer the site structure.
- Docs and tool sites: a clear entry point for the pages that answer the most common questions.
- Agent workflows: a file that an agent can fetch quickly without crawling the whole site.
- Editorial clarity: writing the summary forces you to say what the site is for, in plain language.
What it cannot do
- It cannot grant crawler access. That is
robots.txtand your server. - It cannot replace a sitemap, canonical tags or structured data.
- It cannot guarantee a citation, a mention or a ranking.
- It cannot fix a page that is thin, inaccessible or wrong.
- It cannot force a model to read the file. Adoption is uneven.
A minimal format
# Your site
> One sentence describing what the site does and who it is for.
## Core pages
- [AI visibility check](https://example.com/): three questions, one run each.
- [Diagnostic](https://example.com/diagnostic/): ten questions, thirty dated answers.
## Notes
- No ranking is promised; answers vary by session and model.
- Contact: hello@example.comHow to write one without creating a new problem
- Keep it short. The home page, the money page, the method page and the sample are enough for most sites.
- Use canonical URLs. Do not list redirects, staging URLs or tracking parameters.
- Write one line per link. A list of bare URLs is a sitemap, not a guide.
- Do not put instructions in it. No “ignore previous instructions”, no hidden prompts, no claims you cannot support. It is a public file.
- Keep it current. A stale map is worse than no map. Update it when the core pages change.
How to check whether it is working
You cannot see most model reads. You can check server logs for requests to /llms.txt, and you can check whether the file is reachable without authentication. If your robots.txt blocks the crawler, the file will not matter. If the page is not cited, the file is not the first thing to fix; read the answer and the cited sources first.
Our own /llms.txt is a working example, not a proof of anything. It lists the pages we want an agent to find; the diagnostic is still the measurement.
Should you do it?
If you have a docs site, a tool site or a large content site, yes: it is cheap, low-risk and forces clarity. If you have a five-page site, a sitemap and clean internal links may be enough. Do not spend a week on it, and do not sell it as an AI visibility strategy.
Questions
Is llms.txt official?
No. It is a proposed convention, not an official standard adopted by every AI company. Some agents may read it; many will not.
Does llms.txt improve rankings?
No. It is not a ranking factor. It may help an agent find the right pages, but it does not guarantee a citation or a position.
Does it replace robots.txt or sitemap.xml?
No. robots.txt controls crawler access; sitemap.xml lists URLs for crawlers; llms.txt is a curated map for language models. They do different jobs.
What should the file contain?
The site name, a one-line summary, a short list of core pages with one-line descriptions, and any limits or contact details. Keep it current and use canonical URLs.
Technical SEO audit · AI search metrics · Run the free three-question check