Table of Contents
What is llms.txt?
llms.txt is a proposed standard: a plain Markdown file at the root of your site that gives AI systems a curated, readable map of your most important content. It was put forward in 2024 and has been adopted voluntarily by a number of sites, particularly in software documentation.
Set expectations first
It is a proposal, not a ratified web standard, and the major AI companies have not committed to reading it. Treat it as a low-cost, unproven experiment — not as a ranking factor and not as a substitute for the work that does move the needle.
How is it different from robots.txt?
They solve opposite problems. robots.txt is about permission — which crawlers may access which paths. llms.txt is about comprehension — here is what matters on this site and here is a clean version of it.
| robots.txt | llms.txt | |
|---|---|---|
| Purpose | Controls access | Offers curated content |
| Format | Directives | Markdown |
| Status | Long-established, universally honoured | Proposed, adoption voluntary |
| If you omit it | Crawlers assume access is allowed | Nothing changes |
Because it is additive, llms.txt cannot hurt your visibility. It also cannot rescue a site whose real problem is that its content is unclear or its pages cannot be crawled.
What does the file look like?
The format is deliberately simple: an H1 with the site name, a short summary in a blockquote, then linked sections with one-line descriptions.
# Evoba
> SEO, AEO, and web development for businesses that need to be found
> in search engines and in AI-generated answers.
## Services
- [SEO Services](https://www.evoba.com/seo): Organic search strategy and execution.
- [AEO Services](https://www.evoba.com/aeo): Getting cited by AI answer engines.
- [Local SEO](https://www.evoba.com/local-seo): Map pack and local search visibility.
## Guides
- [SEO Fundamentals](https://www.evoba.com/learn/seo-fundamentals): How search works.
- [AI Search & AEO](https://www.evoba.com/learn/ai-search): Getting cited by AI engines.
## Contact
- [Contact](https://www.evoba.com/contact): Get in touch.Some sites also publish llms-full.txt, which contains the full text of key pages rather than links to them. That is more useful to a system that wants content without crawling, and more work to keep current.
Should you add one?
The honest answer is that it is cheap enough to be worth doing and unproven enough that you should not expect a result. An hour of work, no downside, uncertain upside.
It makes more sense if: you have substantial documentation or a large guide library, your site is JavaScript-heavy and awkward to crawl, or you want the option in place should adoption arrive.
It matters less if: your site is small and already clear, or you are treating it as a shortcut instead of fixing crawlability and content quality — in which case it is a distraction from the things that genuinely determine whether you get cited.
Order of operations
Confirm your pages are crawlable, indexed, and answer questions directly first. Those decide whether you get cited. Add llms.txt afterwards as a low-cost extra, not as the strategy.
How to add it
- Write the file following the structure above — site name, summary, linked sections with descriptions.
- Save it as
llms.txtin your web root so it resolves atyourdomain.com/llms.txt. - Serve it as
text/plainand confirm it loads in a browser. - Keep it short and curated. It is an index of what matters, not a sitemap of everything.
- Review it when your site structure changes; a stale file is worse than none.
Key takeaways
llms.txtis a proposed convention, not an adopted standard — no major AI provider guarantees it is read.- It offers curated content;
robots.txtcontrols access. They are complementary, not alternatives. - Cheap to add, no downside, unproven benefit. Do it after the fundamentals, not instead of them.
- Keep it curated and current; an outdated file misleads anything that does read it.
Common Questions
Does llms.txt improve my rankings?
No. It is not a ranking factor in any search engine, and no major AI provider has committed to reading it. It is a low-cost convention that may help systems that choose to support it.
Does llms.txt replace robots.txt?
No. They do different jobs. robots.txt controls which crawlers may access which paths and is universally honoured. llms.txt offers a curated view of your content and is entirely voluntary. Keep both.
What is the difference between llms.txt and llms-full.txt?
llms.txt is an index — links to key pages with short descriptions. llms-full.txt contains the full text of those pages in one file, which is more useful to systems that want content directly but harder to keep current.
Where does the file go?
In your web root, so it resolves at yourdomain.com/llms.txt, served as plain text. Same placement convention as robots.txt.
Want the fundamentals handled first?
We audit crawlability, structure, and AI visibility, then fix what is actually holding you back.
See our AEO service All AI Search & AEO guides