What is llms.txt and how to add one to your site
Updated
llms.txt is a proposed Markdown file that gives AI tools a short summary of your site and links to key pages. The format, an example, and what it can and can't do.
llms.txt is a proposal, published by Jeremy Howard of Answer.AI in September 2024 at llmstxt.org, for a plain Markdown file at /llms.txt that tells large language models what a site is and where its most useful content lives. Think of it as a short, human-written table of contents for AI tools, as opposed to robots.txt (which sets access rules) and sitemap.xml (which lists every URL).
The format
The file is Markdown with a simple structure:
- An H1 with the name of the site or project (the only required part).
- A blockquote with a one or two sentence summary.
- Optional paragraphs with more detail.
- H2 sections containing lists of links, each with an optional short description.
The proposal also describes an optional section titled "Optional" for links that can be skipped when context is short, and a companion /llms-full.txt that contains the full text of the important pages in one file.
Example
# Acme Invoicing > Acme sends automatic invoice reminders for freelancers. Plans start at $15/month. ## Key pages - [Pricing](https://acme.example/pricing): plans and what each includes - [How it works](https://acme.example/how-it-works): setup in 5 minutes - [FAQ](https://acme.example/faq) ## Contact - [Contact](https://acme.example/contact): email support, reply within one business day
How to add it
- Save the file as
llms.txtand serve it from the root of your domain as plain text (text/plainortext/markdown). - Make sure your framework or host doesn't return your HTML homepage for unknown paths. A common mistake is an SPA fallback that answers
/llms.txtwithindex.html. - Keep it short and current. Update it when your pricing or key pages change.
What it does and doesn't do
llms.txt is not an official standard, and the major AI providers have not publicly committed to reading it, so don't expect it to change how ChatGPT or Google treat your site on its own. It's useful because it's cheap: it gives any AI tool or agent that does look for it (including coding assistants and tools that users point at your docs) an accurate summary and the right links. It doesn't replace robots.txt, crawlable HTML or structured data.
BotVisible's free scan checks whether your site serves a valid llms.txt, and the full report generates one pre-filled with your site's own pages.