What is llms.txt and how to add one to your site

Updated

llms.txt is a proposed Markdown file that gives AI tools a short summary of your site and links to key pages. The format, an example, and what it can and can't do.

llms.txt is a proposal, published by Jeremy Howard of Answer.AI in September 2024 at llmstxt.org, for a plain Markdown file at /llms.txt that tells large language models what a site is and where its most useful content lives. Think of it as a short, human-written table of contents for AI tools, as opposed to robots.txt (which sets access rules) and sitemap.xml (which lists every URL).

The format

The file is Markdown with a simple structure:

  1. An H1 with the name of the site or project (the only required part).
  2. A blockquote with a one or two sentence summary.
  3. Optional paragraphs with more detail.
  4. H2 sections containing lists of links, each with an optional short description.

The proposal also describes an optional section titled "Optional" for links that can be skipped when context is short, and a companion /llms-full.txt that contains the full text of the important pages in one file.

Example

# Acme Invoicing

> Acme sends automatic invoice reminders for freelancers. Plans start at $15/month.

## Key pages
- [Pricing](https://acme.example/pricing): plans and what each includes
- [How it works](https://acme.example/how-it-works): setup in 5 minutes
- [FAQ](https://acme.example/faq)

## Contact
- [Contact](https://acme.example/contact): email support, reply within one business day

How to add it

What it does and doesn't do

llms.txt is not an official standard, and the major AI providers have not publicly committed to reading it, so don't expect it to change how ChatGPT or Google treat your site on its own. It's useful because it's cheap: it gives any AI tool or agent that does look for it (including coding assistants and tools that users point at your docs) an accurate summary and the right links. It doesn't replace robots.txt, crawlable HTML or structured data.

BotVisible's free scan checks whether your site serves a valid llms.txt, and the full report generates one pre-filled with your site's own pages.

Check your site in 10 seconds

BotVisible fetches your page like an AI crawler and scores it from 0 to 100. Free, no signup.

Run a free scan

More guides