BLOG

Does llms.txt help? What it does and what is proven

What llms.txt is, what Google said on 15 June 2026, what a 300,000-domain study can and cannot prove, and who really reads the file. Checked 1 October 2026.

By Published 9 min read

llms.txt is a small Markdown file at the root of a website that points AI agents to the pages that matter. It is easy to make, and a lot of advice promises that it gets you cited by ChatGPT or ranked in AI search. This article sorts what is documented from what is hoped. The short version: llms.txt does not help you in Google Search, there is no evidence that it raises AI citations, and it is genuinely useful for one job: helping agents read documentation. We keep ours, and we explain below why that is not a contradiction.

What llms.txt is

The proposal comes from Jeremy Howard and was published on llmstxt.org on 3 September 2024. The page now carries version 2 of the proposal and was last modified on 10 August 2026. The idea: web pages are built for people, with navigation, scripts and layout around the text. An agent that needs to understand a site works better with a short, clean summary and links to Markdown versions of the important pages.

The format is simple and fixed:

  • an H1 with the name of the site (the only required part),
  • a blockquote with a short summary,
  • optional paragraphs with details,
  • sections under H2 headings, each a list of links with an optional note,
  • by convention, a section called “Optional” for links an agent can skip.

The proposal says plainly what the file is for: it is read “on demand, when an agent needs information about a topic while assisting a user”. It is not a permission file. Access rules live in robots.txt, and llms.txt cannot allow or block any crawler.

A minimal file for a SaaS product could look like this:

# ExampleApp

> ExampleApp schedules social posts for small teams. Paid plans from $9 per month; free plan with 3 channels.

## Docs
- [Quick start](https://example.com/docs/start.md): set up the first channel
- [API reference](https://example.com/docs/api.md): REST endpoints and limits

## Product
- [Pricing](https://example.com/pricing.md): plans, limits, refunds

## Optional
- [Changelog](https://example.com/changelog.md)

What Google says

Google answered the question directly. Its guide to optimizing for generative AI features lists llms.txt among the things you can ignore for Google Search:

“You don’t need to create new machine readable files, AI text files, markup, or Markdown to appear in Google Search (including its generative AI capabilities), as Google Search itself doesn’t use them.”

On 15 June 2026 Google added a clarifying note, recorded in its documentation updates log. The guide now reads:

“It’s completely fine if you decide to create and maintain LLMS.txt files (or other similar files) for other services or systems that use these files. Doing so will neither harm nor help your site’s visibility or rankings in Google Search, as Google Search ignores them.”

That covers Google Search, including AI Overviews and AI Mode. It does not say anything about ChatGPT, Claude or Perplexity, and we should not stretch it that far.

What about the others? On 1 October 2026 the crawler pages of OpenAI, Anthropic and Perplexity describe robots.txt tokens such as OAI-SearchBot, Claude-SearchBot and PerplexityBot as the way to manage search visibility. None of them mentions llms.txt as a ranking or citation signal. We found no statement either way, so “they ignore it” would be as unproven as “they use it”. Our guide on product visibility in AI search covers those crawlers and what actually decides whether you can be cited.

What the data says, and what it cannot say

The study most often quoted is SE Ranking’s analysis of nearly 300,000 domains, published on 7 November 2025. Its findings:

  • 10.13 % of the domains had an llms.txt file.
  • Adoption was almost flat across traffic levels: 9.88 % for sites with 0 to 100 visits, 10.54 % for 1,001 to 5,000, 8.27 % for sites with more than 100,000.
  • Spearman correlation and an XGBoost model found no effect of llms.txt on how often a domain was cited by AI models. Removing the variable made the model more accurate.

We read the method section before using these numbers, and three limits matter:

  1. It is a correlation study. It compares sites that have the file with sites that do not. It cannot show what would happen if you added one, in either direction.
  2. The citation data is not described in detail. The article does not say which models, which prompts, which period or how many answers were sampled, and it publishes no effect sizes. The authors themselves call their findings “context-dependent”.
  3. It predates version 2 of the proposal and comes from a company that sells AI visibility tracking. That does not make it wrong, but it is worth knowing.

So the honest reading is: in this dataset, having an llms.txt file did not go together with more AI citations. That is useful, and it matches Google’s statement. It is not proof that llms.txt can never matter for any engine. If someone promises you citations from llms.txt, ask for evidence that shows cause, not correlation. We have not seen any.

Who actually uses llms.txt

The file is not useless. It has found a real job, just not the one most advice sells:

  • AI companies use it for their own developer docs. On 1 October 2026, developers.openai.com/llms.txt, platform.claude.com/llms.txt (redirected from docs.claude.com) and docs.perplexity.ai/llms.txt all returned 200. OpenAI’s crawler documentation even opens with “For the complete documentation index, see llms.txt.”
  • Chrome’s Lighthouse checks for it. Lighthouse has an experimental “Agentic browsing” category (documentation last updated 5 May 2026). Its llms.txt audit flags a server error when the file cannot be fetched and marks a missing file (404) as not applicable, “as providing the file is optional at the moment”. It is a readiness check for agents, not a ranking factor, and it needs Chrome 150 or later.
  • Coding agents read it for documentation. The proposal’s author writes that the files are “used most heavily for software documentation, where coding agents follow them to find API references and tutorials”. That is the “on demand” use the proposal describes.

The pattern: llms.txt helps when an agent is already on your site, or about to be, and needs a map. It does not bring the agent there.

What we do at LaunchRepo

We keep both files: /llms.txt and a longer /llms-full.txt. Here is why, with the facts as of 1 October 2026:

  • They cost us nothing to maintain. Both are generated at build time from the same data as our pages: prices, catalog counts, guides, comparisons. They cannot drift from the site.
  • They serve a real reader. LaunchRepo is a toolkit for coding agents. When someone asks their agent whether LaunchRepo fits their product, a clean summary with prices and limits helps that agent answer correctly.
  • The live llms.txt is 63,009 bytes with 19 sections and 177 links. That is larger than the proposal’s idea of a file that fits easily into context; agents can read the H2 sections they need.
The top of LaunchRepo's llms.txt file in a browser: H1 with the product name, a blockquote summary and the first key facts

What we do not claim: that the file brings us traffic, citations or rankings. We have not analysed our server logs for requests to /llms.txt, so we cannot say how often it is read or by whom, and we will not guess. Our own data work goes into pages people and engines can cite, such as the directory status index and our article on what directory backlinks can and cannot prove.

Should you make one?

Your situation Worth it? Why
You have API or developer docs that agents read Yes This is what the format was made for; doc platforms often generate it
You want better Google rankings or AI Overviews No Google Search ignores the file
You want more ChatGPT, Claude or Perplexity citations Not as a lever No evidence of an effect; allow their search crawlers instead
Your site is generated from structured data anyway Cheap to add Generate it, so it never goes stale
You would write and maintain it by hand Low priority Spend the time on a page with facts nobody else has

If you make one, keep it accurate: price, plans, limits and what the product does not do. An agent that quotes a stale price from your llms.txt does more harm than no file at all. And do not create special pages for AI only: Google’s spam policies apply to generative AI answers too, and its guide calls out making content just to manipulate them.

Frequently asked questions

Does llms.txt improve Google rankings?

No. Google states that Google Search ignores llms.txt and similar files, and that having one will neither harm nor help visibility or rankings, including in AI Overviews and AI Mode.

Does ChatGPT use llms.txt?

OpenAI has not said so. Its crawler documentation names OAI-SearchBot and robots.txt as the controls for ChatGPT search and does not mention llms.txt. OpenAI does publish an llms.txt for its own developer docs, which agents can read on demand.

Is llms.txt the same as robots.txt?

No. robots.txt tells crawlers what they may access. llms.txt is a reading guide for agents and cannot allow or block anything. To control AI crawlers, use robots.txt with their published tokens.

Can llms.txt block AI training?

No. Use robots.txt tokens such as GPTBot, ClaudeBot or Google-Extended for that. They are separate from the search crawlers, so you can block training and still appear in AI search.

Does an llms.txt file hurt my site?

Google says it neither harms nor helps in Google Search. The only real risk is outdated information: if the file says something your pages no longer say, agents may repeat the wrong version.

What is llms-full.txt?

It is not part of the llms.txt proposal, which centres on a short index plus Markdown versions of pages. Some sites, including ours, publish a second file with the full text of the important pages, so an agent can read everything in one request. Ours was 174,811 bytes on 1 October 2026.

To see where llms.txt sits next to everything that does influence AI answers, read our guide on product visibility in AI search. If your next step is getting listed where people and engines look, see pricing.

Sources, all checked on 1 October 2026: llmstxt.org (v2, 10 August 2026); Google Search Central, “Optimizing your website for generative AI features” (updated 10 July 2026) and documentation updates of 15 June 2026; SE Ranking, “Does LLMs.txt impact your AI visibility and citations?” (7 November 2025); Chrome for Developers, Lighthouse llms.txt audit and agentic browsing scoring (5 May 2026); OpenAI, Anthropic and Perplexity crawler documentation; HTTP status of the llms.txt files named above; launchrepo.dev/llms.txt as served on 1 October 2026.

  • evidence
  • ai-search
  • llms-txt
  • method

Give your agent the whole list

LaunchRepo is the repository your AI coding agent runs: 299 directory playbooks, a product template and a tracker. Pay once at the current €49.00 price, use it for every product you build.

See pricing