Does llms.txt work? What the evidence says for stores

llms.txt was proposed in 2024 as a Markdown file at a site's root that tells language models what the site is and where its useful content lives. By 2026 it had become a fixture of AI-optimisation checklists, and SEO tools began flagging stores that lacked one.

The question merchants keep asking is simple: does it work? There is now enough data to answer it, and the answer depends on what you mean by "work". This post reviews the evidence and gives a recommendation for online stores.

What is llms.txt?#

A plain Markdown file at /llms.txt that summarises a website for language models and links to its most useful pages. The format, published at llmstxt.org, has an H1 with the site name, a short summary, and sections of links with optional notes. A companion /llms-full.txt can carry the full text.

It is a suggestion to AI systems, not a standard any of them is obliged to follow, and not an access control. Nothing about it stops or allows a crawler; that is still robots.txt's job.

Do AI crawlers read llms.txt?#

Almost never. Several independent measurements in 2026 point the same way:

• Ahrefs looked at 137,210 domains in May 2026 and found that 97% of llms.txt files received zero requests. Fewer than a fifth of the requests that did arrive came from named AI tools, led by GPTBot and Claude Code.
• Otterly ran a 90-day server-log test on one site with a correctly built file: of more than 62,100 AI bot visits, 84 touched llms.txt, about 0.1%.
• Limy reported 408 direct requests for llms.txt across more than 500 million AI bot visits over 90 days.

The pattern is consistent: the crawlers that feed AI search fetch HTML pages and skip the file.

llms.txt files by requests received, May 2026

Received zero requests

97%

Received at least one

3%

Does llms.txt help you get cited by AI assistants?#

There is no evidence it does. SE Ranking examined around 300,000 domains, of which about 10% had an llms.txt file, and found no clear link between having the file and being cited by AI systems. Only one of the 50 most-cited domains in its sample had one.

Google has been explicit. Its staff said in 2025 that Google does not support llms.txt and has no plans to; in 2026 Google's guidance on AI features listed llms.txt among tactics not needed for AI Overviews and AI Mode, and said machine-readable AI files neither help nor hurt appearing in Search.

Where does llms.txt still matter?#

In a few tools that read it deliberately, and in audits that check for it.

• Chrome Lighthouse added an experimental agentic browsing audit that checks for llms.txt. That matters commercially: developers who run Lighthouse in CI will see the gap flagged, whatever search engines do.
• Developer and coding tools. Some AI coding assistants and documentation tools read llms.txt to find docs quickly. Useful for API and documentation sites; marginal for a store.
• Browser agents. An agent landing on an unfamiliar site can use a summary and a map of key pages. Today few do, but it costs little to offer one.

This is why tragent reports a missing or broken llms.txt at low severity: a hint worth having, never a blocker, and never ranked above a failure that stops an agent from buying.

How good are the llms.txt files that exist?#

Mostly not very. Common Crawl analysed /llms.txt responses in its July 2026 crawl and found that a large share of sites answering at that path were not serving an llms.txt at all.

Content served at /llms.txt, of successful responses

Text or Markdown (an actual llms.txt, or an empty file)

46.5%

An HTML page (usually a site's app shell)

40.4%

A robots.txt file at the wrong path

10.6%

Something else

2.5%

Among the real files, 68% were generated from templates, with one website builder accounting for 41% of the corpus. About half had the full structure the format describes, 23% contained no links at all, and a small number carried text trying to instruct AI systems, which is exactly the kind of content agents should treat with suspicion.

The lesson for a store: an llms.txt that is really your homepage's HTML, or an empty file, is worse than none, because it tells any tool that does look that your site is careless with machine-readable data.

Why did llms.txt catch on if crawlers ignore it?#

Because it was easy, it sounded like robots.txt for the AI era, and checklists rewarded it. Publishing a text file is a ten-minute task with a visible result, at a moment when merchants were being told AI search would change everything and few other actions felt as concrete.

Three things kept it going:

• Tools scored it. SEO and AI-readiness tools added "llms.txt present" as a check, and a missing file became a red mark on a report. Some website builders began generating one automatically, which is why so many existing files are templated.
• It sounds like a standard. The name and location echo robots.txt and sitemap.xml, both of which crawlers do read. llms.txt is a proposal with no obligation behind it.
• It is not harmful. Nothing in the evidence suggests a correct file hurts. When the cost is near zero and the downside is nil, people publish just in case.

None of that makes it effective. It makes it cheap, which is the right lens for deciding how much time it deserves.

What actually helps AI assistants understand a store?#

The same machine-readable facts that help search engines and shopping feeds, served in pages agents can reach. Assistants answering "where can I buy X" or "what is this store's return policy" draw on:

• Product structured data with price, currency, availability and identifiers that match the page.
• A sitemap that lists product URLs, so crawlers can find the whole catalogue rather than the pages linked from the homepage.
• Policy pages that are reachable. Shipping, returns and contact details in plain HTML, not behind scripts and not disallowed in robots.txt; Shopify's default robots.txt, for example, disallows /policies/.
• Product feeds to Google Merchant Center and to ChatGPT that agree with the website.
• Fast, server-rendered HTML. Crawlers that skip llms.txt read HTML. Content that appears only after JavaScript runs is often missed.

Each of these is read by the systems that matter today. That is where the effort belongs.

Should an online store publish llms.txt?#

If you can publish a correct one cheaply, yes, after the things that actually stop agents buying. In priority order:

1. Let verified agents through your bot manager. An agent refused at the door never reads anything. See why AI shopping agents get a 403 at your checkout.
2. Make product data complete and accurate. Price, currency and availability in structured data that matches the page. See product data AI agents can read.
3. Get robots.txt right for search and assistant crawlers. See robots.txt for AI agents.
4. Then, optionally, llms.txt. Keep it short and true: what you sell, where the catalogue and policies are, and links to your sitemap, shipping, returns and contact pages.

A minimal, correct file for a store looks like this:

# Ridgeline Outdoor

> Trail running shoes and outdoor apparel, shipped from the US. Free returns within 30 days.

## Shop
- [All products](https://shop.example/collections/all): the full catalogue
- [Sitemap](https://shop.example/sitemap.xml): every product URL

## Policies
- [Shipping](https://shop.example/policies/shipping): rates and delivery times
- [Returns](https://shop.example/policies/returns): 30-day free returns
- [Contact](https://shop.example/pages/contact): support by email and chat

Serve it as text/plain or text/markdown, not HTML, and check it after platform or theme changes that touch routing.

Frequently asked questions

Does llms.txt improve AI search visibility?

The evidence says no. Ahrefs found 97% of llms.txt files received no requests in May 2026, SE Ranking found no clear link with AI citations, and Google says its AI features do not use it.

Does Google use llms.txt?

No. Google has said it does not support llms.txt, and its 2026 guidance lists it among things not needed for AI Overviews and AI Mode.

Is there any reason to publish llms.txt?

A small one: Chrome Lighthouse's experimental agentic-browsing audit checks for it, and some developer tools and agents read it. A short, correct file costs little.

Is llms.txt the same as robots.txt?

No. robots.txt tells crawlers which paths not to fetch. llms.txt is an optional summary and map of a site for language models, and it controls nothing.

What should an online store put in llms.txt?

A one-line description of what you sell, and links to your catalogue, sitemap, shipping, returns and contact pages. Serve it as plain text or Markdown, never as an HTML page.

Sources

1. The /llms.txt file — llmstxt.org
2. Does llms.txt work? The honest answer, with receipts — Optimizely. October 2026; compiles the Ahrefs, SE Ranking, Otterly and Limy studies and Google's statements
3. A content analysis of llms.txt files from the July 2026 crawl archive — Common Crawl. August 2026

Written by the tragent team and checked against the sources listed above on October 19, 2026. See how we write and correct posts in our editorial policy.