Does Perplexity read llms.txt?

Ahrefs observed PerplexityBot requests, but AI retrieval bots represented only 1.1% of all measured llms.txt requests in its May 2026 sample. A request does not prove use in an answer.

Should I add llms.txt even if major AI crawlers do not yet support it?

Publish one when a maintained resource map serves a defined client or workflow. Do not assume automatic discovery, citations, rankings or zero maintenance cost.

Which clients fetch llms.txt?

Q: Does ChatGPT read llms.txt?

OpenAI documents separate user agents for ChatGPT Search, training and user-initiated visits. It does not document automatic llms.txt discovery or special treatment of the file.

Q: Does Claude read llms.txt?

No public provider documentation establishes automatic llms.txt discovery across Claude. A request log can show a fetch, not whether the file influenced a response.

Q: What is the llms.txt crawler user agent?

There is no dedicated universal user agent. Logs contain many client strings, which should be verified where possible because a user-agent label can be spoofed.

Request logs can identify fetches and client labels. They cannot prove that a model read the file, used it, or cited it in an answer.

Last updated: August 12, 2026

The short answer

There is no verified complete list of products that automatically discover, parse and use llms.txt. The strongest public evidence is request-log data: it shows that named clients fetched files in a defined sample, not what a model did with the bytes afterward.

V2 allows a map at the origin root or a more specific path and defines rel="describedby" for discovery. A page may also advertise a Markdown representation with rel="alternate" type="text/markdown". Clients must implement those relations for them to have an effect.

What the request logs measured

Ahrefs analysed requests to /llms.txt across 137,210 domains during May 2026. Roughly 38,000 domains served a response that Ahrefs accepted, but only 3% of those files received any request in the month. The panel is technical and SEO-oriented, so its rates are not Web-wide estimates.

Named AI bots accounted for 19.5% of the measured requests. SEO audit tools, general crawlers, profiling tools and unidentified clients together accounted for much more. Slackbot fetched the file more often than PerplexityBot in the sample.

Which client categories fetched the file

Named AI category	Share of all measured requests
Agents and agentic infrastructure	10.5%
Training crawlers	5.3%
AI assistants	2.5%
Retrieval bots for live AI search	1.1%

Agentic clients formed the largest named AI category, and Claude-Code generated more requests than any retrieval bot, assistant or training crawler in the dataset. That makes developer-initiated workflows a plausible audience, but it does not establish automatic discovery or product-wide support.

Search, training and user visits are different

OpenAI's crawler documentation separates OAI-SearchBot for ChatGPT Search, GPTBot for content that may be used to train foundation models, and ChatGPT-User for certain actions initiated by a user. OpenAI states that ChatGPT-User is not used for automatic web crawling and is not used to decide whether content appears in Search.

This distinction is useful when classifying logs. It does not establish that any of those user agents automatically discover, parse or give special treatment to llms.txt: the public documentation does not make that claim.

What a request cannot prove

A request proves that a client retrieved a response, not that a model read or retained it.
A user-agent string is a declared label and can be spoofed unless its origin is verified.
A fetch does not show that the file influenced an answer, citation, ranking or training corpus.
A manual request to an explicit URL is different from automatic discovery.
One technical panel and one month cannot establish behavior across the entire Web.

How to measure your own file

Log requests to every root and path-level llms.txt URL.
Keep status, timestamp, requested URL, final URL and user-agent string.
Verify known bots through the provider's documented method when attribution matters.
Separate human, audit, general crawler, agentic, training, assistant and retrieval traffic.
Report requests separately from referrals, citations and downstream conversions.

The [monitoring method](/blog/how-to-monitor-llms-txt/) explains denominators and response states. For the measured category breakdown, read who fetches llms.txt.

FAQ

Does ChatGPT read llms.txt?

No OpenAI documentation establishes automatic llms.txt discovery or special treatment. OpenAI documents OAI-SearchBot for Search and ChatGPT-User for certain user-initiated visits. Those roles should not be conflated, and a fetch to an explicit URL is not proof of product-wide support.

Does Claude read llms.txt?

No public provider documentation establishes automatic llms.txt discovery across Claude. A log entry can show that a client fetched a response; it cannot show that the file was parsed, retained or used in an answer.

What is the llms.txt crawler user agent?

There is no single user agent for llms.txt crawlers. Each AI system uses its own crawler identity (PerplexityBot, GPTBot, ClaudeBot, etc.). llms.txt is a passive file, crawlers must explicitly fetch it.

Who uses llms.txt, real-world adoption evidence.
llms.txt format reference, spec details.
How to create llms.txt, step-by-step for any stack.
Validator · Generator.