llms.txt, frequently asked questions

A Markdown resource map published at the root or a more specific path for compatible agents. It was proposed by Jeremy Howard (Answer.AI) in September 2024; the August 2026 proposal is documented at llmstxt.org.

No. It is a community proposal hosted at llmstxt.org. There is no IETF, W3C, or ISO process behind it. That said, it is widely adopted in the documentation ecosystem (Anthropic, Cloudflare, Stripe, Mintlify, Vercel, Perplexity).

Not as of April 2026. John Mueller (Google) has publicly questioned its value as a Search signal. Treat llms.txt as an LLM-grounding play, not an SEO play.

No major LLM provider has publicly committed to fetching llms.txt automatically. Request logs show some agentic and retrieval user agents fetching files, but a fetch does not prove that a model read or used the content.

There is no public evidence that it improves Google rankings or AI citations. Treat it as experimental agent-facing infrastructure and measure only outcomes you can observe.

Publish one when a maintained, curated map serves a defined consumer or workflow. Documentation and API sites are plausible candidates; do not assume citation or ranking gains.

robots.txt publishes crawl instructions for compliant clients; it does not enforce access. llms.txt is a curated resource map for compatible agents. They solve different problems.

A sitemap aims for completeness (every URL, with metadata). llms.txt aims for curation (the dozen pages that matter most to an LLM). Different scopes, complementary files.

A sibling convention popularized by Mintlify with Anthropic. It consolidates page content into one Markdown resource for clients that explicitly support bulk ingestion.

Usually at https://yourdomain.com/llms.txt. V2 also allows files at more specific paths, such as /docs/llms.txt, with the most specific applicable file describing the page scope.

Plain Markdown: one H1 (project name), an optional blockquote summary, optional free body, then any number of H2-delimited sections with list items in the form `- [name](url): notes`. See the spec at llmstxt.org/core.html.

There is no spec count. Ten to thirty is only a starting heuristic. Let the intended tasks determine the list, and use path-level files for real content boundaries.

Absolute. Relative URLs are technically allowed but become ambiguous when the file is copied or fetched out of context. Most validators (including ours) flag relative URLs as warnings.

The proposal does not prescribe one i18n architecture. You can use a default-language root file and locale-scoped files, then declare the applicable map with rel=describedby on localized pages.

It is a public file. Treat everything in it as broadcast. Never list staging URLs, auth-gated pages, or URLs with secrets in query strings.

No. The whole point is that LLM clients can fetch it. Make sure your robots.txt does not Disallow /llms.txt or /llms-full.txt.

Yes. Generate it from a CMS or content collection at build time, or render it from a server route. Review the output and run validation plus link checks in CI.

The proposal sets no byte limit. This site emits an informational review notice above 50 KB, a local curation heuristic rather than a validity threshold.

Watch server logs for requests to the file and classify user agents carefully. A request proves retrieval, not reading or use. Track downstream referrals or citations separately and do not infer causation from either signal alone.

See our gallery at /examples/, it lists production llms.txt files from Anthropic, Cloudflare, Stripe, Mintlify, Vercel, Perplexity, with notes on what makes each one interesting.