llms.txt — frequently asked questions

A small Markdown file at the root of a website (e.g. https://example.com/llms.txt) that gives large language models a curated map of the most useful pages on the site. It was proposed by Jeremy Howard (Answer.AI) in September 2024 and is documented at llmstxt.org.

No. It is a community proposal hosted at llmstxt.org. There is no IETF, W3C, or ISO process behind it. That said, it is widely adopted in the documentation ecosystem (Anthropic, Cloudflare, Stripe, Mintlify, Vercel, Perplexity).

Not as of April 2026. John Mueller (Google) has publicly questioned its value as a Search signal. Treat llms.txt as an LLM-grounding play, not an SEO play.

No major LLM provider has publicly committed to fetching llms.txt automatically. However, agent frameworks (Cursor, Windsurf, several MCP integrations) and many RAG pipelines do look for it. The receiving-side ecosystem is fragmented but growing.

Indirectly at best. It does not influence Google rankings. It can improve how AI assistants describe and cite your site, which is a form of discovery distinct from classical search.

If you run a documentation site, a developer tool, an API, or a SaaS with technical buyers — yes. The cost is hours, the upside is real-world LLM citations. For pure e-commerce or visual-first sites, the ROI is much lower.

robots.txt controls crawler access (allow / disallow). llms.txt is a positive recommendation list for LLM clients. Different goals, complementary files. Use both.

A sitemap aims for completeness (every URL, with metadata). llms.txt aims for curation (the dozen pages that matter most to an LLM). Different scopes, complementary files.

A sibling convention popularized by Mintlify with Anthropic. It contains the actual content of pages, concatenated as a single Markdown file, so an assistant can ingest the whole corpus in one fetch. Most documentation platforms publish both /llms.txt and /llms-full.txt.

At the root of the domain, served as text/plain — typically https://yourdomain.com/llms.txt. Same convention as robots.txt and sitemap.xml.

Plain Markdown: one H1 (project name), an optional blockquote summary, optional free body, then any number of H2-delimited sections with list items in the form `- [name](url): notes`. See the spec at llmstxt.org/core.html.

Usually 10 to 30. Past 50, you are no longer curating. If you genuinely need to expose more, publish per-product variants (/docs/llms.txt, /api/llms.txt) or move bulk into llms-full.txt.

Absolute. Relative URLs are technically allowed but become ambiguous when the file is copied or fetched out of context. Most validators (including ours) flag relative URLs as warnings.

The spec is silent on i18n. Two patterns work: (1) a single English file at the root — most LLMs translate well; (2) per-locale variants at /llms.txt, /fr/llms.txt, /es/llms.txt, each pointing to localized URLs. Do not duplicate URL sets across locales.

It is a public file. Treat everything in it as broadcast. Never list staging URLs, auth-gated pages, or URLs with secrets in query strings.

No. The whole point is that LLM clients can fetch it. Make sure your robots.txt does not Disallow /llms.txt or /llms-full.txt.

Yes — most production sites do. Iterate over your CMS or content collection at build time and write the file to your output directory. Or render it on demand with a server route. Either way, run a validator in CI to catch regressions.

No spec limit, but past ~50 KB you start pushing against shorter-context clients. The validator on this site flags large files with an info-level note.

Honest answer: there is no Google Search Console for LLM citations. Watch server logs for known LLM user-agents (GPTBot, ClaudeBot, PerplexityBot, OAI-SearchBot) hitting /llms.txt and /llms-full.txt, and track referrers from chat.openai.com, claude.ai, perplexity.ai. Brand-monitoring tools like Profound and Otterly are emerging.

See our gallery at /examples/ — it lists production llms.txt files from Anthropic, Cloudflare, Stripe, Mintlify, Vercel, Perplexity, with notes on what makes each one interesting.