llms.txt Adoption, Who Uses It?

Adoption snapshot

Since Jeremy Howard proposed llms.txt at Answer.AI in September 2024, adoption has grown from a handful of early experiments to a convention served by many of the best-known developer-facing companies. Growth accelerated in 2025 when documentation hosting platforms added automatic generation.

The clearest signal of a format's success is when the infrastructure adopts it, platforms serve thousands of sites each, multiplying reach without requiring individual developers to take action.

Measured adoption (our tracker)

Claims like “tens of thousands of sites use llms.txt” circulate without a source. So we measure it ourselves: a fixed panel of 219 notable hosts across eight sectors, fetched and parsed every week by an automated checker. Latest run: 2026-06-15.

Methodology

The panel is a fixed, curated list of 219 hosts (published at /adoption-panel.json); membership does not change between runs except by documented addition. For each host the checker requests https://<host>/llms.txt, follows redirects, and counts the host as serving only if the response is HTTP 200, plain text (not an HTML app shell), and starts with a valid Markdown H1. Unreachable hosts (4 in the latest run) are excluded from the base. Raw results: /adoption-stats.json. The 112 positive entries are listed in the verified directory.

Limits: the panel skews toward developer-facing companies (that is where adoption concentrates, and where the file is most useful), so the headline rate should not be read as an all-of-the-web figure. It is a tracked cohort, comparable run over run.

Documentation platforms

The largest force-multipliers in llms.txt adoption are documentation hosting platforms that generate the file automatically for every project:

• Mintlify, one of the first major platforms to ship automatic llms.txt generation. Thousands of API documentation sites powered by Mintlify expose llms.txt and llms-full.txt without any developer configuration.
• GitBook, added llms.txt auto-generation in 2025. GitBook hosts documentation for hundreds of developer-facing companies; all of them now serve the file automatically.
• Read the Docs, the widely-used open-source documentation hosting platform added llms.txt support, covering a large portion of Python ecosystem projects.
• Docusaurus (Meta), a popular plugin generates llms.txt for Docusaurus sites. Many of the largest open-source project doc sites (React, Babel, Jest, etc.) use Docusaurus.
• VitePress / VuePress, community plugins add llms.txt generation to these static documentation generators, used by Vue.js ecosystem projects.

Companies and projects

Notable early adopters across developer tooling, AI, and infrastructure:

• Anthropic, anthropic.com/llms.txt and docs.anthropic.com/llms.txt provide structured access to Claude documentation.
• Cloudflare, developer documentation at developers.cloudflare.com/llms.txt with a comprehensive index of Workers, Pages, and AI Gateway docs.
• Vercel, vercel.com/docs/llms.txt serves an index of platform documentation for AI coding assistants.
• Supabase, supabase.com/docs/llms.txt covering database, auth, storage, and edge function documentation.
• PlanetScale, Neon, Turso, database-as-a-service providers that cater to AI developers have been quick adopters.
• FastAPI, Pydantic, Ruff, leading Python ecosystem projects expose llms.txt to improve how AI assistants understand their APIs.
• Tailwind CSS, tailwindcss.com/llms.txt helps AI coding assistants provide accurate class suggestions.

AI tools with native support

Beyond the sites that publish llms.txt, these tools actively read it:

• Cursor, the AI code editor fetches llms.txt when you add a URL as a project context source. This is the primary driver of developer awareness of the format.
• Perplexity, has officially announced support, with PerplexityBot reading llms.txt to prioritize indexing.
• LlamaIndex, built-in LlmsTxtReader that parses the file and fetches linked pages to populate a vector index.
• LangChain, community LlmsTxtLoader integration for building RAG pipelines from any llms.txt-enabled site.
• Firecrawl, uses llms.txt to guide site crawls, prioritizing linked pages when building knowledge bases.
• Cline, Aider, Continue.dev, open-source AI coding tools that support URL context all benefit from llms.txt.

Why adoption is accelerating

Three factors are driving rapid adoption:

1. Platform multipliers, when a documentation platform ships auto-generation, thousands of sites adopt it overnight without any developer effort.
2. AI coding assistant demand, Cursor, Copilot, and Cline users actively request llms.txt from their libraries. Maintainers are responding to real demand.
3. Zero cost, immediate value, a static text file with no infrastructure requirements and measurable benefit to AI tool users is an easy yes for most developer-facing projects.

Check if a site has llms.txt by visiting https://<domain>/llms.txt or using the validator to analyze the file structure.

Related pages

• Which AI crawlers read llms.txt, detailed breakdown.
• llms.txt format reference, spec details.
• How to create llms.txt, add it to your site in minutes.
• Validator · Generator.