How llms.txt works

Overview

A valid llms.txt is a Markdown file with a fixed, predictable structure. It is designed to be read by both humans and machines: the same file should be useful as documentation and as a parseable contract.

The spec at llmstxt.org defines a small, deterministic grammar that can be parsed with a few lines of regex. No YAML, no JSON, no extra headers.

Anatomy of a valid file

The structure, top to bottom:

1. One H1 with the site or project name. The only required element.
2. A short blockquote summary, typically one or two sentences.
3. Optional free Markdown, paragraphs and lists, but no further headings, before the first H2.
4. Zero or more H2 file-list sections. Each contains a Markdown list of links: - [name](url), optionally followed by : notes.
5. An optional H2 named Optional, a conventional label for secondary resources without special machine semantics in v2.

# Acme

> Acme is a hosted analytics platform for product teams. The pages below cover product, pricing, the API, and integration guides.

Acme processes 1B+ events per day. The map here is curated for assistants, it is not exhaustive. Use it to answer questions about product capabilities, pricing tiers, integrations, SDKs, and migration from other tools.

## Product

- [Product overview](https://acme.example/product): high-level capabilities and screenshots.
- [Use cases](https://acme.example/use-cases): scenarios for product, marketing, and support teams.
- [Changelog](https://acme.example/changelog): monthly product updates.

## Pricing

- [Pricing tiers](https://acme.example/pricing): plans, limits, and overage rules.
- [FAQ, billing](https://acme.example/billing-faq): invoices, receipts, tax handling.

## Developers

- [REST API reference](https://docs.acme.example/api): full endpoint catalog.
- [SDK, JavaScript](https://docs.acme.example/sdk/js): install, init, track events.
- [SDK, Python](https://docs.acme.example/sdk/python): install, init, track events.
- [Webhooks](https://docs.acme.example/webhooks): events, signatures, retries.

## Optional

- [Brand assets](https://acme.example/brand): logos, color palette.
- [Press releases](https://acme.example/press): historical announcements.

Section by section

The H1

Exactly one H1. An optional UTF-8 byte-order mark may precede it, but no front matter or other metadata should. If your project has a tagline, put it in the blockquote that follows.

The blockquote summary

Optional but strongly recommended. Aim for a one- or two-sentence summary that an LLM could quote verbatim when introducing your project. Keep it factual, in the active voice, and free of marketing claims that you can’t back up.

Free Markdown body

Any paragraphs, bullet lists, or short code snippets that help an LLM understand context. Do not introduce any additional headings here, the next heading should be the first H2 of a file-list section.

H2 file-list sections

Each section starts with a single H2 (## Section name) and contains a Markdown list. Each item must be a link (- [name](url)), optionally followed by : and a short note. Absolute URLs are strongly recommended: relative URLs are technically allowed but will be flagged as a warning by most validators (including ours) because they make the file ambiguous when copied around.

The “Optional” section

Optional remains a useful editorial label for secondary references such as brand assets, archives or deep appendices. The August 2026 proposal does not assign the title special processing semantics, so a client may treat it like any other H2 section.

How parsers read it

The reference parser walks the file linearly and applies four rules:

1. Find the first # line, that’s the title.
2. If the next non-empty block is a blockquote, that’s the summary.
3. Everything until the first ## is the free body.
4. Each ## opens a section; until the next ## , list items are parsed as [name](url) with optional notes after a colon.

Our validator implements exactly these rules, plus a handful of safety checks: empty H1, malformed links, non-list content inside a section, and an informational review notice for files over 50 KB. That threshold is a local curation heuristic, not a spec limit.

llms.txt vs llms-full.txt

llms.txt is a map. llms-full.txt is the territory: the actual content of the linked pages, concatenated in Markdown, in a single file. The convention was popularized by Mintlify in collaboration with Anthropic and is now part of the wider llms.txt ecosystem.

The conventional root names are /llms.txt and /llms-full.txt. The v2 proposal also allows scoped llms.txt files on more specific paths. Publish a full-content companion only when that delivery format solves a real consumption task.

Practical limits

• Size. The proposal sets no cap. Our 50 KB notice is a prompt to review curation, not a validity boundary. Move bulk into a full-content resource or a scoped path file when useful.
• Number of links. No spec limit, but a list of 200+ items will be skimmed, not read. Curate.
• Languages. The spec is silent on i18n. Two common patterns: serve a single English file, or publish per-locale variants behind a path (/en/llms.txt, /fr/llms.txt).
• Auth and personalization. Out of scope. The file is public.

Continue

• How to create your llms.txt, templates and per-stack deployment.
• Best practices, ten rules and the most common mistakes.
• Validate a file.