How to measure AI visibility without crediting llms.txt

The measurement challenge

Visibility reports for AI search are becoming available, but they are not attribution reports for llms.txt. Google Search Console reports selected visibility in Google's generative Search and Discover features. Bing Webmaster Tools reports citations across its supported AI experiences. Neither report contains a field that says an impression or citation was caused by this file.

The right question is therefore not "did llms.txt work?" It is which observable event changed, in which product, over which period, and what competing explanations remain. A file fetch, referral, citation and answer quality are separate observations.

The four main measurement approaches are:

1. First-party reports, where the relevant platform makes them available.
2. Server logs, direct evidence that a verified client fetched your file.
3. AI referral traffic, visits whose referrer is observable in your analytics.
4. Documented manual checks, repeatable observations of answers and cited URLs.
5. Brand monitoring, third-party measurements whose method you can inspect.

Use first-party AI visibility reports

Google announced dedicated generative AI performance reports in Search Console in June 2026. At publication, Google said the reports were rolling out to a subset of websites and could show impressions, appearing URLs, countries, devices for Search and time ranges. Use them when the report is present in your property, but do not infer a result when it is absent.

Bing's AI Performance dashboard reports total citations, average cited pages, grounding query phrases, page-level citation activity and trends. Bing explicitly says those citation metrics do not indicate ranking, authority, importance or placement inside an individual answer. Neither Bing nor Google says those metrics attribute credit to llms.txt.

Check your server logs

A server access log is the direct record of a request to /llms.txt. It can show the requested path, time, response, referrer, IP address and declared user-agent. Before naming a crawler, verify the request against the provider's documented identity method when that matters.

• Whether a verified client fetched the file.
• How often that verified client returned during the period you measured.
• Which declared client categories are represented in your data.

Start with documented user-agent roles rather than treating all AI-related names as equivalent:

• OAI-SearchBot, the OpenAI bot documented for ChatGPT Search.
• GPTBot, the OpenAI bot documented for content that may be used to train foundation models.
• ChatGPT-User, a user-initiated OpenAI visit, not automatic web crawling and not a Search eligibility signal according to OpenAI.
• Other declared agents only after checking their own current provider documentation.

If you run Nginx or Apache, a simple log grep looks like:

# Nginx access log, filter for llms.txt requests from AI crawlers
grep "llms.txt" /var/log/nginx/access.log | grep -E "GPTBot|ClaudeBot|PerplexityBot|OAI-SearchBot"

Do not turn a user-agent match into a claim that a model read the file. A validated request confirms a fetch, not parsing, retention, training, ranking, citation or influence on an answer.

Monitor AI referral traffic in GA4

A referrer can show that a visitor arrived from a recognisable AI product domain. It may be a useful business signal, but it does not by itself identify the answer, prompt, cited URL or the role of llms.txt. Referrer handling also varies by browser, app and privacy setting.

In Google Analytics 4, set up a comparison or exploration that segments sessions by referrer matching these domains:

• perplexity.ai
• chat.openai.com and openai.com
• claude.ai
• copilot.microsoft.com
• gemini.google.com

Track this segment over time and annotate material content, product and measurement changes. A sustained change is an observation worth investigating, not an attribution to a single file: new content, demand, model updates and referral handling can all affect the series.

Also inspect the landing pages. This can tell you which pages receive observable referrals; it cannot establish why an AI product selected them or whether their inclusion in llms.txt mattered.

Test citation accuracy

A manual check can reveal whether a particular product gives a useful answer on a particular date. Use questions that represent genuine customer needs, keep the wording stable and record the product, date, answer and cited URLs. Do not describe the result as a controlled experiment unless you have controlled the other variables.

Specific things to check:

• Are factual claims about your product accurate? (Correct pricing tiers, correct feature names, correct API endpoint paths)
• Are your pages cited as sources when relevant?
• Is the answer based on current information or does it reflect outdated information?
• Are competitor pages cited instead of yours for questions you should own?

Repeat the same check only where the product and access method are available to you. Keep the raw observations, including no-result cases, before and after a documented site change.

This qualitative check is limited, but it can answer a practical question: on the conditions you recorded, did the product give users accurate information about your business?

Track brand mentions in AI outputs

Several third-party tools have emerged to track brand mentions in AI-generated responses. As of early 2026, this space includes early-stage products focused on monitoring how often and how accurately your brand appears in AI answers. These tools typically work by running a set of queries across multiple AI assistants and aggregating the results.

These tools are still early-stage and their methodologies vary. If you are considering one, evaluate what queries they run, which AI systems they test against, and how they define a "mention." The category is evolving quickly, check current tool availability at the time you are reading this.

What these tools can give you that manual checks cannot: volume and trend data over time, without requiring you to run manual queries repeatedly.

Setting a baseline

Before making changes to your llms.txt (or publishing one for the first time), establish a baseline across each measurement:

1. Server logs: Note the current frequency of AI crawler fetches to /llms.txt (or zero if you have not published one yet) and to your key pages.
2. AI referral traffic: Record your current weekly session volume from Perplexity, ChatGPT, Claude, and similar referrers in GA4.
3. Citation accuracy: Run manual checks on 5–10 common questions about your brand. Record the quality of answers and which pages (if any) are cited.

Choose a comparison period that fits the frequency and volume of your data. State the period in the report, then compare it with the baseline without assuming when every external product will refresh its index or answers.

What not to confuse with impact

A few metrics that are easy to confuse with llms.txt impact:

• Google Search Console metrics. Google now reports selected generative AI visibility for eligible properties, but Google says Search ignores llms.txt. These metrics do not attribute visibility or ranking changes to the file.
• Direct traffic increases. Users typing your URL directly into a browser are not coming from AI citations. This metric is unrelated.
• Bot traffic volume alone. Some sites see large volumes of bot traffic from AI crawlers. The fact that a crawler fetched your page does not mean your content appeared in any AI-generated answer. Volume of crawl does not equal volume of citation.
• Short-term fluctuations. AI system behavior changes with model updates, retrieval index refreshes, and product changes at the AI companies. A short-term change in your AI referral traffic may have nothing to do with your llms.txt.

Continue reading

• Does llms.txt work?, an honest assessment of the evidence by use case.
• AI crawlers explained, understand which bots are fetching your files.
• llms.txt and SEO, what it does and does not do for search rankings.
• Validator, make sure your file is spec-compliant before tracking impact.