Does llms.txt Actually Do Anything? The Honest Verdict

The Verdict

No - llms.txt has not been shown to improve AI-search visibility in any measurable way, as of mid-2026. Not "adoption is still emerging." Not "some providers check it, some don't." In Ahrefs' May 2026 sample of 137,210 traffic-receiving domains using Ahrefs Web Analytics, about 38,000 published a valid llms.txt file, and 97% of those received zero requests of any kind that month - a strong one-month base rate against usage, not proof that every implementation everywhere goes unread, or that it always will. Google's own Search Central documentation says site owners don't need new machine-readable or AI text files to appear in AI Overviews or AI Mode - narrower than "Google doesn't use them," and worth stating precisely rather than rounding up. If you're deciding whether llms.txt deserves an engineering ticket this quarter ahead of access, rendering, and content work: it doesn't. But treat it as low-cost supporting infrastructure, not literally free or risk-free - more on that below.

What llms.txt Is Supposed to Do

llms.txt is a proposed plain-text file, placed at a site's root, meant to give an AI system a curated, markdown-formatted summary of a site's most important pages - the idea being that a language model could read this one file instead of trying to parse an entire site. It's a reasonable idea in principle. The problem isn't the concept - it's that current usage is limited and uneven: most published files go unread, and no major AI-search system has demonstrated that reading one changes what gets cited or ranked.

What the Evidence Actually Shows

Two numbers tell most of the story, and they point in different directions. Adoption is genuinely growing: Originality.ai's own tracker, which monitors more than three million websites, recorded 4,088 llms.txt instances in June 2025 and 36,120 by May 2026 - an 8.8x increase in twelve months, within that monitored population, not a census of the entire web. Usage is lagging badly behind adoption: in Ahrefs' May 2026 sample of 137,210 traffic-receiving domains, about 38,000 had a valid llms.txt file, and 97% of those got zero requests of any kind that month. Ahrefs itself notes its customer base skews more technical and SEO-aware than the web generally, so treat both figures as a strong signal from a specific monitored population, not a universal law.

That's not quite the same as "nothing reads it." Of the roughly 3% of files that did get traffic, Ahrefs counted about 22,000 total requests, and roughly one in five (19.5%) came from named AI tools rather than generic or unidentified traffic - plus a small number (about 22, or roughly 1 in 1,000 of all fetches) that were Chrome's own Lighthouse audit checking whether the file exists at all. Some AI labs and documentation ecosystems publish or consume the format directly: OpenAI, Anthropic, and Google all publish llms.txt files for their own developer docs, and Anthropic's and Perplexity's are both live, current resources as of this writing. None of that demonstrates the file changes what gets ranked, cited, or included in an AI answer - it demonstrates limited, uneven, mostly tooling-and-documentation use. Not zero use, and not proven visibility impact either.

What the AI Companies Themselves Say

This isn't a case of "no comment" from the major providers. Google Search's own current documentation states: "You don't need to create new machine readable files, AI text files, or markup to appear in these features" (AI Overviews and AI Mode) - in Google's own words, "there are no additional technical requirements." That's narrower than "Google doesn't use these files anywhere": it's a statement about what site owners don't need to create, not a blanket claim about every Google product or crawler - and Chrome's own Lighthouse tool now includes an optional audit that checks whether a site's llms.txt file is reachable, which is evidence that at least one Google-owned tool looks for the file, even though Google Search itself says it isn't required. Google's John Mueller put it more informally on Reddit: "AFAIK none of the AI services have said they're using LLMs.TXT…it's comparable to the keywords meta tag" - a reference to the old meta keywords tag that search engines have publicly ignored for decades. Google's Gary Illyes has also been reported saying Google has no plans to support it, generally attributed to a Google Search Central Live appearance; we couldn't independently confirm a specific date for that statement, so it's included here as reported rather than dated. Neither OpenAI, Anthropic, nor Perplexity has published anything committing to honor a third-party site's llms.txt file - though notably, all three publish their own llms.txt files for their own developer documentation, which is a different claim than committing to read everyone else's.

Should You Still Add It?

If your CMS or static site generator can generate and maintain a small, accurate llms.txt automatically, the marginal cost and risk are low - but "low" isn't "zero." A stale file that lists moved or removed pages, or one an attacker manages to alter, misleads any agent that trusts it; Ahrefs' own study flags this directly, recommending version control, restricted edit access, and change alerts for anything sensitive enough to matter, and keeping entries as plain links and descriptions rather than instruction-shaped text a compromised file could weaponize. What doesn't make sense is treating it as a project: don't delay a real fix (SSR, prerendering - see the pages below) to prioritize llms.txt, don't expose private or stale resources in it, and don't present it to a client or a boss as a meaningful AI-visibility improvement. The evidence right now says it isn't one.

What to Do Instead

The things that actually move the needle for AI crawler access are rendering, access, and discovery fixes, not a courtesy file - and it's worth being precise that those are three different jobs, not one. robots.txt controls whether a crawler is permitted to request a URL at all; sitemap.xml helps a crawler discover and prioritize which canonical URLs exist; making sure your content actually exists in the raw HTML response is a separate rendering/delivery question, covered elsewhere in this cluster. Getting content into raw HTML is a necessary condition for a non-rendering bot to read it at all - but it isn't the whole visibility equation on its own. Relevance, authority, indexability, freshness, and each provider's own retrieval and ranking systems still determine whether that content gets selected or cited once a crawler can see it. robots.txt and sitemap.xml, unlike llms.txt, are protocols the major crawlers actually parse and act on for access and discovery - they just don't solve rendering by themselves, and none of the three substitute for the others.

← Back to the full pillar page
The fix that actually matters: SSR vs. prerendering →
The full implementation checklist →

---

Sources: Adoption growth (4,088 → 36,120 instances, 8.8x, June 2025-May 2026, across a 3M+-site tracked population) from Originality.ai's own tracking study, cited directly; ppc.land kept as optional secondary coverage. Usage data (137,210 domains studied, ~38,000 with a valid file, 97% of those zero-request, ~22,000 total requests, 19.5% from named AI tools, May 2026) from Ahrefs' primary study, including its own stated sample-skew caveat and its security/maintenance guidance. Google's official language on AI Overviews/AI Mode requirements, quoted directly rather than via a secondary summary, is from Google Search Central's AI-features documentation; the optional llms.txt check is from Chrome Lighthouse's llms.txt audit documentation. Mueller's Reddit quote is corroborated via Search Engine Journal's reporting; the Illyes statement is included as reported secondhand, without a confirmable date. Provider llms.txt publication confirmed directly at Anthropic's own llms.txt and Perplexity's own llms.txt; general format background from the llms.txt proposal's own site. robots.txt and sitemap definitions are from Google's own robots.txt introduction and sitemap-building documentation. This verdict reflects the evidence as of mid-2026 - worth rechecking if you're reading this more than a few months later, since this is exactly the kind of claim that could change with one announcement.