A new B2B software benchmark says 41% of the companies in its sample have already published an llms.txt file. That sounds like rapid adoption for a web convention proposed only two years ago. The more revealing number, however, may be the one the study cannot provide: how many of those companies can demonstrate that the file caused an AI system to cite, recommend or even retrieve them more often.

The finding comes from AI visibility company Treyci and was examined by NetContentSEO on September 5. Treyci says 41 of 100 B2B SaaS companies in its scan served an llms.txt file. Its public announcement does not present a controlled comparison between adopters and non-adopters, so the result measures implementation rather than effectiveness.

That distinction has become unusually important because llms.txt sits at the intersection of two very different ideas. One is an engineering convention for helping agents navigate websites and documentation efficiently. The other is a marketing claim that publishing the file improves visibility in consumer AI answers. Evidence for the first is accumulating. Evidence for the second remains weak.

The original proposal was about machine usability, not rankings

Jeremy Howard's llms.txt proposal was introduced in September 2024 as a simple way to give language models a concise, predictable map of a website. Instead of forcing an agent to extract meaning from navigation, advertising, JavaScript and large HTML documents, a site can publish a Markdown file that explains what the project is and links to important resources.

The proposal has evolved with real-world use. Version 2, updated in August 2026, says thousands of sites now publish the file, documentation platforms generate it automatically and coding agents use it to locate API references and tutorials. It also adds standard link relations that can advertise Markdown versions of pages and the relevant llms.txt file to clients.

That is a legitimate interoperability story. A coding agent explicitly sent to a documentation site can save tokens and time if the site provides a clean map to the exact reference material it needs. The mechanism is straightforward and testable: observe whether the agent requests the file, follows its links and completes the task more efficiently.

What does not automatically follow is that ChatGPT, Gemini, Perplexity or another answer engine will rank a company higher because the file exists. A machine-readable navigation aid is not inherently a recommendation signal.

Google has now made its position unusually explicit

For Google Search, the ambiguity is largely gone. In its 2026 guidance for generative AI search, Google specifically names llms.txt among the tactics site owners do not need for AI Overviews or AI Mode. Google says Search does not use the file and that publishing one will neither improve nor damage visibility or rankings in Google Search.

That statement matters because some GEO and AEO advice still presents llms.txt as a generic AI-search optimization requirement. It cannot accurately be described that way when one of the world's largest AI-search products explicitly says it ignores the file for Search.

Google's position does not mean the convention is useless. Chrome's agentic browsing work and the broader agent ecosystem address a different problem from Google Search ranking. An agent operating a website on a user's behalf may value a compact machine-oriented guide even though Google's search systems do not.

The distinction is becoming a recurring theme in AI optimization: search engines, model-training crawlers, retrieval bots, browser agents and coding assistants are all machines, but they are not one audience.

The strongest large-scale evidence is surprisingly pessimistic

The most substantial public traffic study so far comes from Ahrefs, which analyzed 137,210 domains using its Web Analytics product and their bot traffic during May 2026. Ahrefs found valid llms.txt files on about 28% of those domains.

Adoption was not the surprising part. Of the valid files, 97% received no requests at all during the month studied. Among the small minority that were fetched, only 19.5% of requests came from categories Ahrefs classified as AI bots. Retrieval bots associated with live AI search represented just 1.1% of total requests to the files.

The sample has limitations. Ahrefs says its analytics customers skew toward technically sophisticated and SEO-aware websites, so the 28% adoption figure should not be treated as representative of the whole web. A single month also cannot establish future behavior, and a crawler may obtain information through another path.

Still, the study introduces a hard constraint on causal claims. If the AI system a marketer wants to influence never requests the file, it is difficult to argue that the contents of that file directly changed a live recommendation through retrieval.

Even a fetch does not prove influence

Server logs can establish that a file was requested, but that is only the first link in a longer evidence chain. A bot may fetch llms.txt without parsing it. An agent may parse it without following any links. It may retrieve linked content without using that content in generation. It may use the information without changing which brand it recommends.

This is why claims that llms.txt “works” need an explicit outcome. Does it reduce tokens used by an agent? Improve task completion? Increase factual accuracy? Raise citation frequency? Increase brand mentions? Increase the probability of being recommended in a commercial prompt?

Those are different experiments. A file could be excellent for developer documentation and irrelevant to brand recommendation. It could help an agent find a pricing page without making the product more competitive. It could even improve factual extraction while reducing the need for the agent to visit other pages.

The phrase “AI visibility” is broad enough to conceal all of those distinctions.

Adoption is spreading faster than causal evidence

Treyci's 41% figure is not an isolated sign of adoption. A separate August census by AfterLaunch checked 170 SaaS and developer-oriented domains and found that 45.9% served an llms.txt file. Adoption reached 84.2% among 19 well-known SaaS and developer-tool companies in that particular sample and 68.8% among companies selling AI-search tools.

Different samples produce different percentages, which is expected. What is consistent is that the convention has moved well beyond a tiny experiment. CMS products and documentation platforms increasingly generate the file automatically, reducing implementation cost close to zero.

That creates an adoption problem in reverse. Normally a useful standard struggles because nobody implements it. Here, implementation can spread before efficacy is settled because publishing one Markdown file is cheap, visible and easy to turn into a checklist item.

The resulting feedback loop can look like evidence. Competitors publish the file, consultants recommend it because competitors have it, platforms add generators because customers request them, and higher adoption then becomes the next argument for implementation. None of those steps requires a measurable improvement in AI recommendations.

The agentic web may ultimately justify the convention

There is nevertheless a stronger case for llms.txt than simply betting on an SEO trick. The web is becoming more agentic. Coding assistants already read documentation, browser agents can navigate sites and commercial agents increasingly need to locate product information, policies and instructions.

The August update to the specification reflects that shift. It now describes agents as the primary audience more directly and adds mechanisms for clients to discover Markdown alternatives without guessing URLs. The design is moving toward practical machine navigation rather than an abstract request that every LLM somehow inspect a root file.

Ahrefs' own traffic data offers some support for this interpretation. Among the files that were requested, agentic tools were a larger AI category than live AI-search retrieval bots. Claude Code was among the notable consumers. That is much closer to the original engineering rationale: a tool navigating technical resources, not a search engine awarding visibility points.

For developer-facing companies, that distinction can still have commercial value. If prospective customers ask coding agents to evaluate libraries, integrate APIs or troubleshoot products, making documentation easier for those agents to consume may improve the product experience even if it never moves a ChatGPT citation metric.

Recommendation depends on evidence outside the vendor's site

The challenge becomes harder when the desired outcome is inclusion in a buying shortlist. Treyci says its monitoring runs roughly 100 buying-intent questions per category across ChatGPT, Gemini, Perplexity and Grok, repeating prompts and separately scoring mentions, recommendations and citations. In one category, it observed almost a twofold difference between engines in how often tracked brands appeared.

The company also reports that commercial answers frequently cite review platforms, comparison articles and industry publications rather than the vendors themselves. If that pattern holds, a perfectly constructed llms.txt file addresses only one small part of the evidence environment.

An answer engine deciding which CRM, analytics platform or developer tool to recommend may consider product capabilities, pricing, documentation, reviews, third-party comparisons and reputation. A vendor-controlled summary can make first-party facts easier to locate, but it cannot manufacture independent corroboration.

That makes original research, clear product information, credible reviews, digital PR and authoritative third-party coverage difficult to replace with any single technical artifact.

The missing experiment is now straightforward to design

The industry's next step should be measurement rather than another round of implementation tutorials. A company can establish a stable panel of commercially relevant prompts and run each repeatedly across the AI engines that matter to its customers. It can record mention rate, recommendation rate and citation rate before publishing llms.txt.

After deployment, server logs can show whether relevant agents actually request the file. The same prompt panel can then be repeated over several weeks. Ideally, comparable products, site sections or domains that did not receive the change can act as controls.

This will never be a perfect laboratory experiment. Models, indexes and retrieval systems change continuously. But it is far stronger than adding the file alongside a redesign, PR campaign and documentation rewrite and then attributing every subsequent AI mention to llms.txt.

Engine-level reporting is essential as well. Google Search has already said it ignores the file. An agentic tool may actively consume it. Another AI product may change behavior later. Averaging those systems into one “AI visibility score” could erase the very effect the experiment is trying to detect.

Low cost makes llms.txt a reasonable experiment, not a proven best practice

There is little reason for the debate to become binary. A company does not need to believe llms.txt is a ranking factor to justify spending a small amount of engineering time on it. For a documentation-heavy product, the file may already serve useful agents. It is inexpensive optionality if maintained accurately.

But low implementation cost does not lower the standard of evidence for marketing claims. “We have llms.txt” is an implementation metric. “Agents that read it complete documentation tasks more reliably” is an outcome. “Our recommendation rate increased after deployment under repeated controlled testing” would be stronger evidence still.

The current state of the evidence supports the first statement widely and the second in particular agent workflows. It does not yet establish the third as a general rule for AI search.

That makes the 41% adoption figure interesting for a reason almost opposite to the obvious one. It does not show that llms.txt has won. It shows how quickly the web can standardize around a plausible, inexpensive intervention before anyone has convincingly measured the outcome marketers care about. The convention's future may still be substantial, especially for agents. But in 2026, the right label for llms.txt as an AI-visibility tactic remains simple: testable, increasingly common and not yet proven.