llms.txt: Does It Actually Do Anything Yet?

llms.txt: Does It Actually Do Anything Yet?

Bottom line: No - as of mid-2026, llms.txt does almost nothing for AI search visibility. No major AI engine (Google, OpenAI, Perplexity) fetches it to rank or cite you, and a 300,000-domain study found zero measurable citation lift. It's a proposed standard the ecosystem hasn't adopted. Skip it as a ranking play; ship clean HTML, schema, and crawlable content instead.

Most GEO checklists floating around right now put "add an llms.txt file" near the top, right next to schema and internal linking. We audit growth-stage sites for AI-search visibility every week, and here's the uncomfortable part: in every single case where a team had shipped an llms.txt file, the server logs told the same story - the AI crawlers they built it for never requested it. Not once. The file sat there as a well-intentioned artifact that no engine read. That's not a niche edge case. It's the base rate.

Key Takeaways:

• —Google publicly does not support llms.txt. Gary Illyes said so at Search Central Live in July 2025, and John Mueller compared it to the deprecated keywords meta tag - a signal engines learned to ignore.
• —A SE Ranking study of ~300,000 domains found 10.13% adoption overall, 0% among the top 1,000 sites by traffic, and no citation lift once authority, schema, and content recency were controlled for.
• —Ahrefs tracked 137,210 domains hosting the file and found 97% got zero requests for it in a single month - the file exists, but nothing reads it.
• —The one place llms.txt is genuinely used today: IDE coding agents (Cursor, Cline, Continue) and some documentation-heavy dev-tool sites - not consumer AI search.
• —What actually earns citations right now: crawlable HTML, structured data, entity clarity, and content freshness. Ship those first.

What is a llms.txt file, exactly?

llms.txt is a proposed text file you place at your root (yoursite.com/llms.txt) that gives large language models a curated, markdown-formatted map of your most important content. Think of it as a table of contents written for machines: a short description of your site plus links to the pages you most want an LLM to read, sometimes with a companion llms-full.txt that inlines the actual content.

The idea, proposed by Jeremy Howard of Answer.AI in late 2024, is reasonable on paper. LLM context windows are finite. HTML is noisy - nav bars, cookie banners, ad slots, JavaScript. A clean markdown digest would help a model spend its tokens on your substance instead of your chrome. robots.txt tells crawlers what they can't touch; llms.txt was pitched as the inverse - here's what you want read, pre-cleaned.

Good idea, wrong assumption. It assumes the AI engines were waiting for a curated file. They weren't. They already render full pages, parse structured data, and extract passages at scale. A separate file they have to trust - and can't verify against what human visitors see - solves a problem they'd mostly already engineered around.

Does llms.txt actually work in 2026?

The honest answer, backed by the two largest datasets published so far: no, not in any way you can measure.

The SE Ranking analysis of roughly 300,000 indexed domains is the one to anchor on. Overall adoption sat at 10.13% - roughly one in ten sites after 18 months of hype. But the revealing cut is by traffic tier: adoption among the top 1,000 sites by traffic was 0%. The sites winning AI citations most often are precisely the ones that skipped the file. When SE Ranking's model controlled for domain authority, schema density, and content recency, llms.txt added no predictive value for citations. Their XGBoost model actually got more accurate when the llms.txt variable was removed - meaning, statistically, the file behaved like noise, not signal.

From the read side, Ahrefs looked at 137,210 domains hosting an llms.txt file and found 97% received zero requests for it in the month they measured. That's the whole ballgame. You can't get a citation benefit from a file that nothing fetches.

None of this means the concept is dead - it means the adoption hasn't happened. Those are different failure modes, and the distinction matters for how you should act.

Which AI crawlers actually support llms.txt? (The support-status audit)

This is the part every "add llms.txt today" post skips: an honest, engine-by-engine read on who fetches and uses the file versus who has publicly declined. We pulled this together from vendor statements, documented crawler behavior, and the adoption studies above. Here's the current status.

The pattern is clear once you lay it out. The only place llms.txt reliably does something today is developer tooling - coding assistants pulling clean docs into a context window on demand. That's a real, useful function. It is also almost entirely disconnected from the reason most marketers are being told to ship the file, which is consumer AI search visibility.

Worth flagging one nuance so you don't over-correct: several AI companies publish an llms.txt for their own docs (OpenAI and Anthropic both do). That's a signal the file is useful for agent-readable documentation - not evidence that their crawlers consume yours for ranking. Don't confuse "they host one" with "they read yours."

What should you implement instead while the standard settles?

Here's the contrarian-but-boring truth: the things that get you cited in AI answers are the same technical fundamentals that have mattered all along, executed with more discipline. Google itself has said you don't need llms.txt, special markup, or markdown to appear in AI Overviews or AI Mode. So spend the effort here instead, in priority order:

1. Make your content actually crawlable. If GPTBot, Google-Extended, or PerplexityBot can't render your page - heavy client-side JavaScript, aggressive bot-blocking, slow server response - no file fixes that. This is where most "invisible in AI" cases actually originate. Run an AI SEO audit to see exactly which AI user-agents can reach your key pages.
2. Ship structured data that maps your entities. Article, FAQPage, Organization, and Product schema give engines machine-readable facts they can lift with confidence. This is the closest working substitute for what llms.txt promised. See our breakdown of which schema types actually help you get cited.
3. Answer questions in liftable passages. AI engines quote self-contained blocks - a direct answer in the first sentence, then evidence. Front-load the answer, then support it.
4. Build entity and topical clarity. Consistent naming, an "about" that states who you are plainly, and coverage depth on your core topics. This is the mechanics behind how large language models pick their sources.
5. Keep content fresh. Recency was one of the few variables SE Ranking found did correlate with citations. Update your cornerstone pages on a schedule.

If you want the full mental model behind all five, our Generative Engine Optimization guide walks through the ranking signals engines actually weigh.

How much does it cost to add llms.txt - and is it worth it?

Almost nothing, which is exactly why so many people ship it: a small text file, maybe an hour to generate and maintain. The cost isn't the file. It's the opportunity cost and the false confidence.

We repeatedly see teams add llms.txt, check the "AI-ready" box, and move on - while the actual blockers (a JS-rendered pricing page GPTBot can't parse, missing Organization schema, a robots.txt that accidentally blocks AI crawlers) go untouched. The file becomes a placebo. It feels like AI-search work without moving the metric.

So is it worth it? If it takes 30 minutes and you've already nailed the fundamentals above, sure - ship it as a low-cost hedge in case adoption accelerates. It won't hurt you. But if adding llms.txt is your first AI-search move, you've optimized the one thing with a proven-zero return and skipped the things with measurable impact. That's the trap.

Will llms.txt matter in the future?

Maybe - and that's a defensible reason to keep a lightweight file around without betting your strategy on it. Standards do get adopted after slow starts. If a major engine reverses course and starts honoring llms.txt, the sites that already maintain a clean one benefit on day one at near-zero switching cost.

But treat that as a call option, not a core holding. The base rate for proposed web standards that lack buy-in from the dominant crawlers is not encouraging - Mueller's keywords-meta-tag comparison is pointed precisely because that tag was also a reasonable idea that engines chose to ignore once it became a cloaking and spam vector. llms.txt carries the same structural risk: it lets you show bots one curated version and humans another, which is the definition of the cloaking pattern engines are built to distrust.

Watch for one specific trigger: a public statement from Google, OpenAI, or Perplexity that their search crawlers consume llms.txt for ranking or citation. Until that happens, the file is documentation infrastructure for coding agents - useful in that lane, inert in yours. Our journal tracks these shifts as they land.

How We Assessed This

This verdict isn't a hot take - it's built from the two largest public datasets on llms.txt plus our own audit pattern. We anchored on SE Ranking's ~300,000-domain analysis (adoption by traffic tier, plus a controlled model isolating llms.txt's effect on citations) and Ahrefs' 137,210-domain read-side study measuring how often the file is actually requested. We cross-checked engine positions against on-record statements from Google's Gary Illyes and John Mueller, and against what OpenAI and Anthropic publish for their own docs.

On the client side, our GEO audits inspect server-log requests from named AI user-agents (GPTBot, Google-Extended, PerplexityBot, ClaudeBot), crawlability of key pages, schema coverage, and citation share across ChatGPT, Perplexity, and Google AI Overviews. As an AI-native SEO agency running 12-month optimization cycles for growth-stage sites, the consistent finding is that llms.txt correlates with zero crawler activity in the logs, while schema, crawlability, and freshness correlate with real citations. We report what the data shows, including the parts that contradict popular advice.

FAQ

Should I add an llms.txt file to my site in 2026?

Only after your fundamentals are done. If AI crawlers can already reach your pages, your schema is clean, and your content is fresh, a 30-minute llms.txt file is a harmless hedge. If it's your first AI-search move, you're optimizing the one thing with proven-zero return - fix crawlability and schema first.

Does Google use llms.txt for AI Overviews or rankings?

No. Google's Gary Illyes confirmed no support and no plans at Search Central Live in July 2025, and John Mueller compared the file to the deprecated keywords meta tag. Google has separately stated you don't need any special file to appear in AI Overviews or AI Mode.

Does llms.txt help me get cited by ChatGPT or Perplexity?

There's no evidence it does. OpenAI hasn't adopted it for crawling and Perplexity has made no commitment. The 300,000-domain SE Ranking study found no citation lift attributable to llms.txt once authority, schema, and recency were controlled for.

What's the difference between llms.txt and robots.txt?

robots.txt tells crawlers what they may not access and is broadly honored by every major AI crawler operator. llms.txt tries to tell LLMs what they should read - but almost no consumer AI engine fetches it. One is enforced infrastructure; the other is a proposal awaiting adoption.

Who actually reads llms.txt today?

Primarily IDE coding agents - Cursor, Cline, and Continue - which pull clean documentation into a context window when you point them at a domain. That's a genuine, working use case, but it's about developer tooling, not the AI search visibility most marketers are chasing.

If llms.txt does nothing, what actually gets me into AI answers?

Crawlable HTML, entity-mapping structured data, self-contained answer passages, topical depth, and content freshness. These are the signals the data ties to real citations - and they're where your effort belongs while the llms.txt standard settles.

Get an honest read on your AI-search visibility

If you've shipped an llms.txt file and you're not sure whether AI engines can even reach your important pages, don't guess - check the logs. Run our free AI SEO audit to see which AI crawlers actually access your site, where your schema and crawlability gaps are, and what's really standing between you and a citation in ChatGPT, Perplexity, or Google AI Overviews.

Want a second opinion from a team that reports the losses alongside the wins? Book a strategy call and we'll walk your site's AI-search footprint with you - no placebo checkboxes, just the fixes that move citations.