There’s a new file fighting for a spot in your root directory. llms.txt is a proposed standard that hands AI systems a Markdown-formatted summary of your site’s most important content, placed at yourdomain.com/llms.txt. Think of it as a curated menu for large language models, written in a format they process efficiently. Jeremy Howard of Answer.AI proposed the spec in 2024, and adoption has moved fast since then.
At Gorilla Marketing, we test emerging AI standards against real data as part of our AI optimization work. The verdict on llms.txt right now? Easy to set up, zero proven citation impact, but enough platform-level signals that ignoring it entirely might be premature. Here’s the full breakdown.
What llms.txt Actually Does
The file lives at your domain root alongside robots.txt and sitemap.xml. Each of those three files talks to machines, but they say different things.
robots.txt sets boundaries. It tells crawlers where they can’t go. Sitemaps provide a complete inventory. They tell crawlers what exists. llms.txt does something neither of those files does: it curates. It points AI systems to the 10-30 pages you most want them to understand and reference, presented in clean Markdown instead of HTML.
The Markdown structure follows a specific pattern:
An H1 with your site name
A blockquote summarizing what the site covers
H2 sections grouping content by category (“Documentation”, “Services”, “Blog”)
Bulleted links under each section, each with a brief description of the page
There’s also an optional companion file, llms-full.txt, containing the full Markdown content of your selected pages. Mintlify’s data shows AI systems access the full-text version more frequently than the pointer file, which makes sense. AI wants the content itself, not just a directory.
The technical advantage is token efficiency. A typical HTML page wastes thousands of tokens on nav elements, JavaScript, CSS and boilerplate before the AI reaches anything useful. The Markdown version strips all of that away and delivers the same information at a fraction of the token cost. For AI systems working within context window limits or per-page token budgets, that’s a meaningful difference.
Adoption: Fast Growth, Uneven Distribution
Over 844,000 sites had llms.txt in place by October 2025. When SE Ranking examined 300,000 domains, about 10% had implemented the file.
The adopter profile tells a story. Anthropic, Cloudflare, Vercel, Stripe, Hugging Face, Zapier, Cursor. These are technology companies with documentation-heavy sites that AI coding tools already consume constantly. The standard was born in that world, and that’s where it’s sticking.
One counterintuitive finding from SE Ranking: mid-tier sites (1,001-5,000 monthly visits) adopted at a higher rate (10.54%) than high-traffic domains with 100,000+ visits (8.27%). The big players aren’t leading the charge here. Mid-size tech companies are.
CMS platforms are removing the implementation barrier entirely. Yoast, Wix and multiple static site generators now generate the file automatically. Once creating the file requires checking a box instead of writing Markdown, adoption will accelerate regardless of whether the evidence supports it.
The Citation Evidence: Not Good
Here’s where the hype meets reality.
SE Ranking studied 300,000 domains and found zero correlation between llms.txt and AI citation frequency. Their XGBoost model actually predicted citations more accurately when llms.txt variables were removed. The file was adding noise, not signal.
ALLMO.ai analyzed over 94,000 cited URLs and found the same result: no citation uplift from having llms.txt in place.
OtterlyAI ran a 90-day tracking experiment on a site with llms.txt active. Out of 62,100 total AI bot visits to the site, only 84 hit the /llms.txt file. That’s 0.14% of AI crawler traffic. Bots weren’t even looking at it.
Google has been direct about its position. Gary Illyes said in July 2025 that Google doesn’t support llms.txt and has no plans to. John Mueller put it more bluntly, comparing it to the keywords meta tag, a standard that Google never used for rankings despite years of SEO industry obsession.
And the bottom line: no major AI platform (OpenAI, Google, Anthropic) has publicly stated that llms.txt factors into source selection decisions.
The Counterarguments Worth Hearing
The data above is damning. But dismissing llms.txt completely requires ignoring some signals.
Google built llms.txt into its A2A (Agent-to-Agent) protocol. This doesn’t mean Google Search reads the file, but it tells you Google sees llms.txt as part of how AI agents will interact with websites going forward. Search and agentic AI are different use cases, and llms.txt may matter more for the second one.
Anthropic, the company that built Claude, has asked partners to implement llms.txt specifically. They see value in the format for their systems, even if no study has captured that value yet.
Profound’s crawl data shows Microsoft and OpenAI bots actively visiting llms.txt files. The bots are reading the file. Whether the citation algorithms care about what they find is a separate question.
Vercel attributes 10% of its signups to ChatGPT and considers llms.txt part of the broader AI optimization strategy that produced those results. Isolating the specific contribution of one file is impossible, but the company isn’t removing it.
Mintlify’s framing resonates: documentation is now written for two audiences, humans and LLMs. If that’s true for documentation, llms.txt is just the logical extension.
The honest read: llms.txt does nothing measurable for AI citations today. But the platforms know about it, some crawl it, and the direction of AI search points toward more structured signals mattering over time. Whether that “over time” means six months or six years is anyone’s guess.
Where Implementation Actually Makes Sense
Four situations justify the effort.
You run a documentation-heavy site. If AI coding tools like Cursor, Copilot or ChatGPT already consume your docs, llms.txt gives them a cleaner path to the content that matters. This is the original use case and the one with the most practical value right now.
Your site has hundreds or thousands of pages. For large, complex sites, llms.txt functions as a curated shortlist. Even if AI systems ignore the file today, the exercise of identifying your 10-30 most important pages and writing crisp descriptions of each has strategic value on its own.
You want a cheap hedge. Building a basic llms.txt file takes under an hour. Zero downside risk. If the standard starts mattering later, you’re already in position. The effort-to-potential-upside ratio is about as good as it gets in SEO.
You’re already doing everything else. Content quality, entity optimization, schema markup, answer-first content structure. If those fundamentals are in place, llms.txt is a reasonable addition at the margin. If they’re not, start there.
Building the File
The process takes minutes once you’ve made the content decisions.
Pick your pages. Choose 10-30 URLs. Target the pages you most want AI to understand and reference: service pages, flagship guides, documentation, key informational content. Consider what users will ask AI tools about your business or industry and pick the pages that answer those questions. Your sitemap handles comprehensive discovery. llms.txt handles curation. Don’t confuse the two.
Write it in Markdown. Here’s the template:
# Your Company Name
> One or two sentences describing what your business does and who it serves.
<h2>Services</h2>
<a href="https://yoursite.com/service">Service Page</a>: What this service covers and who it's for
<a href="https://yoursite.com/another">Another Service</a>: Brief description
<h2>Guides</h2>
<a href="https://yoursite.com/guide">Guide Title</a>: What this guide covers
Deploy to root. Upload to yourdomain.com/llms.txt. Confirm HTTPS access, 200 status code, text/plain content type.
Consider llms-full.txt. If you want AI to access full content without crawling HTML, create a companion file with the complete Markdown text of your selected pages.
Keep it current. Quarterly reviews minimum. Update when significant new content publishes. An outdated llms.txt actively hurts by directing AI to stale pages.
Four things to avoid: listing every page on the site (that’s a sitemap, not curation), using HTML formatting (Markdown only), skipping the blockquote description (it’s context AI systems need), and treating implementation as a one-time task.
Putting llms.txt in Perspective
This is a minor tactical element. It is not a strategy and it’s not a priority. The actions that actually move AI citation rates (content quality, topical authority, proprietary data, answer-first structure, entity clarity) should get your time and budget first.
llms.txt is the last 2% of an AI visibility strategy. Worth doing once the other 98% is handled. Not worth doing instead of it.
Gorilla Marketing’s AI optimization services and technical SEO work evaluate emerging standards like llms.txt against actual client data, not industry hype. Get in touch for an honest assessment of where it fits for your site.
