llms.txt for SEO: What It Actually Does in 2026

Every AI visibility vendor is pitching the same fix right now: llms.txt for SEO. Add one file to your root directory, the pitch goes, and ChatGPT and Perplexity will finally understand your site. The file is real. Jeremy Howard proposed it in September 2024, and a fair number of serious technical SEO teams already use it. What it is not is a ranking signal or a citation guarantee. It carries nothing close to robots.txt's enforcement weight. Here's what llms.txt actually does, what the current evidence says, and when building one earns the hour it costs.

What an llms.txt file actually is

An llms.txt file is a plain markdown document that lives at yourdomain.com/llms.txt. It lists a site's most important pages, each with a short, factual description. That lets a language model find what matters without parsing an entire page of navigation, scripts, and boilerplate.

The format borrows its placement from robots.txt and its ambition from a sitemap. robots.txt tells a crawler where it's allowed to go. A sitemap tells a search engine what pages exist. llms.txt tries to add a third layer: which pages actually matter, and why. The format costs a model a fraction of the tokens a full HTML crawl would. A companion file, llms-full.txt, goes further. It bundles entire page content into one markdown dump, so an agent can read a whole site in a single fetch instead of crawling page by page.

If you're still working out how AEO stacks up against SEO, that distinction matters here too. llms.txt is an AEO-side tactic, built for how models read, not for how Google ranks.

What AI crawlers actually need from your site

Googlebot can afford to crawl a ten-thousand-word page, full navigation and all. It has the time, the storage, and a mature ranking pipeline built for exactly that job. A language model answering a question in real time doesn't get that luxury.

Every page it reads costs tokens, and tokens cost the AI company money and the user latency. That's the actual gap llms.txt was built to close. It hands a model a curated, markdown-only summary of what to read. That beats making it parse your whole page to find out.

File	Built for	What it controls
robots.txt	Search engines and AI crawlers	Access, where a bot can go
sitemap.xml	Search engines	Discovery, what pages exist
llms.txt	AI models and agents	Context, which pages matter and why

None of the three can force a model to do anything. llms.txt carries no directives. A model can read it, ignore it, or never fetch it at all. Nothing on your site changes either way.

Does llms.txt for SEO actually work? What the evidence shows

The honest answer is thin, so far. OpenAI, Google, and Anthropic haven't confirmed their models read llms.txt files when building an answer. One audit of the fifty domains cited most often by AI search tools found only one had an llms.txt file live. That isn't an adoption curve yet. It's a handful of early movers.

So why do technical SEO teams keep shipping it anyway? A narrower slice of AI agents does use it: tools built to explicitly fetch llms.txt when they find one. Think developer tools, research assistants, and documentation crawlers. If that's the kind of visitor your site serves, the signal lands. If your traffic comes from ChatGPT or Google's AI Overviews summarizing a general question, it mostly doesn't, at least not yet.

That gap is worth sitting with before you brief a developer. Our AEO and generative engine optimisation work treats llms.txt as one small piece of a bigger structural job, not the whole project. The content, schema, and answer formatting underneath it do more of the actual work.

Where llms.txt for SEO earns its keep, and where it doesn't

The audience your site actually attracts decides whether this file is worth building. A SaaS company with a documentation site, an API reference, or a developer-facing product sits in the category where llms.txt has real uptake. Coding assistants and technical research tools are the ones most likely to fetch it on purpose. Their users ask precise, implementation-level questions, and a curated file answers those faster than a full crawl.

A local service business, an e-commerce store, or a consumer brand sits in a different category. The AI traffic reaching those sites mostly comes through general-purpose chat and AI Overviews summarizing broad questions. Neither currently confirms reading llms.txt at all. For that kind of site, the file is a low-cost bet with no measurable downside, not a priority project.

Either way, a stale llms.txt is worse than no file. Pages move. Content changes. A file that still lists outdated pages, or describes content that no longer matches the live page, actively misleads any agent that reads it. Treat it the way you'd treat a sitemap. It needs an owner and a refresh cadence, not a one-time ship-and-forget project.

How to build an llms.txt file properly

Building one takes an afternoon, not a sprint. The file is plain markdown. It needs no CMS plugin. It follows a fixed structure: an H1 with your site name, a one-line summary, then H2 sections grouping links by topic.

• Audit your highest-value pages first. Service pages, your strongest guides, pricing or comparison pages, anything you'd want a model to find in one read.
• Write a one-line, factual description for each link. Skip the marketing copy. A model is parsing for meaning, not getting persuaded.
• Group pages under H2 headings. Docs, Guides, Pricing, whatever matches your site's actual structure.
• Keep it lean. A few hundred lines is a ceiling, not a target. The file exists to save a model time, and padding it defeats the point.
• Point to it from robots.txt. One line, LLMs: https://yourdomain.com/llms.txt, helps a crawler that checks robots.txt first find the file sooner.
• Check your server logs for hits to /llms.txt. That's the only real way to know whether anything is actually reading it.

Common mistakes that waste the afternoon

Most llms.txt files fail quietly, rather than causing an obvious problem. These are worth checking before you call the project done.

• Listing every page instead of the important ones. A file that mirrors your full sitemap gives a model nothing it couldn't already infer. The curation is the entire value.
• Reusing marketing copy as the description. A model reads these lines to decide whether to fetch a page. A headline built to persuade a human reads as noise to a machine parsing for facts.
• Letting it go stale after a site redesign. If you're already planning a rebuild, run llms.txt through our website redesign checklist as one line item. Don't leave it as an afterthought once the new site ships.
• Treating it as a substitute for structured, answer-ready content. The file points a model toward your pages. It does nothing to make those pages easier to quote once a model arrives.

llms.txt for SEO and your wider AI search visibility plan

Treat llms.txt as a small, low-cost bet inside a bigger plan, not the plan itself. How to rank in ChatGPT search covers the levers that currently move citations. Those include direct answers near the top of a page, consistent facts across the web, and content a model can quote cleanly. llms.txt sits alongside those levers, and it doesn't replace any of them.

The sites winning AI citations right now built their content to be answer-ready years before llms.txt existed. For a deeper look at what that actually takes, read how AI chatbots decide what to cite. It's more useful than another llms.txt tutorial. The file is a convenience layer on top of that groundwork, not a substitute for it.