# llms.txt: what it actually is, what it isn't, and why it might matter later. > Google said llms.txt doesn't matter for search. They also put it in their agentic search documentation. Both things are true — and they are talking about different things entirely. **Published:** Jun 2026 · 5 min read · AI SEO --- ## What is llms.txt llms.txt is a proposed convention — think robots.txt, but for language models. You place a file at `yoursite.com/llms.txt` that contains a simplified, markdown-formatted summary of your site: what it is, what it does, links to key pages, and optionally a stripped-down version of your content that is easy for an LLM to parse without fighting through navigation, ads, cookie banners, and layout HTML. The format was proposed by Jeremy Howard (fast.ai) in 2024. It is not an official standard. No standards body has ratified it. Most major LLMs do not specifically look for it. But a small and growing number of developer tools, documentation systems, and AI agents do. ## What Google actually said Google has been clear: llms.txt does not affect your search rankings. It is not a ranking signal. Googlebot does not treat it as a preferential crawl target. If you publish one, it will not help you rank higher on Google Search. This is accurate and important to say plainly, because a wave of SEO vendors has started selling llms.txt creation as an AI SEO service. It is not. For traditional search — and [as we have established, traditional search is what feeds AI search results](/blog/what-is-ai-seo) — llms.txt is irrelevant. ## What Google also said, in a different document Here is where it gets more nuanced. Google's documentation explicitly references llms.txt in the context of agentic search — their guidance for AI agents that browse and retrieve content on behalf of users. In that context, the file is useful: it gives an agent a structured, parseable overview of a site without requiring it to crawl every page and extract meaning from HTML. So Google's position is not contradictory. It is two separate statements about two separate use cases: - **For search rankings:** llms.txt does not matter. - **For AI agents navigating your site:** llms.txt is useful reference material. The confusion comes from treating these as the same claim. ## Where llms.txt genuinely helps The use case where llms.txt has real value is not SEO at all. It is **documentation, developer tools, and technical reference sites** — places where AI agents and coding assistants are the primary visitors, not human search users. If you maintain an API, a developer SDK, or a technical documentation site, an AI agent trying to help a developer use your product will benefit significantly from a well-structured llms.txt. Instead of crawling hundreds of pages and trying to reconstruct your API's structure, it reads your llms.txt and gets a clean map of what exists and where. This is already how tools like Claude Code, Cursor, and GitHub Copilot access documentation when assisting with code. A developer asks their AI assistant how to authenticate with your API. The assistant fetches your docs. If your docs are structured clearly — and especially if llms.txt points it to the right pages — it gets a useful answer faster. If your docs are a maze of nested HTML with version selectors and pop-up login prompts, it may fail entirely. For a B2B SaaS company with a developer audience, this is worth caring about. For a marketing site, it is largely irrelevant right now. ## This is currently a Google AI ecosystem thing It is worth noting that Google's explicit support for llms.txt is tied to their AI ecosystem — Gemini, AI Overviews, and their agentic products. Other major AI players have not formally adopted the convention. OpenAI has not published guidance on it. Anthropic has not. Perplexity has not. That limits its current reach. If you are optimising for [visibility in ChatGPT or Claude](/blog/why-not-showing-up-on-chatgpt), llms.txt is not the lever to pull. Your search ranking still determines that, regardless of any files in your root directory. ## Why it might matter more later This is my own read, not established fact: I think llms.txt becomes more valuable as AI harnesses become more prevalent. A harness — Claude Code, Cursor, a custom AI agent built on the Anthropic API — can access URLs directly. When a developer uses one of these tools to research, build, or debug, the AI routinely fetches pages to read them. It is not using a search index. It is not Googlebot. It is an agent pulling a URL and reading whatever it finds. As more work gets done through these harnesses, the way content gets consumed changes. An agent that fetches your documentation URL and hits a clean llms.txt gets oriented immediately. One that hits a wall of JavaScript-rendered HTML with no clear entry point gets less useful information and may hallucinate the rest. If adoption grows — if more AI tools start looking for llms.txt as a convention the way browsers look for favicon.ico — then having one becomes an ambient advantage. Not a ranking signal. Not an SEO tactic. Just good practice for a web where agents are increasingly common visitors. We are not there yet. But the direction of travel seems clear enough to make it worth implementing on documentation-heavy sites, and worth watching on everything else.