llms.txt and GEO: making your website readable for AI

llms.txt is a small file that helps AI tools understand your website and find their way around it. It has been around since 2024 and still isn't an official standard. Yet few people know about it. Sobra.be has one.

What llms.txt is

Jeremy Howard of Answer.AI proposed it on 3 September 2024. It's a Markdown file at /llms.txt that tells a language model in a few lines what your site is about and which pages matter. Only a title is required. Usually a short summary and a few lists of links follow.

The proposal also asks you to offer a Markdown version of important pages, at the same URL with .md added. That way a language model doesn't have to dig through menus, scripts and cookie banners to reach the text.

What GEO is

GEO stands for generative engine optimization: making sure your content shows up in the answers of AI search engines, such as ChatGPT, Perplexity or Google's AI Overviews. The term comes from a paper from November 2023, presented at KDD 2024.

Google itself calls it still SEO. According to Google, AI Overviews and AI Mode follow the same basics as the rest of Search. So to me, llms.txt is a small extra on top of those basics.

Who reads it

Not Google Search. Google writes that you don't need separate AI files, and that an llms.txt won't help or hurt your visibility in Search. Google's John Mueller compared it to the keywords meta tag in April 2025.

At the same time, Lighthouse has had an llms.txt check since version 13.3, under agentic browsing. So Google doesn't quite agree with itself yet. Anthropic publishes one for its documentation, but I found no official statement that its models read it on other sites. Nor from OpenAI.

What to watch out for

Don't expect extra visitors from it. Think of it as a business card for AI agents and developer tools that do fetch it.

Keep it short and factual: what you do, for whom, and which pages explain the rest. And it has to be right. An llms.txt that lags behind tells AI tools things about your business that are no longer true.

Have it generated

That's why I'd never maintain it by hand. On sobra.be every build regenerates it, from the same copy as the site, together with the sitemap. A new blog post is in it straight away.

You don't need to write your own build for that. According to llmstxt.org, Yoast SEO, AIOSEO and Wix among others create it automatically, and there are plugins for VitePress and Docusaurus. As long as it keeps itself up to date, it costs you nothing.