llms.txt: What It Is and Whether Your Website Needs One

The llms.txt proposal describes a markdown file placed at /llms.txt. It starts with the site's name and a short summary, then lists key pages with a one-line description of each, grouped under headings. The idea is that AI systems, which work within limited context windows, can read a concise, curated overview instead of crawling and parsing every HTML page.

robots.txt tells crawlers what they may and may not access. A sitemap lists all the URLs you want indexed. llms.txt does neither: it's a human-written guide to what matters on your site and why, in plain language. It doesn't grant or block access, and it doesn't replace a sitemap.

llms.txt is a community proposal, not an adopted standard, and major AI providers haven't committed to using it for search answers. Adding one won't guarantee citations. Google, for example, says no special files or markup are needed for its AI features — the usual SEO fundamentals apply.

So treat llms.txt as a small, low-cost addition: it takes an hour, it may help some tools, and it forces you to write a clear summary of your site — which is useful in itself.

A simple file for a small services business might look like this, written in markdown:

On a static site, add a plain text file named llms.txt to the public root. On frameworks like Next.js, a file in the public folder is served at the root automatically. Keep it under version control with the rest of the site so updates are deliberate and reviewable.

If you want AI search tools to read and cite your site, check robots.txt allows their search crawlers — an llms.txt file is pointless if the crawlers that would read it are blocked. OpenAI and Google both document their crawler names and what each one is used for.

Some sites also publish a companion file containing the full text of their key documentation in one markdown document, so a model can read everything in a single request. This is most useful for documentation-heavy products and developer tools. For a typical business website, a concise llms.txt with good links is usually enough.

The opening summary is the part most likely to be read and reused, so write it as you'd want your business described: who you are, what you do, for whom, and where. Use plain, factual language rather than slogans. Avoid claims you can't support — for example, 'the best agency in India' — because they add nothing and can undermine trust if repeated.

If different parts of your business serve different audiences, a sentence on each helps a model route questions to the right section of your site.

Open yourdomain.com/llms.txt in a browser and confirm it loads as plain text. Click every link. Then paste the file into an AI assistant and ask it to describe your business from it — if the description is wrong or vague, the file needs clearer wording.

Documentation sites, software products, knowledge bases and service businesses with many detailed guides benefit most, because they have a lot of content a model could use and a clear structure worth summarising. A small brochure site can add one too, but its effort is better spent first on clear service pages and FAQs.

On sites built from a CMS or database, llms.txt can be generated from the same data as the pages — services, key guides, contact details — so it never drifts out of date. For a static file, add a reminder to your content update routine.

If your site has several languages, list the main pages in each language with clear labels, or link to language-specific sections. Keep the summary in the language most of your audience uses, and make sure every link points to a live, indexable page.

This website publishes an llms.txt at its root, listing its main pages, services, locations and guides, plus clear notes on who owns which project. Writing it was also a useful exercise in stating facts precisely — which is exactly what any AI system describing a business needs.

Whatever you put in llms.txt may be repeated back to people by an AI system. Only include facts you'd stand behind, update it when services or pages change, and make sure it matches your website. An outdated llms.txt describing services you no longer offer does more harm than none at all.