What is llms.txt and how do you create one?

llms.txt is a proposal, published in 2024, for a Markdown file at the root of a website that tells large language models what the site is about and where its most useful content lives. The idea is that a model or an agent visiting a site can read one short, curated document instead of inferring the site's purpose from navigation and HTML. A companion convention, llms-full.txt, contains the full text of the key documents in one file for systems that want everything at once.

It is explicitly not a crawl-control mechanism. It does not allow or block anything; robots.txt and CDN rules do that.

The proposal specifies Markdown with a fixed shape: an H1 with the site or project name; a blockquote with a one-paragraph summary; optional free-form paragraphs with essential context; then H2 sections, each containing a list of links in the form "[title](url): description". An "Optional" H2 marks links that can be skipped when context is limited. SerionFlow's own llms.txt follows this structure, with sections for what the product does, the workflow, hosting and routing, plans, and an index of public pages including free tools and comparison pages.

robots.txt controls which paths crawlers may request and is honoured by the major AI crawlers. sitemap.xml lists every URL you want indexed, for search engines. llms.txt is a curated, human-written summary of the site for models; it lists a few dozen important pages with descriptions, not every URL, and it carries no rules. A site that wants AI visibility needs all three doing their separate jobs, plus pages whose full content is in the static HTML, which matters more than any of the three files.

Adoption is uneven and mostly undocumented. Some developer-tool and documentation sites report that assistants and agents read llms.txt when present, and several documentation platforms generate it automatically. No major AI search product has committed publicly to prioritising it. The honest position in 2026 is that llms.txt is cheap to publish, cannot hurt if kept accurate, and may help agents and retrieval systems summarise your site correctly, but it is not a lever that moves citations on its own.

Write the H1 and summary by hand; they are the parts that matter most. List the twenty to fifty pages that best explain the site, grouped by purpose, each with a one-line description in plain language. Include product facts a model should not get wrong, such as what the product does not do and any important limits. Save as UTF-8 plain text, serve at /llms.txt with a text/plain or text/markdown content type, and reference it from robots.txt if you like, though that is not required. SerionFlow's free llms.txt generator drafts the file from a site URL so you can edit rather than start blank.

A stale llms.txt teaches models outdated facts, which is worse than letting them read the live pages. Regenerate it whenever pricing, plans, or key pages change, and review it quarterly. Sites that publish pages programmatically should append the new hubs and key pages to the file as part of the release process; SerionFlow lists its public page index in its own llms.txt for this reason.