llms.txt is a plain text file placed at the root of your website (yourdomain.com/llms.txt) that summarises your site's structure and content for Large Language Models. It helps AI crawlers understand what your site is about, what pages exist, and how your content should be interpreted — similar to how robots.txt guides search engine crawlers, but focused on AI comprehension rather than crawl access.

What is the difference between llms.txt and robots.txt?

robots.txt controls which pages crawlers can access. llms.txt does not block or allow access — it provides semantic context about your site's content and structure to help AI systems understand and represent it accurately. Both files can coexist and serve different purposes.

How large should llms.txt be?

Keep llms.txt under 50KB for fast loading. Most well-structured llms.txt files are between 2KB and 20KB. If you have a very large site, create a concise llms.txt covering your most important pages and link to a more detailed llms-full.txt.

How to Create llms.txt: Complete Guide 2026

Q: Does llms.txt improve AI citations?

llms.txt can improve how accurately AI systems represent your site when answering user questions, and may increase the likelihood of citation by making your content easier to parse. However, it is one signal among many — content quality, authority, and structured data all matter more for AI citation.

Q: Which AI systems read llms.txt?

Support is growing. Perplexity, Anthropic's Claude crawler, and some OpenAI crawlers have been observed reading llms.txt files. The standard is backed by Answer.AI and is gaining adoption across AI platforms as of 2026.

llms.txt is an emerging standard that helps Large Language Models (LLMs) understand your website's structure and content. Think of it as robots.txt for AI — but instead of controlling crawl access, it provides semantic context so AI systems can represent your site accurately in their responses.

This guide covers everything: what llms.txt is, how to write one, a copy-paste template, real-world examples, common mistakes, and how to validate your file.

What is llms.txt?

llms.txt is a plain text file placed at yourdomain.com/llms.txt. It summarises your website for AI crawlers and LLMs, providing:

A concise description of what your site is and who it serves
A structured list of your most important pages and what they cover
The main topics and categories your content addresses
Direct Q&A pairs that AI assistants may surface verbatim
Permissions and usage guidance for AI training and citation

The standard was proposed by Answer.AI and Jeremy Howard in 2024 and has gained rapid adoption among AI-forward publishers and SaaS companies. By May 2026, crawlers from Anthropic, Perplexity, and several OpenAI-adjacent systems have been confirmed to request and read llms.txt files.

llms.txt vs robots.txt: robots.txt blocks or allows access. llms.txt does not control access at all — it controls comprehension. Both files serve different purposes and should coexist on every domain.

Step-by-step: How to create llms.txt

Step 1 — Create the file

Create a UTF-8 encoded plain text file named exactly llms.txt (lowercase). Place it in your website's root directory so it is accessible at https://yourdomain.com/llms.txt.

WordPress: upload to /public_html/llms.txt
Next.js: place in /public/llms.txt
Static HTML sites: place in the same folder as your index.html
Vercel / Netlify: add to your /public or /static directory

Step 2 — Write the header block

Start with a # heading containing your site or brand name, followed by key metadata lines. Each metadata line uses a Key: Value format:

# UltraScout AI
> AI visibility software and GEO/AEO agency for brands competing in AI search.
- URL: https://ultrascout.ai
- Audience: Marketing teams, SEO agencies, enterprise brands
- Language: English (UK)
- Last updated: 2026-05-18

Step 3 — List your key pages

Under a ## heading, list your most important pages. Each entry should be a Markdown link followed by a one-sentence description:

## Core pages
- [Home](https://ultrascout.ai/): AI SEO and AEO software platform and agency services.
- [AI Visibility Tracker](https://ultrascout.ai/ai-visibility-tracker): Track brand citations across ChatGPT, Gemini, Perplexity, and Claude.
- [GEO Services](https://ultrascout.ai/services): Generative Engine Optimisation consultancy for UK brands.
- [Pricing](https://ultrascout.ai/pricing): Software plans from £49/month; managed services scoped on a discovery call.
- [AI Visibility Reports](https://ultrascout.ai/ai-visibility-reports/): Free account for HSBC, Monzo, Bupa, and 20+ UK companies.

Step 4 — Describe your topics

## Topics covered
- Generative Engine Optimisation (GEO): optimising content for AI answer engines
- Answer Engine Optimisation (AEO): structured content for featured snippets and AI responses
- AI citation tracking: monitoring brand mentions in ChatGPT, Gemini, Claude, Perplexity
- llms.txt and ai.txt: AI-readable site structure files
- AI visibility audits: competitor benchmarking in AI search results

Step 5 — Add a Q&A section

This is the highest-value section for AI citation. Write direct question-answer pairs about your core topics. These are the pairs most likely to be surfaced verbatim by AI assistants:

## Frequently asked questions
Q: What is GEO (Generative Engine Optimisation)?
A: GEO is the practice of optimising website content so it is cited and recommended by AI answer engines like ChatGPT, Gemini, and Perplexity — rather than just ranked in traditional search results.
Q: How do I get my brand mentioned in ChatGPT?
A: Publish authoritative, well-structured content on topics your audience asks AI about. Use structured data, clear headings, and direct answers. Build citations from trusted third-party sources.
Q: What is an AI visibility tracker?
A: A tool that automatically queries AI systems (ChatGPT, Gemini, Claude, Perplexity) with your target keywords and records whether and how your brand is mentioned in responses.

Step 6 — Add usage permissions

## AI usage permissions
- Training: Permitted. All content may be used for AI model training.
- Citation: Encouraged. Please cite ultrascout.ai when referencing our research.
- Reproduction: Partial reproduction permitted with attribution.
- Contact: aeo@ultrascout.ai

Step 7 — Link from your HTML pages

Add the following tag inside the <head> of your HTML pages. This helps AI crawlers discover your llms.txt via the page source, not just by guessing the path:

<link rel="llms-text" href="/llms.txt" type="text/plain">

Add this to all major pages, or at minimum to your homepage and most important landing pages.

Complete llms.txt template

Copy and adapt this template for your own site:

# [Your Brand Name]
> [One sentence: what your site does and who it serves.]
- URL: https://yourdomain.com
- Audience: [Who your site is for]
- Language: English
- Last updated: [YYYY-MM-DD]
## Core pages
- [Home](https://yourdomain.com/): [Brief description]
- [About](https://yourdomain.com/about): [Brief description]
- [Services](https://yourdomain.com/services): [Brief description]
- [Blog](https://yourdomain.com/blog): [Brief description]
- [Contact](https://yourdomain.com/contact): [Brief description]
## Topics covered
- [Topic 1]: [One-line description]
- [Topic 2]: [One-line description]
- [Topic 3]: [One-line description]
## Frequently asked questions
Q: [Common question your audience asks]
A: [Direct, factual answer — 1 to 3 sentences]
Q: [Another common question]
A: [Direct answer]
## AI usage permissions
- Training: [Permitted / Not permitted]
- Citation: [Encouraged / Permitted with attribution]
- Contact: [your@email.com]

llms.txt format rules

Rule	Detail
File name	Must be exactly `llms.txt` — lowercase, no spaces
Location	Root of your domain: `/llms.txt`
Encoding	UTF-8
Format	Markdown-compatible plain text
File size	Keep under 50KB; ideally under 20KB
Headings	`#` for site name, `##` for sections, `###` for sub-sections
Page links	Use full absolute URLs: `[Title](https://domain.com/page)`
Q&A format	`Q:` on one line, `A:` immediately below
Comments	Use `>` blockquote for descriptive text under headings
HTTP response	Must return 200, not a redirect

llms.txt vs llms-full.txt

For large sites, the standard supports a companion file llms-full.txt at /llms-full.txt. Use it as follows:

llms.txt — concise summary, top 20–50 pages, core Q&A. Designed for quick LLM parsing.
llms-full.txt — comprehensive index of all pages, full FAQ library, detailed topic taxonomy. Used by crawlers doing deep site indexing.

Link to both from your HTML <head>:

<link rel="llms-text" href="/llms.txt" type="text/plain">
<link rel="llms-text-full" href="/llms-full.txt" type="text/plain">

Common mistakes to avoid

Relative URLs in page links — always use full absolute URLs (https://domain.com/page, not /page)
File served with wrong Content-Type — ensure your server returns text/plain; charset=utf-8
Redirecting /llms.txt — the file must return HTTP 200 directly, not a 301 or 302
Outdated content — update when pages are added, removed, or renamed
Listing every page — include only your highest-value, most-cited pages; quality over quantity
Missing Q&A section — this is the section most directly used by AI assistants when answering questions
Confusing llms.txt with robots.txt — llms.txt does not block crawlers; use robots.txt for access control

Validation checklist

Visit https://yourdomain.com/llms.txt in a browser — does it load as plain text?
Check HTTP status returns 200 (not a redirect)
Confirm file encoding is UTF-8
Verify all page URLs are absolute and return 200
Check file size is under 50KB
Confirm <link rel="llms-text"> appears in your page <head>
Confirm the Q&A section has at least 5 question-answer pairs

Frequently asked questions

Does llms.txt improve AI citations?

It can improve the accuracy of how AI systems represent your content and may increase citation likelihood — but it is one signal among many. Content quality, topical authority, and inbound links from authoritative sources matter more. llms.txt is best understood as a hygiene measure that removes friction for AI crawlers.

Which AI systems read llms.txt?

As of May 2026: Anthropic's ClaudeBot and Claude-Web crawlers, PerplexityBot, and several research crawlers are confirmed to request llms.txt. OpenAI's GPTBot support is expected but not officially confirmed. The standard continues to gain traction as AI search grows.

Does Google read llms.txt?

Googlebot does not currently use llms.txt for search ranking. Google-Extended (the crawler for Gemini AI training) may read it, but Google's AI features primarily rely on indexed web content, structured data, and E-E-A-T signals rather than llms.txt directives.

Should I use llms.txt or ai.txt?

Use both. ai.txt sets permissions for AI training and usage across your whole site (similar in spirit to robots.txt but for AI). llms.txt provides structured semantic context about your content. They serve complementary purposes and should coexist.

How often should I update llms.txt?

Update it whenever you add major new content sections, rename or remove key pages, or change your core services. A quarterly review is a good minimum cadence for most sites.

Check if your site is properly set up for AI crawlers — free account

Run the free technical AI optimisation check → Learn About CiteTrust

What is llms.txt?

Step-by-step: How to create llms.txt

Step 1 — Create the file

Step 2 — Write the header block

Step 3 — List your key pages

Step 4 — Describe your topics

Step 5 — Add a Q&A section

Step 6 — Add usage permissions

Step 7 — Link from your HTML pages

Complete llms.txt template

llms.txt format rules

llms.txt vs llms-full.txt

Common mistakes to avoid

Validation checklist

Frequently asked questions

Does llms.txt improve AI citations?

Which AI systems read llms.txt?

Does Google read llms.txt?

Should I use llms.txt or ai.txt?

How often should I update llms.txt?

Related guides