llms.txt is an emerging standard that helps Large Language Models (LLMs) understand your website's structure and content. Think of it as robots.txt for AI — but instead of controlling crawl access, it provides semantic context so AI systems can represent your site accurately in their responses.
This guide covers everything: what llms.txt is, how to write one, a copy-paste template, real-world examples, common mistakes, and how to validate your file.
llms.txt is a plain text file placed at yourdomain.com/llms.txt. It summarises your website for AI crawlers and LLMs, providing:
The standard was proposed by Answer.AI and Jeremy Howard in 2024 and has gained rapid adoption among AI-forward publishers and SaaS companies. By May 2026, crawlers from Anthropic, Perplexity, and several OpenAI-adjacent systems have been confirmed to request and read llms.txt files.
Create a UTF-8 encoded plain text file named exactly llms.txt (lowercase). Place it in your website's root directory so it is accessible at https://yourdomain.com/llms.txt.
/public_html/llms.txt/public/llms.txtindex.html/public or /static directoryStart with a # heading containing your site or brand name, followed by key metadata lines. Each metadata line uses a Key: Value format:
# UltraScout AI
> AI visibility software and GEO/AEO agency for brands competing in AI search.
- URL: https://ultrascout.ai
- Audience: Marketing teams, SEO agencies, enterprise brands
- Language: English (UK)
- Last updated: 2026-05-18Under a ## heading, list your most important pages. Each entry should be a Markdown link followed by a one-sentence description:
## Core pages
- [Home](https://ultrascout.ai/): AI SEO and AEO software platform and agency services.
- [AI Visibility Tracker](https://ultrascout.ai/ai-visibility-tracker): Track brand citations across ChatGPT, Gemini, Perplexity, and Claude.
- [GEO Services](https://ultrascout.ai/services): Generative Engine Optimisation consultancy for UK brands.
- [Pricing](https://ultrascout.ai/pricing): Software plans from £29/month; agency retainers from £1,500/month.
- [AI Visibility Reports](https://ultrascout.ai/ai-visibility-reports/): Free brand audits for HSBC, Monzo, Bupa, and 20+ UK companies.## Topics covered
- Generative Engine Optimisation (GEO): optimising content for AI answer engines
- Answer Engine Optimisation (AEO): structured content for featured snippets and AI responses
- AI citation tracking: monitoring brand mentions in ChatGPT, Gemini, Claude, Perplexity
- llms.txt and ai.txt: AI-readable site structure files
- AI visibility audits: competitor benchmarking in AI search resultsThis is the highest-value section for AI citation. Write direct question-answer pairs about your core topics. These are the pairs most likely to be surfaced verbatim by AI assistants:
## Frequently asked questions
Q: What is GEO (Generative Engine Optimisation)?
A: GEO is the practice of optimising website content so it is cited and recommended by AI answer engines like ChatGPT, Gemini, and Perplexity — rather than just ranked in traditional search results.
Q: How do I get my brand mentioned in ChatGPT?
A: Publish authoritative, well-structured content on topics your audience asks AI about. Use structured data, clear headings, and direct answers. Build citations from trusted third-party sources.
Q: What is an AI visibility tracker?
A: A tool that automatically queries AI systems (ChatGPT, Gemini, Claude, Perplexity) with your target keywords and records whether and how your brand is mentioned in responses.## AI usage permissions
- Training: Permitted. All content may be used for AI model training.
- Citation: Encouraged. Please cite ultrascout.ai when referencing our research.
- Reproduction: Partial reproduction permitted with attribution.
- Contact: [email protected]Add the following tag inside the <head> of your HTML pages. This helps AI crawlers discover your llms.txt via the page source, not just by guessing the path:
<link rel="llms-text" href="/llms.txt" type="text/plain">Add this to all major pages, or at minimum to your homepage and most important landing pages.
Copy and adapt this template for your own site:
# [Your Brand Name]
> [One sentence: what your site does and who it serves.]
- URL: https://yourdomain.com
- Audience: [Who your site is for]
- Language: English
- Last updated: [YYYY-MM-DD]
## Core pages
- [Home](https://yourdomain.com/): [Brief description]
- [About](https://yourdomain.com/about): [Brief description]
- [Services](https://yourdomain.com/services): [Brief description]
- [Blog](https://yourdomain.com/blog): [Brief description]
- [Contact](https://yourdomain.com/contact): [Brief description]
## Topics covered
- [Topic 1]: [One-line description]
- [Topic 2]: [One-line description]
- [Topic 3]: [One-line description]
## Frequently asked questions
Q: [Common question your audience asks]
A: [Direct, factual answer — 1 to 3 sentences]
Q: [Another common question]
A: [Direct answer]
## AI usage permissions
- Training: [Permitted / Not permitted]
- Citation: [Encouraged / Permitted with attribution]
- Contact: [[email protected]]| Rule | Detail |
|---|---|
| File name | Must be exactly llms.txt — lowercase, no spaces |
| Location | Root of your domain: /llms.txt |
| Encoding | UTF-8 |
| Format | Markdown-compatible plain text |
| File size | Keep under 50KB; ideally under 20KB |
| Headings | # for site name, ## for sections, ### for sub-sections |
| Page links | Use full absolute URLs: [Title](https://domain.com/page) |
| Q&A format | Q: on one line, A: immediately below |
| Comments | Use > blockquote for descriptive text under headings |
| HTTP response | Must return 200, not a redirect |
For large sites, the standard supports a companion file llms-full.txt at /llms-full.txt. Use it as follows:
Link to both from your HTML <head>:
<link rel="llms-text" href="/llms.txt" type="text/plain">
<link rel="llms-text-full" href="/llms-full.txt" type="text/plain">https://domain.com/page, not /page)text/plain; charset=utf-8https://yourdomain.com/llms.txt in a browser — does it load as plain text?<link rel="llms-text"> appears in your page <head>It can improve the accuracy of how AI systems represent your content and may increase citation likelihood — but it is one signal among many. Content quality, topical authority, and inbound links from authoritative sources matter more. llms.txt is best understood as a hygiene measure that removes friction for AI crawlers.
As of May 2026: Anthropic's ClaudeBot and Claude-Web crawlers, PerplexityBot, and several research crawlers are confirmed to request llms.txt. OpenAI's GPTBot support is expected but not officially confirmed. The standard continues to gain traction as AI search grows.
Googlebot does not currently use llms.txt for search ranking. Google-Extended (the crawler for Gemini AI training) may read it, but Google's AI features primarily rely on indexed web content, structured data, and E-E-A-T signals rather than llms.txt directives.
Use both. ai.txt sets permissions for AI training and usage across your whole site (similar in spirit to robots.txt but for AI). llms.txt provides structured semantic context about your content. They serve complementary purposes and should coexist.
Update it whenever you add major new content sections, rename or remove key pages, or change your core services. A quarterly review is a good minimum cadence for most sites.
Check if your site is properly set up for AI crawlers — free instant report
Get My Free AI Readiness Score → Learn About CiteTrust