Technical AI Optimisation Checklist 2026: 12 Fixes So ChatGPT, Gemini and Perplexity Can Read Your Site

Technical AI optimisation makes sure ChatGPT, Gemini, Claude, Perplexity and Copilot can reach your website, read it and understand who you are. It covers three things: access (AI crawlers are allowed in and get a working page), understanding (structured data, clear headings and consistent facts) and discovery (sitemaps, IndexNow and llms.txt so new pages are found quickly). This checklist lists the 12 fixes that matter most, in the order we check them.

Technical AI optimisation vs technical SEO

Technical SEO prepares a site for search engine crawlers that render pages and rank links. AI platforms add new crawlers with different rules: some only collect training data, some fetch pages live when a user asks a question, and most do not run JavaScript. A site can rank well on Google and still be invisible to an AI assistant because a firewall blocks its crawler, the content only appears after JavaScript runs, or the facts about the company contradict each other across pages. Technical AI optimisation closes those gaps on top of good technical SEO.

Access: can AI crawlers reach your pages?

1. Allow the AI crawlers you want in robots.txt

Each AI company uses several crawlers for different jobs. OpenAI uses GPTBot (training), OAI-SearchBot (ChatGPT search) and ChatGPT-User (pages fetched when a user asks). Anthropic uses ClaudeBot, Claude-SearchBot and Claude-User. Perplexity uses PerplexityBot and Perplexity-User. Google-Extended controls whether Gemini may use your content, and Bingbot feeds Microsoft Copilot. Blocking a training crawler and blocking a search crawler have very different effects, so decide each one deliberately rather than with a single rule.

2. Check your CDN and firewall, not just robots.txt

Robots.txt can allow a crawler while your CDN or firewall still blocks it. Bot-protection settings, AI-crawler blocking defaults, rate limits (HTTP 429) and challenges (HTTP 403) all stop AI crawlers before they see the page. Check your CDN's bot or AI-crawler report for blocked requests by crawler.

3. Put the content in the HTML

Most AI crawlers read the HTML they receive and do not run JavaScript. If your text, prices or FAQs are only loaded by a script, an AI crawler sees an empty page. Server-rendered or static HTML is the safest option. Collapsed FAQ sections are fine as long as the text is in the HTML.

4. Return clean responses

Every page you want cited should return HTTP 200 quickly, without redirect chains, soft 404s or login walls. Variants such as a trailing slash or www should redirect once to the main address.

Understanding: can AI read who you are and what you offer?

5. Add structured data

Schema.org markup in JSON-LD tells AI systems what a page is: Organization for your company, Article for guides, FAQPage for questions and answers, Product, Service or LocalBusiness for what you sell. It must be valid and must match what the page visibly says.

6. Use one H1 and a clear heading structure

One H1 per page, then H2 and H3 headings that read like an outline. Make sure table-of-contents links actually point to the headings; broken in-page links are common after a redesign.

7. Write accurate titles, descriptions and canonical tags

The title and meta description are often the first summary an AI system sees. The canonical tag tells it which address is the main version, so duplicate pages do not split your citations.

8. Keep entity facts consistent

Your company name, what you do, where you are, when you were founded and how to contact you should be identical across your pages, structured data, llms.txt and profiles elsewhere. Contradictions make AI systems less confident about you. Make sure your contact email is visible in the HTML and not hidden by email-obfuscation scripts that crawlers cannot run.

Discovery: are new and updated pages found quickly?

9. List your XML sitemaps in robots.txt

Keep sitemaps current, include only pages that return 200, and give each URL an honest last-modified date.

10. Use IndexNow

IndexNow lets you notify Bing and other participating search engines the moment a page changes. Bing's index feeds Microsoft Copilot, so this is one of the fastest ways to get updates picked up. Google does not use IndexNow; use Search Console for Google.

11. Add an llms.txt file

llms.txt is a proposed standard: a plain-text summary of your site and its most important pages for AI tools. It is cheap to add and keeps your key facts in one place, but it is not a ranking guarantee, and not every AI platform reads it yet.

12. Give every page a working share image and clean links

Missing Open Graph images and broken internal links create errors and poor previews wherever your pages are shared or summarised. Check that every page references an image that exists.

How to check all 12 in 10 seconds

CiteTrust™ is our AI website checker. Enter a URL and it tests AI crawler access, schema markup, meta tags, Open Graph data and llms.txt, and gives the page a score — free, with no sign-up. The Pro report (£19 per URL) runs the full 40-point check with prioritised fixes, and the Fix Pack (£49) gives you the implementation fixes for the issues it finds.

Run the free technical AI optimisation check →

What we found on our own site

When we rebuilt ultrascout.ai in October 2026 we ran the same checks on ourselves. Every AI crawler we tested could reach the site, but we still found 173 table-of-contents links that pointed to headings without an anchor, 487 references to share images that did not exist, our contact email hidden from crawlers by our CDN's email obfuscation, and page addresses that returned 404 when a trailing slash was added. None of these were visible to us as visitors. All of them were fixed within days of being found — which is the point of checking.

Frequently asked questions