How to Create llms.txt — The Complete Guide to AI Agent Context Files
Your website has robots.txt for search crawlers and sitemap.xml for indexing. Now there's llms.txt — a simple text file that tells AI agents what your site actually does. Here's how to create one that gets you recommended.
What is llms.txt?
llms.txt is an emerging standard for providing AI language models with structured context about your website. Think of it as robots.txt for AI comprehension — while robots.txt tells crawlers where to go, llms.txt tells AI agents what you do.
It's a plain text file placed at your website's root (yourdomain.com/llms.txt) that uses Markdown formatting. When AI agents like GPTBot, ClaudeBot, or PerplexityBot crawl your site, they check for this file to build a richer understanding of your product or service.
The concept was proposed by Jeremy Howard (fast.ai founder) and is gaining traction rapidly as AI-powered search replaces traditional search for millions of users.
Why you need llms.txt now
When someone asks ChatGPT "what's the best project management tool?", the AI doesn't search Google — it synthesizes information from everything it knows. Sites with clear, structured context are more likely to be recommended. Sites without it? Invisible.
Three reasons to create yours today:
- First-mover advantage: Most websites don't have llms.txt yet. Creating one now puts you ahead of 99% of competitors.
- Zero cost, high ROI: It takes 15 minutes to create. The potential upside — being recommended by AI agents to millions of users — is massive.
- Future-proof: As AI search grows (40% of Gen Z already uses AI instead of Google), llms.txt will become as essential as robots.txt is today.
Anatomy of a great llms.txt
A strong llms.txt has five sections:
1. Product identity (required)
Start with your name and a crisp 2-3 sentence description. This is the most important part — AI agents use it as the primary summary.
# Acme Analytics Real-time web analytics that respects user privacy. No cookies, no tracking scripts, no personal data collection. Built for businesses that need insights without compromising user trust.
2. Key features (required)
List your 4-6 most important features. Be specific about what makes you different, not just what you do.
## Features - Real-time dashboard with <1 second data refresh - GDPR/CCPA compliant by design (no cookies, no PII) - Lightweight script (under 1KB, no performance impact) - Custom events and goal tracking - API access for data export - Self-hosted option available
3. Target audience (recommended)
Help AI agents match your product to the right queries.
## Who it's for - Privacy-conscious businesses replacing Google Analytics - SaaS companies needing lightweight, GDPR-compliant analytics - Developers who want a simple, script-tag-and-done solution - EU-based businesses requiring data sovereignty
4. Key URLs (recommended)
Direct AI agents to your most important pages.
## Key URLs - Homepage: https://acme-analytics.com - Documentation: https://docs.acme-analytics.com - Pricing: https://acme-analytics.com/pricing - API Reference: https://docs.acme-analytics.com/api - Blog: https://acme-analytics.com/blog
5. Differentiators (optional but powerful)
What makes you different from the obvious competitors? This is what makes AI agents recommend you specifically.
## Why Acme over alternatives - Unlike Google Analytics: no cookies, no PII, no consent banners needed - Unlike Plausible: self-hosted option with full data ownership - Unlike Matomo: 10x faster setup, no server management required - One-time purchase available (no recurring subscription)
Copy-paste template
Copy this template, fill in your details, and save it as llms.txt in your site root:
# [Your Product Name] [2-3 sentences: what it does, who it's for, what makes it unique] ## Features - [Feature 1 — be specific] - [Feature 2 — mention what's different] - [Feature 3] - [Feature 4] - [Feature 5] ## Who it's for - [Target audience 1] - [Target audience 2] - [Target audience 3] ## Key URLs - Homepage: [URL] - Documentation: [URL] - Pricing: [URL] - API: [URL] - Contact: [URL] ## How it works [1-2 sentences explaining the core workflow or user experience] ## Pricing [Brief pricing info — e.g., "Free tier available. Pro starts at $X/month."]
Real-world examples
Here's what our own llms.txt looks like (you can see it at aeo-checker-0tao.onrender.com/llms.txt):
# AEO Checker — AI Engine Optimization Score Tool
AEO Checker is a free tool that analyzes any website's
visibility to AI agents (ChatGPT, Claude, Perplexity,
Google AI). It provides an AEO Score out of 100 with
actionable recommendations.
## How the AEO Score works
The checker runs 6 automated checks:
1. Structured Data (0-20pts)
2. robots.txt AI Bots (0-15pts)
3. llms.txt (0-15pts)
4. Content Structure (0-20pts)
5. Tool/API Description (0-15pts)
6. Performance (0-15pts)
## Key URLs
- AEO Checker tool: https://aeo-checker-0tao.onrender.com/
- Pricing: https://aeo-checker-0tao.onrender.com/pricing.html
- Blog: https://aeo-checker-0tao.onrender.com/blog-what-is-aeo.html
## API
POST /api/check with JSON body { "url": "..." }
GET /api/badge/:score for embeddable SVG badge
Other notable sites with llms.txt:
- Anthropic —
anthropic.com/llms.txt(they don't have one yet — opportunity!) - Stripe — Early adopter of developer-facing context files
- llmstxt.org — The original proposal and directory
Common mistakes
Keep it simple, structured, and honest. AI agents are good at detecting marketing BS — just describe what you actually do.
Deploy and verify
Step 1: Create the file
Save your content as llms.txt (plain text, UTF-8 encoding).
Step 2: Upload to your site root
Place it alongside your robots.txt and sitemap.xml. The file should be accessible at:
https://yourdomain.com/llms.txt
Step 3: Verify it's accessible
Open yourdomain.com/llms.txt in your browser. You should see the raw text content. If you get a 404, check your web server configuration.
Step 4: Test with AEO Checker
Run your site through our free AEO Checker. The llms.txt check should now show ✅ with a score of 10-15/15 depending on content length.
• WordPress: Upload llms.txt to your
public_html or www directory
• Netlify/Vercel: Place in your
public/ or static/ folder
• Next.js: Place in the
public/ directory
• Shopify: Upload via theme editor under Assets, then create a redirect from
/llms.txt
Frequently Asked Questions
What is llms.txt?
A plain text file at your website root that provides structured context for AI language models. It helps ChatGPT, Claude, Perplexity, and other AI agents understand what your site does and how to recommend it.
Where do I put llms.txt?
In the root directory of your website, at the same level as robots.txt and sitemap.xml. It should be accessible at yourdomain.com/llms.txt via HTTP.
What format should llms.txt use?
Plain text with Markdown formatting. Start with a heading (# Your Product), followed by feature lists, audience, and key URLs. Aim for 500-2000 characters.
Do AI agents actually read llms.txt?
The standard is emerging and growing. AI crawlers like GPTBot, ClaudeBot, and PerplexityBot are increasingly checking for it. Having one positions you ahead of competitors as the standard matures — and it costs you nothing.
How long should llms.txt be?
500-2000 characters is the sweet spot. Under 100 is too sparse to be useful. Over 5000 risks being truncated. Be concise — every line should earn its place.
Related reading: What is AEO? · AEO Optimization Guide · AEO vs SEO · AEO Audit Services