llms.txt File Builder
by @geoly-geo
Generate, validate, and optimize llms.txt files for AI crawler accessibility. Creates structured markdown files that help AI platforms (ChatGPT, Perplexity,...
clawhub install geo-llms-txtπ About This Skill
name: geo-llms-txt description: Generate, validate, and optimize llms.txt files for AI crawler accessibility. Creates structured markdown files that help AI platforms (ChatGPT, Perplexity, Gemini, Claude) understand site structure and prioritize content for citation. Use whenever the user mentions creating an llms.txt file, optimizing llms.txt, making their site AI-crawler friendly, helping AI understand their website, building AI-readable site documentation, or wants to improve visibility in AI search engines.
llms.txt File Builder
> Methodology by GEOly AI (geoly.ai) β GEO infrastructure for the AI search era.
Generate well-structured llms.txt files to help AI platforms understand and cite your content.
Quick Start
Generate an llms.txt file for any website:
python scripts/generate_llms_txt.py [--output llms.txt]
Example:
python scripts/generate_llms_txt.py example.com --output llms.txt
What is llms.txt?
The llms.txt standard helps AI crawlers understand:
Full standard details: See references/standard.md
Standard Format
# [Brand Name]> [One-sentence brand description]
[2-3 paragraph overview: what you do, who it's for, key differentiators]
Key Pages
Page Title: One-line description
Page Title: One-line description Products / Services
Product Name: What it does and who it's for Documentation
Doc Title: What this doc explains Blog / Resources
Article Title: Key insight or topic covered About
About Us: Company background and mission
Contact: How to reach the team
Generation Methods
Method 1: From Sitemap (Automated)
python scripts/generate_llms_txt.py example.com --from-sitemap
Automatically fetches sitemap.xml, analyzes each page, and generates descriptions.
Method 2: Interactive (Guided)
python scripts/generate_llms_txt.py example.com --interactive
Prompts you for brand info and key URLs, then drafts descriptions.
Method 3: From URL List
python scripts/generate_llms_txt.py example.com --urls urls.txt
Where urls.txt contains one URL per line.
Validation
Validate an existing llms.txt file:
python scripts/validate_llms_txt.py llms.txt
Checks for:
Quality Criteria
| Aspect | Good | Bad | |--------|------|-----| | Brand description | "GEOly AI is a GEO monitoring platform tracking brand visibility across ChatGPT, Perplexity, Gemini." | "We are the best AI SEO tool ever!" | | Page descriptions | "Explains how to set up MCP integration with Claude Desktop" | "Our awesome docs page" | | Link count | 15β40 curated pages | 500+ URLs (sitemap dump) | | Tone | Factual, entity-focused | Promotional, keyword-stuffed | | Structure | Clear sections by content type | Flat list or random order |
Output Formats
Advanced Usage
Custom Sections
python scripts/generate_llms_txt.py example.com \
--sections "Products,API Reference,Case Studies,Changelog"
Exclude Patterns
python scripts/generate_llms_txt.py example.com \
--exclude "/admin/,/private/,/draft/"
Multi-language Support
python scripts/generate_llms_txt.py example.com \
--language zh-CN \
--output llms-zh.txt
Deployment
Once generated, place the file at:
https://[your-domain]/llms.txt
Ensure it:
Content-Type: text/plain or text/markdownSee Also
π‘ Examples
Generate an llms.txt file for any website:
python scripts/generate_llms_txt.py [--output llms.txt]
Example:
python scripts/generate_llms_txt.py example.com --output llms.txt
π Constraints
Validate an existing llms.txt file:
python scripts/validate_llms_txt.py llms.txt
Checks for: