Agent Docs
by @tylervovan
Create documentation optimized for AI agent consumption. Use when writing SKILL.md files, README files, API docs, or any documentation that will be read by LLMs in context windows. Helps structure content for RAG retrieval, token efficiency, and the Hybrid Context Hierarchy.
clawhub install agent-docsπ About This Skill
name: agent-docs description: Create documentation optimized for AI agent consumption. Use when writing SKILL.md files, README files, API docs, or any documentation that will be read by LLMs in context windows. Helps structure content for RAG retrieval, token efficiency, and the Hybrid Context Hierarchy.
Agent Docs
Write documentation that AI agents can efficiently consume. Based on Vercel benchmarks and industry standards (AGENTS.md, llms.txt, CLAUDE.md).
The Hybrid Context Hierarchy
Three-layer architecture for optimal agent performance:
Layer 1: Constitution (Inline)
Always in context. 2,000β4,000 tokens max.# AGENTS.md
> Context: Next.js 16 | Tailwind | Supabaseπ¨ CRITICAL
NO SECRETS in output
Use app/ directory ONLY π DOCS INDEX (use read_file)
Auth: docs/auth/llms.txt
DB: docs/db/schema.md
Include:
Layer 2: Reference Library (Local Retrieval)
Fetched on demand. 1Kβ5K token chunks.Layer 3: Research Assistant (External)
Gated by allow-lists. Edge cases only.Why This Works
Vercel Benchmark (2026): | Approach | Pass Rate | |----------|-----------| | Tool-based retrieval | 53% | | Retrieval + prompting | 79% | | Inline AGENTS.md | 100% |
Root cause: Meta-cognitive failure. Agents don't know what they don't knowβthey assume training data is sufficient. Inline docs bypass this entirely.
Core Principles
1. Compressed Index > Full Docs
An 8KB compressed index outperforms a 40KB full dump.
Compress to:
2. Structure for Chunking
RAG systems split at headers. Each section must be self-contained:
## Database Setup β Chunk boundaryPrerequisites: PostgreSQL 14+
1. Create database...
Rules:
3. Inline Over Links
Agents can't autonomously browse. Each link = tool call + latency + potential failure.
| Approach | Token Load | Agent Success | |----------|------------|---------------| | Full inline | ~12K | β High | | Links only | ~2K | β Requires fetching | | Hybrid | ~4K base | β Best of both |
4. The "Lost in the Middle" Problem
LLMs have U-shaped attention:
Solution: Put critical rules at TOP of AGENTS.md. Governance first, details later.
5. Signal-to-Noise Ratio
Strip everything that isn't essential:
Formats like llms.txt and AGENTS.md mechanically increase SNR.
llms.txt Standard
Machine-readable doc index for agents:
# Project Name> One-line project description.
Authentication
Setup: Environment vars and init
Server: Cookie handling Database
Schema: Full Prisma schema
Location: /llms.txt at domain root
Companion: /llms-full.txt β full concatenated docs, HTML stripped
Security Considerations
Inline = Trusted
AGENTS.md is part of your codebase. Controlled, version-pinned.External = Attack Surface
Mitigation: Domain allow-lists, human-in-the-loop for external retrieval.
Anti-Patterns
1. Pasting 50 pages β triggers "Lost in the Middle" 2. "See external docs" β agents can't browse autonomously 3. Generic advice β "Write clean code" (use specific constraints) 4. TOC-only docs β indexes without content 5. Trusting retrieval alone β 53% vs 100% pass rate
Advanced Patterns
For detailed guidance on RAG optimization, multi-framework docs, and API templates, see references/advanced-patterns.md.