llmcpd

The idea

AI coding agents are only as useful as the context you give them. When you're working with a library or framework, you want your agent to actually understand the docs — not hallucinate an API that changed two versions ago or ask you to paste in a wall of text every session.

A growing number of projects publish an llms.txt file: a clean, LLM-friendly version of their documentation stripped of navigation, ads, and the other noise that clutters a normal webpage. llmcpd takes that file, indexes it, and wraps the whole thing in an MCP server that any compatible AI agent can query directly. Search the docs, fetch a specific section, list what's available — all without leaving your agent session.

What it does

Getting started takes about ten seconds:

npx -y llmcpd --url https://example.com/llms.txt

From there, llmcpd crawls the documentation in the background — including any nested markdown files it finds linked from the main doc — and indexes everything for full-text search. If the project publishes an llms-full.txt with the complete expanded docs, llmcpd ingests that too, chunking it by heading so searches stay precise rather than returning giant blobs of text.

Results are cached to disk using standard HTTP validation (ETags, Last-Modified headers), so repeated lookups are instant and the upstream server isn't hammered on every request. When docs update, a reindex either picks that up automatically on a refresh interval or on demand.

Your AI agent gets tools for searching across all indexed content, fetching a specific page or section, listing what's available, checking indexing status, and triggering a manual re-crawl. It's designed to drop into an existing MCP setup with minimal configuration.

License

MIT