Generate llms.txt and per-page Markdown for the docs - #69
Merged
Conversation
Adds mkdocs-llmstxt, which publishes /llms.txt as an index of the documentation and a .md counterpart for every page. Feeding docs to an LLM currently means copying rendered HTML, which loses the structure that makes it useful. The plugin renders each page to HTML and converts it back to Markdown, rather than reading the source. That matters here: the API pages are a single `::: ikpykit.X` directive each, so a source-level export would produce nothing. Going through the rendered output captures the full mkdocstrings result -- signature, description, and parameter tables. It also means the plugin has to run after mkdocstrings and mkdocs-jupyter, hence its position last in the plugins list. Sections mirror the nav. The INNE notebook is listed explicitly because the user_guides/*.md glob does not match .ipynb, and authors.md is included so every page in the nav is covered; verified by diffing the nav against the generated index. full_output produces llms-full.txt (208K) with every page expanded, for pasting a whole corpus in one go. Two caveats worth recording. The plugin is in maintenance mode upstream, its author having moved to Zensical, so it works but will not gain features. And mike deploys per version, so these files land under each version prefix rather than at the site root.
This was referenced Aug 15, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds
mkdocs-llmstxt, which publishes/llms.txtas an index of the documentation plus a.mdcounterpart for every page.Feeding these docs to an LLM currently means copying rendered HTML, which loses the structure that makes it useful.
What gets generated
llms.txt(4K) — an index, one entry per page, grouped by section:llms-full.txt(208K) — every page expanded, for pasting a whole corpus at once.24 per-page
.mdfiles, one per documentation page.Why it works on the API pages
The plugin renders each page to HTML and converts it back to Markdown, rather than reading the source. That distinction matters here. Every API page is a single directive:
A source-level export would produce exactly that and nothing else. Going through the rendered output captures the full mkdocstrings result:
This is also why the plugin has to run after mkdocstrings and mkdocs-jupyter, hence its position last in the
pluginslist. A comment inmkdocs.ymlrecords that, since the ordering is not obvious and silently degrades if changed.Coverage
Sections mirror the nav. Two entries needed care:
user_guides/inne.ipynbis listed explicitly — theuser_guides/*.mdglob does not match.ipynbauthors/authors.mdwas missing from my first passVerified by diffing every page in the nav against the generated index; the difference is now empty.
Caveats worth recording
The plugin is in maintenance mode. Its README states the author has moved to Zensical and is open to transferring maintainership. It works and is stable, but will not gain features. The configuration is about twenty lines in
mkdocs.yml, so backing it out later is cheap.Material has no "Copy as Markdown" button. squidfunk/mkdocs-material#8382 requested exactly that and was closed as a duplicate in August 2025, with the maintainer putting it on the backlog pending the end of feature freeze. So the
.mdURLs are the entry point for now, rather than an in-page button. If Material ships the button later, it composes with this rather than replacing it.mike deploys per version, so these files land under each version prefix (
/0.3.0/llms.txt) rather than at the site root. Whether a root-level/llms.txtis wanted is a separate question — it would need a redirect or abase_urloverride.Minor conversion artifacts. The notebook page picks up one
¶permalink and oneCopied!from the rendered HTML. The plugin supports apreprocesshook to strip these, but for two occurrences in a 75-line file that seemed like more maintenance surface than it is worth.Verification
mkdocs build --strictpasses,uv lock --checkis consistent, and all six pre-commit hooks pass.