DeepWiki to MdBook Converter (docs.deepwiki-to-mdbook.zenosmosis.com)

🤖 AI Summary
DeepWiki-to-mdBook Converter is a containerized pipeline that scrapes AI-generated wiki documentation from DeepWiki.com and converts it into a self-contained, searchable mdBook site (or plain Markdown) so projects can archive, browse, and edit generated docs offline. It solves platform lock-in, client-side-only Mermaid rendering, lack of offline access, and poor searchability by HTTP scraping with requests + BeautifulSoup4, extracting JS-embedded Mermaid payloads (with fuzzy matching), converting HTML to Markdown, and running mdBook with a Mermaid preprocessor to produce a static HTML site with built-in search. Technically the system is three coordinated components inside a Docker image: a Python 3.12 scraper (deepwiki-scraper.py) that fetches HTML, parses to Markdown (html2text), extracts ~461 diagrams and fuzzy-matches ~48 with sufficient context, and writes atomic outputs to /tmp/wiki_temp → /output/markdown; a shell orchestrator (build-docs.sh) that generates book.toml and SUMMARY.md; and mdbook + mdbook-mermaid (Rust binaries compiled in a multi-stage Docker build) to render the final book/ directory including mermaid.min.js and searchindex.json. Configuration is environment-driven (REPO, BOOK_TITLE, BOOK_AUTHORS, MARKDOWN_ONLY), supports auto-detection from Git remotes, requires no authentication, and yields either markdown-only or full HTML output—making DeepWiki content portable, CI-friendly, and easier to maintain.
Loading comments...
loading comments...