DocMind, Streamlit Application Leveraging LlamaIndex, LangGraph, and LLM (github.com)

DocMind AI is a local-first Streamlit application for private document intelligence that combines LlamaIndex pipelines, LangGraph orchestration and a locally hosted LLM stack (Ollama/vLLM/llamacpp) to analyze PDFs, Office files, code and multimedia without sending data to the cloud. It uses a 5-agent supervisor (query router, planner, retrieval expert, synthesizer, validator) to handle complex multi-step queries, extract entities/relations into an optional PropertyGraphIndex, and produce contextual answers with persisted session state and deterministic caching—making it suitable for privacy-sensitive or regulated environments. Technically, DocMind implements hybrid retrieval (dense BGE‑M3 vectors with cosine + sparse FastEmbed BM25/BM42) via Qdrant server-side fusion (RRF/DBSF) plus always-on reranking (BGE cross-encoder for text, SigLIP/CLIP for visuals). Ingestion uses Unstructured parsing, DuckDB caching, AES‑GCM encrypted page images, OpenTelemetry traces and SQLite WAL for concurrency. GPU-accelerated inference leverages vLLM FlashInfer (FP8/INT8 KV cache) to run Qwen3‑4B‑Instruct with up to 128–262K context on capable RTX 4090 hardware, hitting high prefill/decode throughput. The setup is container-friendly, extensible via prompt templates, and designed for offline operation—a practical blueprint for deploying sophisticated, auditable document AI entirely on-premise.