From Claude Code to PageIndex: The Rise of Agentic Retrieval (vectifyai.notion.site)

AI practitioners are revisiting a leaner retrieval model: instead of precomputing embeddings and relying on vector DBs, “agentic retrieval” hands the model simple file tools (fetch, grep, read) plus a compact map (llms.txt) and lets the LLM plan its own search. That approach—popularized in code-focused workflows like Claude Code—has proven faster and more precise for structured codebases because the LLM’s reasoning loop (chain-of-thought) becomes the retrieval algorithm, following imports, filenames and docstrings rather than fuzzy semantic matches. PageIndex extends that paradigm to long-form documents by creating an LLM-native, vectorless hierarchical index (a human-readable table of contents with smart summaries) that sits in the model’s context (exposed via an MCP server). The model browses the tree, drills into sections on demand, and pulls only relevant pages, avoiding chunking, embeddings, and vector stores. Practical wins include shorter, cleaner contexts, faster iteration, human-readable debugging, model portability, and strong benchmark results (98.7% on FinanceBench). Choose PageIndex for structured reports, legal/financial docs, or when traceability and low infra overhead matter; stick with vector search for broad, fuzzy retrieval across many weakly structured sources. The broader implication: retrieval moves inside the model, shifting RAG design from similarity-centric infra to reasoning-centric, agentic systems.