Query Decomposition for RAG (arxiv.org)

The paper frames query decomposition for retrieval-augmented generation (RAG) as a sequential exploration–exploitation problem: complex prompts are split into subqueries, and documents are retrieved one at a time to build a belief about each subquery’s utility before deciding whether to keep exploiting it or explore alternatives. Treating subquery selection as a bandit learning task lets the system dynamically pick the most informative subqueries and stop when additional retrieval would add more noise than signal. The authors test a variety of bandit algorithms and show that combining rank-derived signals with human relevance judgments for document scoring substantially improves retrieval quality. Results are compelling for both retrieval and downstream generation: the proposed methods yield a 35% increase in document-level precision and a 15% improvement in α-nDCG (a diversity-aware ranking metric), and they also boost performance on long-form generation tasks. Practically, this approach addresses the core RAG trade-off of casting a wide net versus limiting noisy, costly retrievals, enabling more efficient, adaptive pipelines. For researchers and engineers, the contribution is a principled, sequential decision formulation plus empirical evidence that bandit-based subquery selection and richer relevance estimation measurably improve RAG systems’ precision, diversity, and generation quality.