Faster Retrieval for Better, More Efficient RAG (machinelearning.apple.com)

Researchers from Apple have introduced a novel approach to improve maximum inner product search (MIPS) for recommendation systems and retrieval-augmented generation (RAG), which is crucial for efficiently matching user queries with relevant documents. Their paper, "Amortizing Maximum Inner Product Search with Learned Support Functions," presented at ICML 2026, proposes training small neural networks that learn to route queries to their best matches more quickly and accurately. This approach overcomes traditional limitations of MIPS that often assume queries resemble key datasets; instead, it tailors techniques to the actual distributions of user-generated queries, significantly enhancing both speed and retrieval efficiency. The two proposed models, SupportNet and KeyNet, demonstrate innovative training methods that leverage the geometry of key clusters to optimize the search process. SupportNet learns to approximate the support function for each cluster, drastically increasing routing accuracy, while KeyNet transforms queries to guide existing approximate search indices to the best matches. This leads to impressive improvements, achieving up to 97% recall in far less time than conventional methods. These advancements highlight a critical shift towards more adaptive and efficient AI systems in query handling, ultimately benefiting user experience by delivering relevant results faster while utilizing computational resources more effectively.