Show HN: Xtriever – offline RAG retrieval on a phone (github.com)

A new offline hybrid retrieval engine called Xtriever has been announced, designed for retrieval-augmented generation (RAG) and capable of running directly on devices like iPhones, Android phones, and laptops without needing an internet connection. Written in Rust, Xtriever processes queries through a four-stage pipeline: lexical search using BM25, dense embeddings via MiniLM, fusion of results, and re-ranking with a cross-encoder. This innovative architecture allows real-time access to a personal index of documents while ensuring consistency across multiple platforms including Rust, Python, Swift, and Kotlin. The significance of Xtriever lies in its ability to perform efficient, reliable searches on mobile devices, which have traditionally been limited by processing and memory capacity. The system integrates advanced features such as deterministic outputs, error handling that degrades gracefully rather than failing, and model verification for added robustness. Notably, it achieves high performance metrics, demonstrated through the evaluation on research datasets, highlighting its capabilities in both speed and accuracy. This development could empower a wide range of applications, from personal information retrieval to enhanced offline capabilities in mobile AI applications, marking a notable step forward in the AI/ML landscape.