We reduced RAG retrieval cost 10× with a hippocampus-inspired memory substrate (www.bricbybric.ae)

A new retrieval system inspired by the hippocampus has been developed, claiming a 10-fold reduction in retrieval costs for retrieval-augmented generation (RAG) pipelines. Traditional methods rely on dense vectors and nearest-neighbor searches, which can be costly and inefficient. The new architecture encodes facts as sparse patterns, activating only a small number of neurons from a larger pool, mimicking the brain's memory retrieval process. This innovative approach enabled the system, referred to as "Hippocampus," to achieve 90.91% contradiction-free accuracy with approximately 12 tokens per answer, significantly outperforming existing methods like MiniLM-filtered and BM25 in both accuracy and efficiency. This development is significant for the AI/ML community as it challenges the reliance on conventional embeddings and models, promising a more efficient and potentially more accurate means of memory retrieval in AI systems. The creators emphasize a rigorous experimental methodology, akin to drug trials, ensuring reproducibility and transparency in results. Looking ahead, they aim to create a versatile TypeScript SDK for various applications, fostering an ecosystem where AI agents can build on past experiences, similar to human memory, paving the way for more sophisticated and capable AI systems that can maintain context across multiple sessions.