MLP Memory: A Retriever-Pretrained Memory for Large Language Models (arxiv.org)

Researchers have introduced MLP Memory, a novel lightweight parametric module designed to enhance the functionality of Large Language Models (LLMs) by effectively integrating retrieval-based knowledge access. Traditional methods face a trade-off between efficient knowledge retrieval and the general capabilities of models, with retrieval-augmented generation (RAG) often resulting in higher latency and limited integration, while fine-tuning approaches risk losing performance over time. MLP Memory circumvents these issues by learning retrieval patterns without needing direct document access. By mimicking a k-nearest neighbors (kNN) retriever during pretraining, it creates a differentiable memory component that synergizes with Transformer architectures. The significance of MLP Memory lies in its demonstrable performance improvements and efficiency. The module achieved notable scaling gains of 17.5% and 24.1% on WikiText-103 and Web datasets, respectively, along with a 12.3% enhancement on five question-answering benchmarks. Additionally, it exhibited a substantial reduction in hallucinations and offered a 2.5x speed increase in inference compared to RAG while maintaining superior accuracy. This advancement presents a compelling alternative to both RAG and traditional fine-tuning methods, providing the AI/ML community a pathway to enhance model performance and reliability in real-world applications without sacrificing efficiency.