Memo: Memory as a Model (arxiv.org)

A recent paper introduces MeMo (Memory as a Model), a novel framework designed to enhance large language models (LLMs) by allowing them to incorporate new knowledge efficiently, without altering their underlying parameters. Traditional LLMs remain static once pretrained, making it challenging to integrate timely, domain-specific information required for real-world applications. MeMo addresses this issue by creating a dedicated memory model that captures complex relationships between documents and is resilient to retrieval errors, ultimately avoiding the problem of catastrophic forgetting that can occur when retraining models. The significance of MeMo lies in its modular approach, which enables seamless plug-and-play integration with both open-source and proprietary LLMs, as it operates independently of the model's weights or output logits. Importantly, the retrieval costs associated with MeMo remain consistent regardless of corpus size during inference, making it a scalable solution. Experimental results on benchmarks like BrowseComp-Plus, NarrativeQA, and MuSiQue demonstrate that MeMo outperforms existing methodologies, paving the way for more adaptive and capable AI systems that can keep pace with evolving information needs in various applications.