Memorilla: Latent Semantic Memory for LLMs (arxiv.org)

Researchers have introduced Memorilla, a novel memory system designed to enhance the capabilities of frozen large language models (LLMs) by allowing them to tap into a broader range of information. While traditional approaches limit the knowledge accessible to the model solely to its context window, Memorilla innovatively encodes document collections into reusable embeddings and retrieves relevant query-dependent information using a mere 16 memory tokens. This method not only outperforms existing baselines—achieving notable accuracy improvements, such as 84.33% on the FactKG benchmark—but also significantly reduces the token count used, making it more efficient. The significance of Memorilla lies in its potential to advance the field of AI and machine learning by facilitating more effective long-context interactions and personalization in language processing tasks. In tests conducted within a TextWorld environment, it demonstrated marked improvements in an agent’s decision-making accuracy, showcasing the system’s capacity for continual learning. By effectively managing memory limitations, Memorilla is a promising step towards creating AI systems that can learn continuously and accumulate knowledge over time, overcoming traditional constraints faced by static LLMs.