HeLa-Mem: Hebbian Learning and Associative Memory for LLM Agents (arxiv.org)

Researchers have introduced HeLa-Mem, a novel memory architecture designed to enhance long-term memory capabilities in Large Language Models (LLMs). Traditional memory systems for LLMs often use static context windows and unstructured embedding vectors, which limit their ability to maintain coherent conversations over longer interactions. In contrast, HeLa-Mem draws inspiration from biological memory processes, specifically Hebbian learning and associative structures, to create a dynamic graph linking memory through co-activation patterns, mirroring the human brain's episodic and semantic memory systems. The architecture features a dual-level organization: an episodic memory graph that evolves as interactions occur, and a semantic memory store that utilizes a method called Hebbian Distillation to identify and distill key memory connections into reusable knowledge. Preliminary results from experiments using the LoCoMo benchmark indicate that HeLa-Mem achieves superior performance across various question categories while requiring significantly fewer context tokens. This approach not only addresses a fundamental challenge for LLMs but also paves the way for more human-like reasoning and memory utilization in AI systems, potentially revolutionizing how language models interact and learn from their environments. The code is publicly available on GitHub, promoting further exploration and development within the AI/ML community.