DeepSeek research touts memory breakthrough (www.tomshardware.com)

DeepSeek has introduced a groundbreaking method called "Engram," which allows AI models to leverage a queryable database stored in system memory for enhanced long-context performance. This conditional memory-based technique alleviates the need for computationally expensive reasoning by committing sequences of data to static memory, enabling the GPU to focus on more complex tasks. The paper highlights that Engram models can outperform standard Mixture of Experts (MoE) models, particularly in tasks requiring extensive memory retention, by reducing reliance on high-bandwidth memory (HBM), which is currently facing a supply squeeze. Engram models integrate N-grams into neural networks, enabling efficient access to factual information without repeated reasoning, significantly improving accuracy in long-context scenarios – as demonstrated by Engram’s notable success in the NIAH benchmark, achieving a score of 97% compared to the MoE model's 84.2%. By optimizing the balance between memory and computation, DeepSeek has presented a new paradigm in AI framework design. This approach could transform the industry, potentially alleviating pressure on the increasingly scarce HBM resources, and paving the way for future AI models built upon Engram's principles. The forthcoming DeepSeek release may be poised to further validate these claims in real-world applications.