Knowledge just moved out of the GPU (Engram architecture overview) (three.dev)

DeepSeek has introduced a groundbreaking architectural concept called Engram, which shifts knowledge storage from traditional GPU-embedded neural network parameters to a hash-based lookup table. This innovation has already been adopted in two industry-grade models: DeepSeek's v4.1-Flash and Qwen3.8-Flash-Next. The Engram architecture significantly alters the economics of large language model (LLM) inference by allowing for a separation of computation and knowledge storage, effectively reducing the memory bandwidth burden on GPUs. This decoupling means that knowledge can be housed in cheaper DDR5 or even SSDs, rather than the expensive high-bandwidth memory (HBM) typically associated with GPUs. The technical implications of Engram are profound. By employing a simple key-based lookup system, the architecture rapidly retrieves contextual knowledge—such as recognizing "New York" as a U.S. city—without needing extensive computation, unlike traditional LLMs that rely heavily on GPU cycles. As a result, these models can operate on lower-spec hardware while still handling substantial parameters effectively. This shift could revolutionize capacity planning for AI applications, fostering new avenues for specialized models that can efficiently manage their knowledge and reasoning independently. The concurrent rollout of this design by multiple labs signals its transition from experimental to a mainstream solution in the AI/ML landscape.