Memory-Centric AI: SanDisk's High Bandwidth Flash (www.sandisk.com)

SanDisk has unveiled its High Bandwidth Flash (HBF) technology, a groundbreaking NAND-based memory architecture designed to tackle the growing compute-memory gap—or "memory wall"—that challenges scaling AI models. Unlike traditional approaches that rely heavily on increasing GPU compute power or face limitations at the edge, HBF delivers 8 to 16 times the capacity of High Bandwidth Memory (HBM) while matching its read bandwidth at comparable cost. This makes it ideal for both datacenter GPUs, where terabytes of memory become accessible, and edge AI devices, enabling complex models previously constrained by power, cost, and size. The innovation is timely as AI models evolve toward Memory-Centric AI, driven by massively increased parameters and context lengths combined with architectural shifts like Mixture of Experts, which lower compute demands but raise memory requirements. SanDisk’s simulations with a 400-billion parameter Llama 3.1 model show HBF’s performance rivals HBM within just 2.2%, defying prior skepticism about NAND latency and endurance. Moreover, collaborations with experts like KAIST and industry leaders such as SK hynix and advisors like David Patterson and Raja Koduri are laying the foundation for an industry-standard ecosystem around HBF, ensuring scalability and commercial viability. Scheduled for initial sampling in late 2026 and inference devices by early 2027, HBF promises to redefine AI memory with massive capacity, high bandwidth, and low footprint. This advance could unlock new AI applications from large-scale datacenter inference to personalized edge computing, marking a pivotal shift toward memory-centric architectures in next-generation AI systems.