Who Needs DRAM? We Have Fiber (arxiv.org)

A new architectural innovation called Fiber Memory has been proposed to address the escalating demand for high-performance memory driven by generative AI and the increasing consumption of DRAM in hyperscale data centers. This approach leverages optical fiber as an active, recirculating delay-line memory specifically designed for immutable data, such as the weights of large language models (LLMs). By utilizing space-division multiplexed multi-core fibers, passive optical components, and regional all-optical regeneration, Fiber Memory aims to streamline memory storage and reduce energy costs dramatically. The significance of Fiber Memory lies in its potential to revolutionize data management in AI applications. By eliminating redundant weight storage across thousands of AI accelerators, this architecture could help alleviate the pressure currently placed on conventional DRAM systems, with the potential to cut weight-delivery energy consumption by over 70% compared to traditional HBM3e configurations. This groundbreaking shift not only enhances the efficiency of AI workloads but also addresses critical supply chain challenges in high-performance memory solutions, making it a game changer for the AI/ML community.