John Carmack muses using a long fiber line as as an L2 cache (www.tomshardware.com)

John Carmack has proposed an intriguing concept of using a long fiber optic line as a cache for AI model weights, potentially revolutionizing data access speed and efficiency in AI systems. Observing that single-mode fiber optics can reach speeds of 256 Tb/s over distances of up to 200 km, Carmack calculated that a fiber line could hold 32 GB of data in transit at any moment. By utilizing this fiber as a data cache, it could keep AI accelerators continuously supplied with necessary information, significantly reducing latency and offering a new avenue for power efficiency compared to traditional DRAM. This idea is notable as it highlights a shift towards optical computing solutions that could outperform conventional semiconductor memory in certain applications. Besides its impressive bandwidth, using light for data transmission promises substantial energy savings and scalability over fiber, although challenges such as the cost of extensive installations and the need for error correction in data transmission remain. Carmack also hinted at the feasibility of combining existing flash memory technologies with AI accelerators, presenting a more practical approach that aligns with current advancements in AI infrastructure. As discussions unfold on this topic, the idea continues to garner interest and input from tech leaders, suggesting potential pathways for next-generation AI systems.