Sophon PFG-1: a monolithic-3D AI ASIC with 330 GB of on-die DRAM and no HBM (www.phantafield.com)

Sophon has unveiled the PFG-1, an innovative monolithic 3D AI application-specific integrated circuit (ASIC) that offers a significant leap in processing capability for machine learning tasks. This 750 mm² die features a groundbreaking 330 GB of on-die DRAM and operates entirely without high-bandwidth memory (HBM). The architecture integrates 32 tiers of 2D Transition-Metal Dichalcogenide (TMD) technology, allowing for unified training and inference. With an impressive capability of 4,200 TFLOPS in FP8 mode and 2,100 TFLOPS in BF16, the PFG-1 can handle both training and inference tasks seamlessly, achieving extraordinarily high throughput without the constraints faced by traditional HBM-bound systems. This advancement is especially crucial for the AI/ML community as it addresses the memory bottleneck that modern AI accelerators face during both training and inference. By leveraging the unique properties of TMD materials, the PFG-1 achieves unprecedented weight bandwidth—191 to 214 times that of HBM4 packages—and maintains low energy consumption, operating at peak efficiencies of 3.72 TFLOPS/W. With its capacity for uninterrupted on-die training and inference, the Sophon PFG-1 can adapt to varying workloads without hardware modifications, making it a highly flexible and cost-effective solution compared to current high-end offerings from competitors like NVIDIA and AMD. This architecture not only optimizes memory usage but also dramatically reduces overall costs, offering a potential shift in how large-scale AI models are implemented and run.