Maia 200: The AI accelerator built for inference – The Official Microsoft Blog (blogs.microsoft.com)

Microsoft has unveiled the Maia 200, a groundbreaking AI accelerator optimized for inference that enhances the economics of AI token generation significantly. Built on TSMC's 3nm process, the Maia 200 features advanced FP8/FP4 tensor cores and a robust memory system, delivering over 10 petaFLOPS in FP4 and more than 5 petaFLOPS in FP8 within a power-efficient 750W envelope. This technology outperforms competitors, boasting three times the FP4 performance of Amazon's Trainium and superior FP8 metrics compared to Google’s TPU. With 216GB HBM3e at 7 TB/s and 272MB on-chip SRAM, it addresses critical data movement bottlenecks, essential for substantial AI model size and operational efficiency. The Maia 200 is set to enhance various applications, notably integrating seamlessly with Azure and supporting advanced models like GPT-5.2. It will aid in synthetic data generation and reinforcement learning, accelerating the creation of high-quality training datasets. The architecture employs a novel two-tier network design that allows scalable performance across large inference clusters, optimizing both cost and power consumption. Furthermore, with a focus on cloud-native development, Maia 200's quick deployment and robust SDK, including tools like the Triton compiler and PyTorch support, invite developers and researchers to explore optimization solutions, solidifying Microsoft’s commitment to defining the future of AI infrastructure.