Google unveils two new TPUs designed for the “agentic era” (arstechnica.com)

Google has introduced its eighth-generation Tensor Processing Units (TPUs), namely the TPU8t for training and TPU8i for inference, marking a shift towards what they term the "agentic era" of AI. This era is characterized by a need for advanced hardware that better supports the complexity and capabilities of next-generation AI models. In contrast to the competition, primarily reliant on Nvidia's solutions, Google's custom TPUs aim for enhanced speed and efficiency tailored to the demands of advanced AI applications. The TPU8t specifically focuses on reducing training times for large AI models, cutting down the weeks-long process significantly compared to previous generations. Equipped with updated Tensor 8t server clusters—housing 9,600 chips and two petabytes of high-bandwidth memory—these new units are capable of scaling linearly, with potential expansions reaching up to a million chips in a cluster. With an impressive compute power of 121 FP4 EFlops per pod, nearly three times that of the previous Ironwood generation, this innovation not only accelerates the training of massive AI systems but also raises the stakes in RAM pricing across the industry, benefiting those involved in cutting-edge AI development.