Google must double AI serving capacity every 6 months to meet demand (www.cnbc.com)

At a Nov. 6 all‑hands, Google Cloud VP Amin Vahdat warned the company must “double every 6 months” its AI serving capacity to meet surging demand — a trajectory he framed as a 1,000× increase over 4–5 years. Vahdat said Google will meet that growth both by building more data‑center capacity and by squeezing far more efficiency from models and custom silicon; Google just launched its 7th‑generation Cloud TPU (Ironwood), which it says is nearly 30× more power‑efficient than its first 2018 TPU. Sundar Pichai and CFO Anat Ashkenazi acknowledged the financial scale — higher capex guidance and hyperscaler peers also upping spend — and warned capacity, not models, is often the bottleneck for wider rollout (e.g., Gemini 3 and the Veo demo couldn’t be served broadly due to compute limits). For the AI/ML community this signals a major systems challenge: serving billions of inference requests will require simultaneous 1,000× gains in compute, storage and networking while holding cost and energy roughly steady. Expect faster co‑design of models and hardware, more aggressive model compression (quantization, sparsity, distillation), compiler/runtime and sharding innovations, and continued consolidation in cloud/infrastructure markets. The pace of capex raises questions about financial sustainability if demand softens, but Google’s message is clear: scaling AI in production now hinges as much on datacenter engineering and custom accelerators as on model breakthroughs.