Data centers use as little as 60% of power consumed for actual AI work, Nvidia claims — failures and restarts 'consume' a whopping 10% of every watt used (www.techradar.com)

Nvidia has revealed that AI data centers are operating at a significant efficiency deficit, utilizing only about 70% of their power for actual AI workloads. The remainder, approximately 30%, is consumed by inefficiencies related to cooling, power distribution, and system failures, including up to 10% lost due to process interruptions and misconfigured racks. This insight shifts the focus from the previously believed bottleneck of chip shortages to the power supply connection and infrastructure of the facilities themselves, emphasizing the urgent need for more efficient data center designs. In collaboration with Vertiv, Nvidia aims to overhaul data center operations to enhance sustainability and efficiency. The two companies plan to modify existing systems to recover some of this wasted capacity by optimizing power and cooling configurations. By addressing design-related inefficiencies, Nvidia and Vertiv believe they can significantly improve operational output, ultimately shifting the paradigm to a model prioritizing effective energy use over sheer power consumption. This initiative has broader implications for the AI/ML community, as it could facilitate more powerful AI systems while promoting environmental responsibility and reducing operational costs.