The coming AI compute crunch (martinalderson.com)

A new analysis highlights a looming "compute crunch" in the AI sector, driven by explosive token consumption as models like GPT-4 and Claude Code significantly enhance user productivity. The number of users engaging with large language models (LLMs) has skyrocketed, currently estimated at around one billion, leading to a dramatic surge in demand for computational resources. This demand strains existing infrastructure, with massive investments in data centers coming from major cloud providers like AWS and Azure, but concerns arise about the actual ability to deploy such capital effectively. One critical constraint identified is the shortage of DRAM, essential for powering the GPUs and TPUs that underpin AI models. Current supply levels can only accommodate the rollout of about 15GW of AI infrastructure, which is insufficient given the rapid expansion of user bases and use cases. Predictably, the imbalance between soaring demand and insufficient supply could lead to increased prices for AI services. This dynamic may spur further research into model efficiencies and alternative memory architectures as companies seek to navigate the constraints of the DRAM shortage and maintain a competitive edge in the fast-evolving AI landscape.