Why AGI Will Not Happen (timdettmers.com)

A recent provocative blog post argues against the feasibility of achieving Artificial General Intelligence (AGI) and superintelligence, emphasizing the fundamental principle that "computation is physical." The author critiques the prevailing optimistically abstract discourse on AGI, particularly from Bay Area rationalist and effective altruism circles, pointing out that such perspectives often ignore the physical limitations inherent in computational systems. As transistors shrink, while computational efficiency improves, memory access becomes increasingly burdensome—a crucial insight that challenges the notion that faster and more capable GPUs can lead to exponential advancements in AI. The piece goes on to highlight that progress in AI, much like other scientific fields, requires exponential resources for linear improvements, and posits that we may have reached the limits of GPU performance. The author suggests that while incremental innovations in AI infrastructure, such as optimizations in cache handling, remain critical, the overall landscape appears stagnant in terms of revolutionary breakthroughs. This candid assessment serves as a reality check for the AI/ML community, encouraging a more grounded understanding of the physical constraints that may ultimately define the trajectory of artificial intelligence development.