More Was Possible: A Review of If Anyone Builds, It Everyone Dies (asteriskmag.com)

Eliezer Yudkowsky and Nate Soares have published If Anyone Builds It, Everyone Dies, a popular-audience plea from MIRI arguing that building advanced AI will almost certainly produce a single, rapidly self-improving superintelligence (an “intelligence explosion” or FOOM) that is alien in values and will likely cause human extinction unless building is stopped. The book restates long-standing MIRI claims and proposes sharply interventionist policies — for example, strict caps and international monitoring on ownership of the most powerful GPUs used for training — grounded in the idea that we get “one shot” to align a superintelligence. This is significant because MIRI’s framing pushes a maximal-risk, prevention-first agenda that diverges from much of contemporary AI safety research, which treats progress as more continuous and amenable to iterative alignment work. Technically, the controversy hinges on empirical claims the book does not convincingly defend: the likelihood and mechanics of FOOM, whether modern neural architectures can produce goal-directed, self-improving agents, and whether current research pathways produce meaningful continuity with future AGI. The reviewer finds the book more accessible but less coherent than earlier writings: it emphasizes hypothetical agent motivations and draconian policies while under-explaining or ignoring the empirical case for rapid uncontrollable self-improvement. The bottom line: the book raises important alarms, but its sweeping conclusions and policy prescriptions depend on contested technical assumptions that merit clearer, evidence-based argumentation.