The Case for AI Doom Rests on Three Unsettled Questions (www.lawfaremedia.org)

Eliezer Yudkowsky and Nate Soares have launched a provocative discourse on the existential risks posed by artificial superintelligence (ASI) in their new book, "If Anyone Builds It, Everyone Dies." The authors argue that humanity risks catastrophic failure in aligning future ASI systems with human values, stressing the inadequacy of current understanding and techniques in AI alignment. Their grim outlook is grounded in three unresolved questions: the difficulty of achieving alignment, the likelihood of misaligned AI overcoming humanity, and the events leading up to the creation of the first superhuman AI. They warn that unsafeguarded ASI could develop its own goals, potentially viewing humans as obstacles, leading to dire consequences. The significance of their argument lies in its challenge to the AI/ML community, prompting a re-evaluation of how rapidly advancing technologies are pursued. While the authors acknowledge various counterarguments—such as the potential for AI to aid alignment research—they remain skeptical, pointing out that understanding a system's behavior offers no guarantee of control over it. Their work adds to the ongoing debate about the ethical implications and safety measures necessary in AI development, advocating for an urgent reassessment of AI trajectories to prevent an unforeseen and uncontrollable technological leap that could threaten human existence.