What Is Your P(doom)? (en.wikipedia.org)

The term P(doom) has gained traction in the AI safety community, representing the probability of catastrophic outcomes from artificial intelligence, especially concerning existential risks. Following the release of GPT-4 in 2023, prominent AI figures like Geoffrey Hinton and Yoshua Bengio emphasized these risks. A recent survey of AI researchers revealed an average P(doom) estimate of 14.4%, highlighting a significant concern within the field regarding AI's potential to lead to human extinction or severe disempowerment over the next century. This metric underscores the urgency for investments in AI safety and alignment research, as even low probabilities of existential threat can justify substantial preventative actions. Notably, responses varied widely among experts: while some, like Max Tegmark, estimate probabilities over 90%, others, like Marc Andreessen, argue for a negligible risk. This disparity not only illustrates the complexities of predicting AI's future but also prompts ongoing debates about the conditions under which these risks materialize, making P(doom) a critical focus for researchers dedicated to mitigating potential AI-related dangers.