The Danger May Be AI as Dumb as Us (foreignpolicy.com)

Recent discussions around the potential dangers of artificial intelligence have shifted focus from the far-off threat of “superintelligence” to immediate concerns about the limitations of current AI systems. The term "super intelligence," popularized by philosopher Nick Bostrom, is now being used by figures like former President Trump, but the real danger may stem from so-called "jagged intelligence." This refers to AI systems that exhibit sophisticated capabilities while still making fundamental errors, which can lead to disastrous outcomes, especially in high-stakes environments like national security. Overreliance on these flawed systems has already resulted in civilian casualties and near-catastrophic military incidents, raising alarms about the integration of AI in defense strategies. As military and corporate sectors continue to adopt advanced AI technologies, there are concerns about automation bias, where users blindly trust AI decisions due to their complexity. Historical parallels draw attention to past technological failures that caused significant crises, suggesting an urgent need for oversight and a reevaluation of automated decision-making processes. Experts warn that rather than creating apocalyptic scenarios, the more immediate risks involve errors from AI hallucinations, misinformation, and misunderstandings that could escalate conflicts, ultimately highlighting that the dangers of AI may not lie in its intelligence alone, but in its unpredictability and the human tendency to misjudge its reliability.