Gambling with our lives: AI researcher quits Anthropic with warning about safety (www.politico.eu)

Jacob Coxon, an AI researcher who recently resigned from Anthropic, has raised alarm over the rapid advancements in artificial intelligence, claiming that both Anthropic and OpenAI are "gambling with our lives." In his post on X, he emphasized that AI systems are on the brink of becoming superhuman, with the potential to hack systems, transform industries overnight, and gain unprecedented power and resources. This statement highlights a growing concern within the AI/ML community about the accelerating pace of technology without adequate safety measures. Coxon's major concern revolves around the concept of self-improving superintelligence, where AI models could develop more advanced versions of themselves, leading to an unstable feedback loop. This scenario poses significant risks if proper safeguards are not implemented, as it could result in AI systems operating beyond human control. His departure serves as a wake-up call to the tech giants involved, urging the industry to prioritize ethical considerations and safety protocols amidst the race for advanced AI capabilities.