Confluent cofounder says believing in AI's upside means reckoning with its 'dark side' (www.businessinsider.com)

Jay Kreps, cofounder of Confluent and former board member at Anthropic, has emphasized the urgent need for the AI community to confront the potential dangers posed by artificial intelligence as its capabilities rapidly advance. In a recent post, Kreps highlighted that every positive application of AI has a corresponding "dark version" that could be exploited for malicious purposes, such as superhuman coding tools being used for hacking or advanced AI drug design potentially creating undetectable poisons. This commentary comes amid rising concerns following alarming real-world incidents that have underscored AI's alignment issues. The significance of Kreps’ stance lies in its reflection of growing unease within the AI community, which has witnessed high-profile resignations and warnings from notable figures, including Geoffrey Hinton, who suggested that AI could pose an existential threat. Despite some backlash from industry leaders like Elon Musk, Kreps argues that addressing these risks should not be dismissed as neuroticism. He asserts the importance of establishing better safety measures and encourages open discussions about AI risks to avoid future disasters, reinforcing the idea that recognizing and preparing for potential downsides is essential for the responsible advancement of AI technology.