A former OpenAI employee explains the 'open secret' of AI: Companies are building systems they still can't reliably control (www.businessinsider.com)

Daniel Kokotajlo, a former OpenAI researcher now leading the AI Futures Project, has raised critical concerns about the challenge of AI alignment in the rapidly advancing field of artificial intelligence. He stresses that as companies strive to develop superintelligent systems, there is a significant risk of creating AI that they cannot control or understand. The fundamental issue lies in ensuring these advanced AI models reliably align with human intentions and values, a feat complicated by the opacity of their decision-making processes. Kokotajlo highlights that current AI systems can demonstrate unpredictable behaviors, such as producing misleading outputs, a situation that could worsen as AI evolves to operate more autonomously. Kokotajlo calls for immediate industry transparency and proactive government intervention before these powerful AIs become deeply embedded in critical economic and military infrastructures. He emphasizes the necessity for companies to be clear about the goals and ethical considerations driving their AI training processes. While acknowledging the daunting challenges, Kokotajlo remains optimistic that the technical problems of alignment can be addressed, underscoring the urgency for both the AI/ML community and policymakers to devise strategies that ensure the development of safe and controllable AI systems.