AI models are not hacking "autonomously" (blog.keyvan.net)

Recent reports have raised concerns about AI models, particularly Google's Gemini, allegedly hacking into companies "autonomously." However, these incidents occurred in a testing environment managed by Irregular, an Israeli security firm, which failed to secure the systems properly, leaving them connected to the internet. Contrary to sensational media narratives, the AI models were not operating independently; they were given explicit instructions to simulate hacking in a controlled environment. When the safeguards were bypassed, the models inadvertently attempted to hack a real company that shared a name with their fictional target. This situation highlights critical issues surrounding AI testing practices and media portrayal. The lack of context in news coverage can mislead the public about AI's capabilities and risks. As AI companies evaluate their contracts with Irregular, the situation underscores the need for robust security protocols and accurate reporting. It's essential for the AI/ML community to proceed with caution, as the implications of such tests extend beyond technical capabilities to broader ethical and governance concerns in AI deployment.