This tiny cybersecurity startup managed to hack OpenAI using Claude, and won a $6,500 bounty (www.businessinsider.com)

A small cybersecurity startup called Hacktron successfully exploited vulnerabilities in OpenAI's infrastructure using the AI model Claude, shortly after the OpenAI Hugging Face hack. Co-founder Zayne Zhang announced that Hacktron investigated security weaknesses in major AI companies, discovering that any user or employee logging into OpenAI's community forum could have had their accounts compromised. By leveraging Anthropic's Cyber Verification Program, which facilitated a more relaxed environment for security research, Hacktron was able to hack into an employee's account, prompting suggested code changes while stopping short of accessing internal code. This incident is significant for the AI/ML community as it highlights emerging security challenges at prominent AI organizations, emphasizing the convergence of AI safety and cybersecurity as critical fields of concern. Hacktron's successful discovery earned them a $6,500 bounty and underscores the importance of proactive security measures. OpenAI has responded by tightening restrictions on community sign-in tokens and revoking affected sessions. As the AI landscape evolves, the need for cybersecurity experts to address vulnerabilities becomes increasingly vital, particularly amid rising fears of potential threats from rogue AI agents.