Open weights models are surprisingly aligned on offensive cyber (pentesttoday.com)

Recent evaluations of open-weights AI models, particularly from China, reveal that they, like their American counterparts, generally refuse to engage in offensive cyber tasks when tested against the OWASP Juice Shop—a training target for penetration testing. Despite concerns raised in the tech community about the potential risks posed by these advanced models, the findings indicate a surprising degree of compliance in avoiding harmful actions, particularly in models like Claude Fable 5.1, which did not find any vulnerabilities during testing. This raises questions about the motivations behind these refusals and the implications for cybersecurity protocols in AI development. The significance of these findings lies in highlighting that both Chinese and American AI models are incorporating safety measures into their design, a trend that is critical as AI capabilities evolve. While open-weights models can theoretically be manipulated due to their accessibility, the presence of robust anti-cyber capabilities suggests a cautious approach to AI deployment in sensitive areas like cybersecurity. The results imply a need for further investigation into how entrenched safety mechanisms operate within various AI frameworks, especially given the potential for those safeguards to be bypassed or altered in less-regulated environments.