How Far Behind the Frontier Are Leading Open-Weight Models on Cyber? (www.aisi.gov.uk)

AISI recently published an analysis highlighting the cyber capabilities of leading open-weight AI models, revealing that they consistently lag behind closed-weight models, which are more secure but restricted in access. This evaluation shows that the most advanced open-weight model, GLM-5.2, trails closed models like Opus 4.6 by approximately 4 to 7 months. Despite improvements in the gap compared to previous evaluations (which indicated a 6 to 10 month delay), this disparity raises concerns as cyber threats evolve rapidly, especially with frontier models making significant leaps in capabilities. The significance of these findings lies not just in performance metrics, but in the implications for cyber defense. Open-weight models allow for private hosting and adaptability, fostering innovation and collaboration; however, their lack of inherent safeguards makes them susceptible to misuse once released. This presents a dual challenge for cybersecurity: leveraging the adaptability of open models while managing the persistent risks they entail. The report underscores the urgency for organizations to bolster their cybersecurity measures ahead of potential threats posed by increasingly capable open models. AISI plans to continue evaluating new releases, including the forthcoming Kimi K3, to monitor trends in this crucial area.