Fable Ban and Reversal Exposed We Cant Measure AI Cyber Risk in Real World (orionpolicy.org)

The recent ban on Anthropic's AI models, Mythos and Fable, by the U.S. Commerce Department highlighted significant gaps in the government's ability to measure real-world AI cyber risks. The initial order aimed to cut off foreign access to these advanced models due to concerns over potential misuse. However, following Anthropic's compliance, which temporarily disabled the models globally, the government reversed its decision after 18 days, restoring access amid the realization that rival models with similar capabilities remained unaffected. This incident underscores the complexities of governing AI technology without clear, evidence-based assessments of its operational role in actual cyber intrusions. For the AI/ML community, the significance of this occurrence lies in the urgent need for a standardized federal record to assess AI's impact on cybersecurity threats. Despite acknowledging that AI can facilitate cyber attacks, the government lacks a system to measure its involvement in attacks accurately. Suggestions, such as adding specific questions to incident reporting rules, aim to clarify AI's role in vulnerabilities and exploitations. By incorporating metrics like whether AI found vulnerabilities or wrote exploits, the proposal seeks to create a data-driven approach to understand AI's real-world implications, guiding future policy decisions and fostering a more robust framework to govern its use in cybersecurity.