OAI: EVM Bench LLM Accuracy on Smart Contract Review and Pentesting (openai.com)

OpenAI, in collaboration with Paradigm, has launched EVMbench, a groundbreaking benchmarking tool designed to assess AI agents' abilities in detecting, patching, and exploiting vulnerabilities in smart contracts. Smart contracts are vital for securing over $100 billion in crypto assets, and with the rise of AI’s code management capabilities, this benchmark addresses the urgent need for robust security measures in the rapidly evolving landscape of decentralized finance. EVMbench utilizes 120 high-severity vulnerabilities from various audits and comprises three evaluation modes: Detect, Patch, and Exploit, which examine how well AI agents can identify risks, mitigate them, and simulate attacks in a controlled environment. The initial results reveal that advanced models like GPT‑5.3‑Codex have significantly improved capabilities, particularly in the exploit mode, achieving a score of 72.2%. However, challenges remain in the detect and patch phases, where agents often underperform in thorough auditing and maintaining contract functionality while closing vulnerabilities. EVMbench serves as both a critical evaluation tool and a call to action for developers and security researchers to incorporate AI into their security protocols. As AI continues to mature, the tool emphasizes the dual-use nature of these technologies, pushing for responsible defensive strategies to enhance the resilience of smart contract ecosystems amidst emerging cyber threats.