What does "verified memory safe" mean? (focs-lab.comp.nus.edu.sg)

On October 22, 2026, Anthropic and Theorem announced a pioneering achievement in AI with the generation of mathematical proofs for programs derived from 123 open-source libraries. This development in automated verification is significant for the AI and machine learning community as it addresses critical memory-safety bugs that can compromise cybersecurity. However, the findings also reveal that while these proofs validate certain properties of programs, they may not capture all potential vulnerabilities; a program validated against an incomplete specification can still harbor memory-safety bugs. Key technical insights from the investigation highlight the importance of precise specifications when delegating verification tasks to AI. The use of AddressSanitizer (ASan) for memory error detection is a crucial aspect of the evaluation, yet gaps remain in how specifications define acceptable memory access and usage. For instance, tests on libraries like coturn and OpenSSL demonstrate that a validated proof could overlook significant safety requirements, like buffer boundaries and reference lifetimes. This underscores a pressing need for rigorous scrutiny of AI-generated proofs, ensuring that all intended behaviors are faithfully captured, thereby preventing false security assurances in software development.