We hid backdoors in ~40MB binaries and asked AI + Ghidra to find them (quesma.com)

Researchers have tested AI agents, specifically Claude Opus 4.6, for their ability to detect backdoors in binary executables, achieving a surprising 49% success rate on relatively obvious threats in small to medium binaries. Collaborating with reverse engineering expert Michał "Redford" Kowalczyk, they created a benchmark that involved injecting artificial backdoors into widely used open-source software to simulate real-world malware detection scenarios. While the initial results demonstrated that these AI models can identify some malicious code, they also exhibited a high false positive rate, indicating that many clean binaries were flagged incorrectly. This research is significant for the AI and ML community, as it highlights both the potential and the limitations of current models in tackling cybersecurity challenges. The findings illustrate that while AI can enhance malware detection through automated binary analysis—transforming low-level machine code back into more interpretable formats—there is still considerable complexity involved. The findings expose that even advanced models may misinterpret obvious indicators of backdoor functionality as legitimate, underscoring the need for further advancements in AI capabilities and human expertise to ensure reliable malware detection and protection against increasingly sophisticated cyber threats.