Show HN: An AI agent fixed 98% of vulnerable deps in one run, 14% in the next (bomly.dev)

🤖 AI Summary
A recent study demonstrated that AI agents can significantly improve their efficacy in fixing vulnerable dependencies when integrated with the Bomly MCP server. By analyzing a complex 13-module Maven project with around 300 dependencies, the study revealed that using the server led to remarkable results—agents achieved 98% of fixable advisories resolved in every connected run. In contrast, without the server, outcomes varied widely, with Claude Code's performance fluctuating between 14% and 98%, while Codex CLI maintained a more consistent, but still lower range of 93-100%. These findings underscore the server's substantial role in dependency discovery for large projects where manual enumeration is impractical. The implications for the AI/ML community are noteworthy. For smaller projects, capable agents performed admirably on their own, achieving high completeness without the server's assistance. However, the integration of the Bomly MCP server was crucial for larger projects, drastically reducing the number of wrong claims and enhancing speed—Codex CLI became about 1.7 times faster with server assistance. This study not only highlights the potential of AI in automating vulnerability remediation at scale but also points to the importance of contextual awareness in complex dependency resolution tasks. The methodologies and data from the research are publicly available for further exploration and reproduction.
Loading comments...
loading comments...