Kimi K3 vs. Claude Opus 5.5: How Two Flagship LLMs Built Pokémon Emerald Worms (www.runsybil.com)

In a groundbreaking experiment, Kimi K3 and Claude Opus 5.5, two flagship large language models (LLMs), competed to find vulnerabilities in Pokémon Emerald by creating network worms that can disrupt gameplay. Despite starting from the same goal, the two models employed distinct strategies: Opus 5.5 demonstrated robust coordination and used specialized subagents to organize their research and handle computational trade-offs, while Kimi K3 focused on iterative debugging and parallel investigations, showcasing its strengths in running multiple trials. The comparison is significant for the AI/ML community as it highlights the contrasting approaches of open-weight versus closed-weight models in security research. Both models achieved similar outcomes in identifying novel exploits, yet their methodologies reveal critical insights into their operational strengths and weaknesses. For instance, Opus excelled at managing evidence and resource allocation, while Kimi thrived in adapting to challenges but occasionally faltered in organization. This study not only questions whether high-cost models are necessary for advanced security tasks but also raises important considerations about the human guidance needed for effectively leveraging LLMs in complex undertakings, potentially guiding future developments in AI-driven security research.