DeepSeek v4.1 Flash Is Now Our Best Hacking Model (enclave.ai)

DeepSeek v4.1 Flash has achieved an impressive milestone in AI hacking by successfully executing code on all 11 vulnerable targets in its benchmark tests while maintaining security on four fixed targets. Remarkably, this was accomplished at a running cost of just $4.65. A comprehensive review of each attack revealed not only six effective solutions that utilized planned vulnerabilities but also five additional unexpected routes that were not initially recognized by the scoring system. This highlights both the model's exceptional capability in hacking scenarios and indicates areas where the benchmark could implement stricter validation measures. The benchmark showcased DeepSeek's proficiency across various applications, including Grafana, Jenkins, and Nextcloud. For instance, it cleverly exploited a timing issue and security gap during command execution to manipulate application behavior, achieving results in notably less time than anticipated. With 2,349 Bash commands executed over nearly 2 hours and 38 minutes, and an efficiency boosted by caching mechanisms, DeepSeek's performance signifies not only a robust advancement in AI hacking capabilities but also underscores the need for evolving standards in benchmarking methodologies. This advancement could set new benchmarks for assessing AI models in cybersecurity challenges, emphasizing the importance of evaluating both end results and the methods employed to achieve them.