Hackers can use 9 of the most popular AI tools to assemble massive botnets (arstechnica.com)

A recent study has unveiled a critical security vulnerability within popular AI tools, highlighting how hackers can exploit a newly identified attack method called HalluSquatting. This pull-based attack leverages the inherent weaknesses in large language models (LLMs), which struggle to differentiate between legitimate and malicious instructions. Unlike traditional prompt injection attacks that target individuals, HalluSquatting allows adversaries to plant malicious commands in code repositories that LLMs frequently access, leading to large-scale infections and potentially massive botnet formations. The significance of HalluSquatting lies in its ability to bypass existing security measures by preying on the LLMs' tendency to generate incorrect resource identifiers—also known as hallucinations. As AI coding assistants like GitHub Copilot and various CLI tools inadvertently pull infected code or commands from compromised repositories, they could unwittingly execute harmful actions, such as launching DDoS attacks or installing malware. This marks a new frontier for prompt-injection attacks, raising alarms in the AI/ML community about the need for more robust defenses and comprehensive threat mitigation strategies to protect against these sophisticated and scalable assaults.