Malicious Agent Skills in the Wild (arxiv.org)

A recent study, “Do Not Mention This to the User,” highlights critical security vulnerabilities in skills developed for large language model (LLM) coding agents. Researchers analyzed over 98,000 skills from two major registries and discovered 157 skills exhibiting malicious behavior, revealing a total of 632 vulnerabilities across 13 different attack techniques. The malicious skills typically featured an average of 4.03 vulnerabilities each, with two primary attack strategies identified: credential theft through remote code execution and manipulation of agents via adversarial instructions in the skills' documentation. Alarmingly, more than half of these malicious cases were traced back to a single threat actor using impersonation tactics. This research is significant for the AI/ML community as it underscores the potential risks associated with third-party extensions in LLM systems, which can operate with full user privileges. The findings also illuminate how attackers are exploiting platform trust mechanisms through sophisticated concealment techniques. Following the study's responsible disclosure process, all identified malicious skills were removed from the registries, reinforcing the importance of proactive security measures in safeguarding AI ecosystems. The publicly available dataset and detection pipeline promise to aid future research aimed at enhancing the security of LLM agent applications.