Profiling the Attack Surface of AI Agent Skills (arxiv.org)

A recent study titled "Beyond Maliciousness: Static and Runtime Attack-Surface Profiling of Agent Skills" introduces a new two-phase methodology for assessing the attack surface of agent skills, which are instructions that tool-using agents follow with specific privileges. Researchers analyzed over 6,300 listings from two marketplaces, revealing that half of the evaluated skills can execute shell commands and many have elevated access capabilities. Notably, many skills under-report their potential permissions, making it challenging for users to assess the risks involved at the time of installation. This research is significant for the AI/ML community as it highlights the limitations of current permission models and the necessity for developers and marketplaces to adopt a capability-oriented attack-surface profiling approach. By focusing on what skills can actually do rather than what they claim, the findings call for enhanced security measures to protect users from unforeseen vulnerabilities. This comprehensive evaluation underscores the urgent need for transparency around agent capabilities, ultimately aiming to bolster trust and security in AI applications.