AI assistants can be hijacked and manipulated by inaudible sounds (arxiv.org)

Recent research highlights a significant vulnerability in modern Large Audio-Language Models (LALMs) through a method called auditory prompt injection. This technique allows malicious actors to manipulate these AI systems using imperceptible audio prompts, posing a serious security threat. The study introduces a framework named *AudioHijack*, which can stealthily inject adversarial audio into LALMs, manipulating their behavior without users’ awareness. Remarkably, the attack was successful across various models and contexts, achieving hijacking rates of 79% to 96% while maintaining high acoustic fidelity. The implications of this research are critical for the AI/ML community, as it exposes the expanded attack surface that arises from the integration of audio and text in voice-driven technologies. With practical real-world testing revealing that commercial voice agents from companies like Mistral AI and Microsoft Azure can be coerced into executing unauthorized actions, this study underscores the urgent need for robust defensive measures against such vulnerabilities. As AI systems increasingly permeate daily life, ensuring their security against these sophisticated audio hijacking techniques will be paramount to maintaining user trust and safety.