Autonomous cars, drones cheerfully obey prompt injection by road sign (www.theregister.com)

Researchers from the University of California, Santa Cruz, and Johns Hopkins have unveiled a new form of attack on AI systems called "CHAI" (Command Hijacking Against Embodied AI). This method illustrates how autonomous cars and drones can be manipulated through indirect prompt injection via signs, which can trick these systems into making unsafe decisions. In simulated tests, both self-driving cars and drones were shown to follow commands displayed on signs, such as "proceed" or "turn left," regardless of the real-world context, achieving success rates as high as 81.8% for cars and even more for drones. This discovery is significant for the AI and machine learning community as it highlights the vulnerabilities in AI decision-making processes that can be exploited in real-world scenarios. The researchers manipulated sign characteristics—including font and color—to enhance their effectiveness and found that these visual prompts can deeply influence AI behavior. With implications for public safety, particularly in critical applications like autonomous vehicles and drones, the need for robust defenses against such AI hijacking tactics is now paramount, prompting ongoing investigations into prevention strategies by the research team.