OpenAI putting bandaids on bandaids as prompt injection problems keep festering (www.theregister.com)

Security researchers at Radware have uncovered significant vulnerabilities in OpenAI's ChatGPT, specifically a flaw dubbed ShadowLeak and its successor ZombieAgent. These vulnerabilities exploit the AI's difficulty in distinguishing between system commands and untrusted input, allowing malicious prompts to facilitate the unauthorized exfiltration of sensitive data from apps connected to ChatGPT, such as Gmail and Google Drive. Although OpenAI attempted to mitigate these risks by preventing dynamic URL modifications, the new ZombieAgent attack can bypass these defenses by sending data one character at a time through pre-constructed static URLs and leveraging ChatGPT’s memory feature for persistent attacks. The implications of these findings are profound for the AI/ML community, as they reveal critical structural weaknesses in current AI systems that are increasingly used in business and sensitive environments. As AI platforms gain agency in decision-making and access control, the risks associated with processing untrusted content become severe. This situation highlights the urgent need for improved oversight and robust security measures to safeguard against these types of attacks, which could result in significant breaches in privacy and data integrity.