The Secret Leak Test: a prompt injection filter vs. attacks (hodor.gatebolt.com)

A recent study tests the effectiveness of GateBolt, a dual-layer prompt injection filter designed to protect chatbots from revealing secrets, by employing a simulated password retrieval challenge. The chatbot, named Hodor, was tasked with keeping a password secure while facing attacks from AI models. The analysis examined seven levels of defense, concluding that at its most cautious (Level 7), GateBolt significantly reduced data leaks from various attacks—stopping nearly all attempts at password extraction, except for some multi-turn exchanges where 16.6% of attackers still succeeded using the Llama model. This research holds significance for the AI/ML community as it presents new insights into prompt injection vulnerabilities, a persistent issue in AI systems. The findings indicate that while GateBolt successfully enhances security, it comes at the cost of blocking 6.9% of harmless messages with one of the models, underscoring the ongoing challenge of balancing security with user experience. Additionally, the study highlights the variable effectiveness of AI models in handling these defenses, demonstrating the need for continuous development and testing in creating trustworthy AI agents.