Defeating Prompt Injections by Design (arxiv.org)

Researchers from Google and ETH Zurich have introduced CaMeL (CApabilities for MachinELearning), a robust defense mechanism designed to protect Large Language Models (LLMs) from prompt injection attacks. As LLMs increasingly interact with untrusted environments, they are vulnerable to adversaries who can manipulate input data to exfiltrate sensitive information or alter their behavior. CaMeL addresses this issue by creating a protective layer that processes user queries through a custom Python interpreter, explicitly extracting control and data flows, which prevents malicious inputs from impacting the system's program flow. This innovation is significant because it moves beyond existing defenses—often relying on modifying LLM behaviors or training to recognize harmful prompts—by incorporating traditional software security concepts like Control Flow Integrity and Information Flow Control. CaMeL effectively blocks unauthorized data flows using capabilities that enforce strict security policies, ensuring that even if part of the system is compromised, the overall security remains intact. In tests, CaMeL demonstrated the ability to solve 77% of tasks securely, a notable improvement in safeguarding against prompt injections, thus enhancing the reliability of LLMs in various applications. The project is now available on GitHub, encouraging further exploration and adoption within the AI/ML community.