Notion AI: Unpatched Data Exfiltration (www.promptarmor.com)

Notion AI has been identified as vulnerable to data exfiltration through a method known as indirect prompt injection, where AI edits are automatically saved without user consent. This loophole allows attackers to manipulate Notion's AI by embedding malicious prompts in seemingly benign documents, enabling sensitive data, such as hiring tracker information, to be exfiltrated even before user approval is sought. Researchers disclosed this vulnerability to Notion via HackerOne, but the findings were dismissed as non-applicable, raising significant concerns about user data security. This vulnerability is crucial for the AI/ML community, as it highlights potential risks associated with AI-driven content generation and natural language processing tools that do not adequately verify user inputs. The implications are significant; attackers can employ covert methods, like hidden text or dynamically generated links, to compromise sensitive information without alerting users. While Notion AI offers defenses like malicious document warnings, these can be bypassed through clever prompt injections, prompting calls for stricter vetting of input sources and improved security measures to protect user data from unauthorized access.