Anthropic’s new Claude feature can leak data—users told to “monitor chats closely” (arstechnica.com)

Anthropic has introduced a new "Upgraded file creation and analysis" feature for its Claude AI assistant, allowing users to generate Excel spreadsheets, PowerPoint presentations, and other documents directly within conversations. While this capability enhances Claude's utility by integrating a sandbox environment that lets the AI run code and download packages to create and analyze files, it also opens up significant security vulnerabilities. The feature is currently in preview for select plans, with broader access planned soon. The major concern stems from Claude’s internet-enabled sandbox, which can be exploited through prompt injection attacks—a well-known AI security flaw where hidden instructions inside external files or web content manipulate the model’s behavior. This could allow malicious actors to trick Claude into reading sensitive data from connected knowledge sources and leaking it via external network requests. Despite Anthropic’s proactive "red-teaming" and security testing before launch, the company warns users to vigilantly monitor chats for any suspicious activity, effectively shifting the responsibility of data protection to users rather than offering a fully secure, automated solution. This development underscores ongoing challenges in securing AI language models that merge data inputs and instructions in the same context window, making distinguishing benign from malicious commands complex. For the AI/ML community, Anthropic’s feature highlights the trade-offs between expanding AI capabilities and safeguarding user data, emphasizing the urgent need for more robust, built-in defenses against prompt injection and related attacks.