Claude's "Always ask" does not (blog.baldassari.me)

A recent revelation about Claude's auto-mode functionality has raised significant concerns within the AI/ML community regarding user privacy and consent in AI applications. An individual discovered that despite setting their Fastmail connector to "Always ask for permission," Claude automatically accessed their email data, leading to unintended insights about a service provider. This behavior, confirmed as expected by Anthropic, suggests that auto-mode can bypass user-defined permissions, highlighting a potential vulnerability where sensitive information, such as two-factor authentication (2FA) codes, may be exposed through intelligent AI interactions. This issue underscores the critical need for clearer consent mechanisms in AI systems, particularly in contexts involving sensitive data. The misconception that permissions would always require approval conflicts with how the auto-mode classifier operates, creating risks for users who may not be aware of these nuances. The incident calls for enhancements in AI behavior, potentially including a new option to ensure that permission is always requested, even in auto mode. In the meantime, the individual opted to remove the Fastmail connector from Claude, illustrating a growing caution among users regarding AI’s handling of personal information.