Instinct's powerful AI assistant is raising privacy and security concerns (techcrunch.com)

Instinct, a new AI personal assistant currently in private testing, has garnered attention for both its groundbreaking capabilities and concerning privacy implications. Designed to integrate seamlessly with users' applications and devices, Instinct can manage tasks such as booking appointments and organizing inboxes through intuitive text and voice commands. However, skepticism has emerged regarding its extensive terms of service, which allow the company broad access to users' data, including the ability to modify user materials and even act on their behalf in a binding manner. This raises critical questions about the trade-offs involved in using hyper-personalized AI tools. The AI community is particularly concerned about Instinct's security model, which testers have flagged for vulnerabilities. For instance, the assistant reportedly retained access to users' email data even after being disconnected, and some users found the bot prematurely executing tasks without prior consent. This situation highlights the tension between the allure of advanced AI functionalities and the essential need for privacy and control. Investors appear undeterred; Instinct recently announced plans for a $250 million Series B funding round at a $2.5 billion valuation, signaling significant interest in the technology amidst ongoing debates about the ethical implications of AI agents.