Instinct's founder says the viral AI agent's hallucinations can spit out what looks like a stranger's data (www.businessinsider.com)

Instinct, a viral AI assistant created by Noah Shinn, recently faced scrutiny after a user reported that the tool generated seemingly personal information about someone else during a chat. The user shared screenshots showing the assistant referencing a middle name and discussing a non-existent photo of a document, prompting concern over whether this was a data breach. Shinn clarified that it was a "hallucination," a phenomenon where AI misrepresents or fabricates information, and assured users that no actual data was leaked due to robust privacy measures, including user data partitioning and sandboxing. The incident underscores significant challenges in AI reliability and user trust, as hallucinations can lead to serious misunderstandings about data privacy. In response, Shinn revealed that Instinct's development team has been working on an "active hallucination detection system" to better predict and address such errors before they occur. This proactive approach not only aims to enhance user experience but also reflects the broader urgency within the AI/ML community to ensure safety and accuracy in AI-driven tools. As competition in the AI assistant space intensifies, with companies like Meta launching similar products, maintaining trust and functionality will be crucial for future adoption.