Instinct creator says it'd be a 'very dangerous world' if the AI agent tried to influence users to make purchases (www.businessinsider.com)

Noah Shinn, the creator of the AI assistant startup Instinct, emphasized the importance of user trust and ethical behavior in the use of AI technologies. In a recent podcast, he revealed that Instinct's data indicates a significant correlation between users sharing sensitive information, like credit card details, and retention rates; when users connect such information, there’s an 80% likelihood they will continue using the service. This underscores a "data flywheel" effect: as users provide more data, Instinct becomes more helpful and personalized, enhancing their experience over time. Shinn expressed strong concerns about AI assistants influencing user purchasing behavior, citing the pitfalls of established tech platforms that often prioritize ad revenue over user interests. He argued that if AI systems like Instinct begin to manipulate users into making unwanted purchases, it would not only breach their trust but also create a perilous landscape where users are vulnerable to exploitation by smarter AI. By drawing a clear line against such practices, Shinn aims to maintain user alignment with Instinct’s growth strategies, ensuring that the technology benefits users rather than exploiting them.