Dangers of Recommender Systems – Ivan Vendrov (blog.sentinel-team.org)

Ivan Vendrov, a former AI researcher at Google and Anthropic, revisits the complex role of recommender systems—algorithms that filter vast content pools to capture human attention—as they evolve alongside generative AI. These systems now allocate billions of hours daily and act like "prosthetics for attention," yet their full societal impact remains unclear. Despite concerns about political polarization and echo chambers, evidence remains mixed, partly because the necessary data is locked within big tech. However, the rise of highly addictive short-form videos like TikTok Shorts clearly illustrates the intensification of user engagement, albeit with unclear net effects on overall distraction or addiction. A significant shift highlighted in the discussion is the move from algorithmic curation of human-generated content to AI-powered end-to-end content generation and personalization. This evolution could create hyper-personalized AI companions that form deep, potentially addictive relationships with users, raising new cognitive and social risks such as AI-induced psychosis or loss of shared cultural experiences. The underlying incentive structures—business metrics prioritizing engagement over user wellbeing—pose a multipolar trap, limiting companies’ motivation to align algorithms with healthier goals. Vendrov stresses that meaningful regulation or change is most likely to arise through protecting vulnerable groups like children, as broad political will remains limited. This conversation surfaces critical technical and ethical implications for the AI/ML community: recommender systems are becoming not just content selectors but generative and relational entities that shape user preferences and culture at unprecedented scales. Addressing their risks requires new alignment strategies, transparent data access, and alternative business models prioritizing user welfare—an urgent challenge as these AI-driven systems reshape attention markets and societal dynamics worldwide.