Hackers influence ChatGPT and Gemini to direct users to scam centers (medium.com)

Recent reports reveal that hackers are conducting a widespread disinformation campaign targeting AI platforms like ChatGPT and Gemini, aiming to manipulate these systems into presenting false information that directs users to phishing scams. Through the strategic flooding of the internet with optimized content—such as fake reviews, support pages, and documents—malicious actors are able to influence AI responses. Major companies, including Delta, Chase, and Airbnb, have been affected, highlighting a significant shift in cyberattack strategies where AI, rather than just humans, becomes the primary target for social engineering. This development is crucial for the future of AI and machine learning, as it underscores the vulnerabilities inherent in how these systems curate information. With users increasingly trusting AI-generated responses—92% reportedly do not verify answers—the risks of manipulated content may lead to broader implications, where attackers effectively control the narratives presented by AI agents. The concept of Generative Engine Optimization (GEO), an evolution of SEO, is particularly concerning as it can be weaponized to ensure that false information gains traction within AI algorithms. This opens up new avenues for attacks, necessitating a reassessment of how we secure and validate AI content to safeguard against these emerging threats.