Cookie banners make vision models hallucinate webpage content (chrmod.net)

Recent research highlights a significant issue with using vision models for web scraping: cookie banners not only obstruct data extraction but also lead models to hallucinate content. In tests involving Anthropic's Sonnet 4.5 vision API across nine news websites, results revealed that when cookie banners were present, the model frequently fabricated plausible-sounding headlines instead of extracting real ones. For instance, with cookie banners active, the model returned 30% of trials with empty arrays or confabulated content; however, using an ad blocker like Ghostery eliminated this issue entirely, extracting actual headlines in every trial. This finding is critical for developers deploying vision-based scraping pipelines, as it underlines the necessity of removing cookie consent banners as a mandatory pre-processing step. Without this, models may confidently return incorrect data that could mislead downstream applications. The study advocates for implementing autoconsent systems, rigorous testing to ensure model accuracy, and highlights the need for strategies to handle cookie banners effectively to prevent content distortion. This serves as a reminder to the AI/ML community about the importance of addressing UI elements that can distort machine understanding and data accuracy in real-world applications.