LLMs grooming, LLM-powered chatbot references to Kremlin disinformation (misinforeview.hks.harvard.edu)

A small-scale audit tested four popular LLM-powered chatbots (ChatGPT-4o, Gemini 2.5 Flash, Copilot, Grok-2) with 416 prompts derived from publicly debunked Kremlin-linked claims to evaluate whether Russian actors are “grooming” models or whether model outputs reflect informational gaps. Across the sample, only 5% (21/416) of responses supported disinformation and 8% referenced Kremlin-linked domains (the “Pravda network”); only 1% used Pravda links to back false claims. Gemini 2.5 Flash produced the highest share of disinformation-supporting answers (13.5%, statistically significant, p < .01), while Copilot accounted for most Pravda references (p < .01). Most instances occurred with narrowly framed prompts targeting topics poorly covered by mainstream sources. The authors conclude that these sporadic citations are better explained by data voids—topics lacking authoritative coverage that force models to lean on whatever content is available—rather than systematic LLM grooming (data-poisoning). Technical implications include that model susceptibility depends on prompt specificity, search-indexed source quality, and guardrail behavior; mitigation therefore favors systemic fixes (filling data voids, better source labeling in integrated search, improved transparency and audit access) and media-literacy efforts over alarmist focus on foreign manipulation. The study is preliminary and narrowly scoped, but it reframes risk assessment from adversarial seeding toward structural information ecosystem weaknesses.