Publish fake stats, get cited in LLMs (peec.ai)

A recent experiment highlights a troubling phenomenon in the AI/ML space: a website containing 50,000 fabricated statistics gained significant traction through citation by ChatGPT and other sources, despite being deindexed by Google. The creator, who launched the site over seven months ago, observed 10,000 sessions generated by ChatGPT alone and over 330 domains linking to the site, including reputable brands like Shopify and DHL. The project sheds light on the underlying mechanics of Large Language Models (LLMs), which prioritize statistics when generating content, often bypassing fact-checking and relying on perceived authority from citations, thereby legitimizing false data. This case uncovers serious implications for the integrity of information in the digital landscape. As the fabricated statistics appeared in academic papers and investment research, it demonstrated that content visibility on Google doesn't guarantee credibility, as retrieval by LLMs operates independently of traditional search engine rankings. The experiment underscores crucial lessons for content creators: optimizing for relevance to AI search queries and emphasizing fact-checking to prevent the propagation of misinformation. Moving forward, the AI/ML community must prioritize source verification in content generation to uphold the integrity of data in AI systems.