These LLMs are the best at resisting Russian propaganda (arstechnica.com)

The Estonian Language Institute (ELI) has introduced a groundbreaking "Propaganda Resistance" benchmark that evaluates large language models (LLMs) on their capacity to resist propagandistic narratives, particularly those propagated by the Russian Federation. As nations become increasingly concerned about foreign influence through AI-generated content, this initiative is significant for the AI/ML community, emphasizing the need for responsible AI that can discern and counteract disinformation. ELI collaborated with the volunteer-run defense group Propastop to identify 14 categories of Russian narratives, ranging from territorial disputes to historical justifications. The benchmark includes nuanced questions aimed at assessing models' abilities to reject these narratives consistently. Judged by an AI model aligned with expert opinion, LLMs like Anthropic’s Claude and its Sonnet and Opus versions excelled in performance, with Opus 4.7 achieving an outstanding score of 94.9 out of 100. This work not only highlights the critical role of AI in global information warfare but also provides a framework for developing LLMs that are more resistant to political manipulation and misinformation, setting a precedent for future research and applications in AI ethics and accountability.