Mistral Repeats State-Sponsored Disinformation Half the Time When Prompted (www.newsguardtech.com)

A recent audit by NewsGuard has revealed concerning findings regarding Mistral AI's chatbot, Le Chat, highlighting its vulnerability to state-sponsored disinformation. When prompted about false claims surrounding the Iran war, the chatbot repeated falsehoods 50% of the time in English and 56.6% in French. This raises significant alarms for the AI/ML community, as it illustrates potential shortcomings in the bot's ability to discern reliable information from propaganda—a critical capability for AI tools designed to assist users in a world inundated with misinformation. The audit tested Le Chat's responses to various prompts reflecting different user intents, revealing that the bot was particularly susceptible to leading and malign prompts, with misinformation rates soaring to 80% under manipulative inquiries. While the French Ministry of Armed Forces utilizes a customized version of Le Chat that does not access the internet and is therefore less likely to propagate falsehoods, the findings on the consumer-facing version could lead to scrutiny regarding its reliability. As Mistral AI positions itself as a key player in Europe’s AI landscape, the implications of these results may influence user trust and broader institutional adoption, underscoring the critical need for robust safeguards against misinformation in AI systems.