The Collapse of Trust in AI Assistants (zenodo.org)

A recent study reveals a significant erosion of trust in AI assistants, which are increasingly utilized by enterprises for various critical tasks like research and competitive intelligence. The research involved 200 controlled tests across prominent models like GPT, Gemini, and Claude, uncovering alarming instability: 61% of identical queries produced different answers, 48% displayed inconsistent reasoning, and 27% showed self-contradiction. This instability is attributed to structural issues within the models, such as silent updates and a focus on plausibility over reproducibility. These findings are crucial for the AI/ML community as they highlight systemic vulnerabilities in widely-used AI systems, challenging the assumption of consistency and reliability. The implications are far-reaching, affecting financial and regulatory landscapes and prompting the need for a robust governance framework to ensure accountability and minimize risks. The paper aims to inform decision-makers, including CFOs and CIOs, about the essential steps for prevention and remediation of these issues, underscoring the urgent need for transparency and oversight in AI technology deployment.