AI chatbots give wrong answers to financial queries 'most of the time' (www.ft.com)

Recent research by tech firm Saturn highlights significant flaws in AI chatbots when addressing financial queries, revealing that popular models like ChatGPT and Claude delivered incorrect answers over half the time—averaging 57% mistakes. For complex inquiries that required multiple calculations, error rates soared to 88%, with some models failing nearly 99% of the time. The study tested over 10,000 responses from 18 AI models on various financial topics, uncovering critical issues such as calculation errors and misleading advice that could lead to serious financial repercussions for users. This research underscores a crucial warning for the AI/ML community: while chatbot technology shows promise for tasks like budgeting, reliance on these systems for financial advice poses significant risks, especially given the current regulatory void. With growing trust in AI among consumers—especially younger audiences—there's an urgent need for regulatory frameworks to ensure user safety and accountability. Currently, users lack the protections afforded to those who consult human advisors, making it essential for industry stakeholders and regulators to act swiftly to enhance consumer safety in the increasingly popular AI-driven financial advisory space.