Google researchers find the best AI model is 69% right (www.businessinsider.com)

Google DeepMind has unveiled the FACTS Benchmark Suite, a new tool designed to assess the factual accuracy of AI models across various domains. The benchmark tests models on four metrics: answering fact-based questions, utilizing web search effectively, grounding responses in lengthy documents, and interpreting images. The top-performing model, Google's Gemini 3 Pro, achieved an accuracy rate of just 69%, highlighting a significant gap in factual reliability when compared to human standards, particularly in fields requiring precise information like finance and law. This revelation is crucial for the AI/ML community as it underscores the current limitations of even advanced AI systems in delivering reliable information. With leading models demonstrating substantial inaccuracies, businesses relying on AI for critical decision-making may face serious risks, as evidenced by a recent incident where a law firm terminated an employee for submitting a document filled with inaccuracies generated by AI. The FACTS Benchmark not only serves as a cautionary indicator of the state of AI but also aims to guide future improvements by pinpointing specific weaknesses, emphasizing the need for continued refinement to enhance factual reliability in AI applications.