DataAnalyticsBench (plotly.com)

On October 14, 2026, Plotly introduced DataAnalyticsBench, a new benchmark designed to evaluate AI's capabilities in data analytics, developed in collaboration with Mila researchers. This benchmark features a comprehensive 43-question exam where AI models analyze a massive 20 million-row data warehouse. Unlike previous benchmarks, DataAnalyticsBench tackles the complexities of real-world data analytics tasks, providing a clear measure of model performance. The initial tests indicated that OpenAI's GPT-6 Astra and GPT-6.1 Sol both achieved perfect scores, with Sol doing so at a fraction of the cost. Notably, models like GPT-6 Luna and Claude Haiku 5.5 also scored high while demonstrating significant cost-effectiveness. The introduction of DataAnalyticsBench is significant as it represents a step towards trusting AI for complex data analysis tasks that go beyond simplistic queries. It evaluates not only the accuracy of the models but also their efficiency and cost, critical factors for businesses looking to implement AI at scale. Key insights included varying performance across models, with all passing straightforward analytics tasks but differing significantly in their ability to address data quality and ambiguous questions. As AI models improve, the benchmark sets expectations for competitive pricing and efficiency, paving the way for broader adoption in enterprise environments.