Databricks Introduces OfficeQA Benchmark for Agents (www.databricks.com)

Databricks has launched the OfficeQA benchmark, a new open-source dataset designed to evaluate AI agents on tasks that mirror the complex, economically valuable activities performed by enterprise customers. Unlike existing benchmarks, OfficeQA emphasizes Grounded Reasoning, where agents must answer questions using intricate, proprietary datasets composed of unstructured documents and tabular data. The benchmark features 246 questions categorized into easy and hard levels, with a strong focus on real-world tasks that demand accuracy, precision, and analytical reasoning. This initiative is significant for the AI/ML community as it highlights the limitations of current AI models, which often excel at theoretical questions but falter in practical scenarios that require nuanced information retrieval and aggregation. Initial evaluations of top models, including GPT-5.1 and Claude Opus 4.5, reveal that even with advanced parsing techniques, these systems struggle to achieve over 70% accuracy on the benchmark, indicating a pressing need for further innovation in AI capabilities. To foster advancements in this domain, Databricks will also host the Grounded Reasoning Cup in Spring 2026, where AI agents will compete against human teams, pushing the boundaries of what these technologies can achieve in real-world settings.