BEAVER: Enterprise benchmark for LLM Text-to-SQL from private data warehouses (beaverbench.github.io)

The BEAVER benchmark, recently introduced in a paper authored by a team of researchers, presents a large-scale enterprise dataset focused on text-to-SQL tasks derived from private data warehouses. This comprehensive benchmark consists of 9,128 queries covering 812 tables across 19 distinct domains, with 7,978 queries made publicly available to facilitate research and development. Additionally, it incorporates detailed annotations for questions, SQL, and subtasks, enhancing the tools available for evaluating and analyzing SQL generation capabilities. The significance of BEAVER lies in its ability to support advancements in understanding how language models can interpret and transform natural language questions into SQL queries, particularly in real-world enterprise contexts. This dataset not only challenges existing models but also encourages innovation in natural language processing and machine learning techniques applicable to data retrieval and management. As organizations increasingly rely on data-driven decision-making, tools like BEAVER are crucial for assessing and improving the accuracy and efficiency of AI systems built to interact with complex databases.