BrowseComp-Plus: A More Fair and Transparent Benchmark of Deep-Research Agent (github.com)

BrowseComp-Plus has been introduced as a robust benchmark for evaluating Deep-Research systems, emphasizing fairness, transparency, and reproducibility. This benchmark is built upon OpenAI's BrowseComp but refines the evaluation process by utilizing a fixed set of approximately 100,000 human-verified documents instead of retrieving data from the live web. This controlled environment allows researchers to isolate and assess the performance of various retrievers paired with a consistent large language model (LLM) agent, thus facilitating more equitable comparisons among different Deep-Research agents. The significance of BrowseComp-Plus lies in its potential to enhance the reliability of results in the AI/ML research community by providing standardized evaluation protocols. It allows researchers to download obfuscated datasets and use pre-established scripts for experiments, making it easier to integrate custom retrievers and evaluate their effectiveness. The benchmark includes detailed assessment metrics that help researchers understand the retrieval success, accuracy, and relevance of the results produced by their Deep-Research agents, ultimately fostering more rigorous and comparable research outputs in the field.