How well does AI predict humans? (simulacrabench.org)

The NeurIPS 2026 competition, SimulacraBench, has been announced to assess AI's effectiveness in predicting human behaviors among hard-to-reach populations using unreleased microdata from UNHCR, UNICEF, and the World Bank. Scheduled for public launch on August 18, 2026, and closing for submissions on November 13, 2026, this initiative invites participants to develop algorithms that could potentially augment traditional survey methodologies, which often face challenges like high costs and data gaps. With approximately 40,000 respondents from 14 countries, the competition aims to explore whether AI can achieve classical survey accuracy at a fraction of the time and cost. This benchmark could signify a substantial advancement for the AI/ML community, particularly in humanitarian efforts, where reliable data is crucial for policy-making and program development. Participants will operate under a closed evaluation framework, submitting code instead of predictions, with the goal of predicting responses for held-out data without access to the actual microdata. With its calibration-first approach and open-source starter kit, SimulacraBench not only fosters innovation in prediction models but also raises ethical concerns regarding data usage, emphasizing the need for responsible AI practices in sensitive contexts.