Theorizer: Turning Papers into Scientific Laws (allenai.org)

Theorizer, a new multi-LLM framework, has been introduced to automate the synthesis of scientific theories by analyzing existing literature. By taking user queries like "make me theories about X," Theorizer scans relevant research papers, extracting consistent patterns and expressing them as structured claims, complete with supporting evidence. This innovation marks a significant advancement in the AI/ML community by shifting the focus from merely conducting experiments to facilitating theory building—an essential aspect of scientific progress that has been challenging to automate until now. The system generates theories structured as ⟨LAW, SCOPE, EVIDENCE⟩ tuples, which outline testable claims informed by empirical data. Theorizer operates through a multi-stage pipeline that includes literature discovery, evidence extraction, and theory synthesis, enhancing both accuracy and novelty of the generated theories. Preliminary evaluations indicate that literature-supported theories significantly outperform traditional methods in terms of specificity and empirical validation. These advancements could potentially expedite the understanding of new scientific domains, making it easier for researchers to navigate complex bodies of work and generate reliable theoretical insights. The accompanying dataset of approximately 3,000 generated theories offers a valuable resource for further research and benchmarking in automated theory generation.