Astra Held Its Ground (Truth) (marginalrevolution.com)

A new suite of 50 complex theoretical physics problems has been introduced, challenging off-the-shelf AI agents to tackle them without human assistance. Among the tested models, GPT-6-Astra, integrated into Codex, achieved unprecedented success with a near-perfect accuracy of 98%, addressing problems solvable by leading experts in a fraction of the time and at an average cost of under $3 per question. One of the standout problems involved constructing a Hamiltonian for a fermion parity pump, illustrating the intricate nature of the tasks that AI can now manage. This development is significant for the AI/ML community as it marks a potential shift in how we validate expert knowledge. In a surprising turn, the AI's responses initially seemed to conflict with human expert answers, revealing subtle errors in the latter that prompted revisions and ultimately led to the AI being deemed a more accurate source of truth. This event highlights the evolving role of AI not just as a tool for assistance but as a key player in theoretical understanding, raising profound questions about the future of research and expertise in physics and beyond.