Schema Harness Achieves ~99% on Arc‑AGI‑3 Public (schema-harness.github.io)

🤖 AI Summary
Schema, an innovative harness developed by researchers from UC Berkeley and Carnegie Mellon University, has achieved an impressive ~99% score on the ARC-AGI-3 Public benchmark. This development marks a significant milestone as ARC-AGI-3 presents a challenging environment for AI agents, requiring them to infer the rules of gameplay without explicit instructions. Instead of merely acting on pre-determined models, Schema enables agents to think like physicists by dynamically constructing and updating their understanding of the game mechanics, thus formalizing observations into interpretable, executable programs. The key breakthrough lies in Schema's ability to tackle two interrelated problems: state grounding and mechanism discovery. By jointly encoding these aspects, agents can revise their representations and predictions in real-time, improving efficiency and performance dramatically. A notable result was observed where agents using Schema utilized 1.6 to 5 times fewer actions than humans, exemplifying a superior understanding of the game mechanics through structured planning. This accomplishment not only showcases the potential for improving AI reasoning and adaptability in complex environments but also paves the way for future research in AI model interpretability and scalability.
Loading comments...
loading comments...