These AI Experts Want to Do High-Stakes Research Out in the Open (www.wired.com)

AI researchers Nathan Lambert and Tom Zick have launched Trillium Labs, a nonprofit dedicated to conducting high-stakes AI research with an emphasis on transparency. Unlike many major AI companies that restrict access to powerful models to ensure safety, Lambert and Zick advocate for publishing detailed experimental results, allowing external scientists to replicate findings and contribute to critical discussions about AI risks. They argue that the current closed model of frontier AI development stifles community scrutiny and innovation, which are essential to advancing understanding and mitigating potential dangers, especially concerning complex areas like recursive self-improvement (RSI) and reinforcement learning. Trillium Labs will initially focus on improving large models post-training while exploring how reinforcement learning affects AI behavior. This method has heightened model capabilities but also risks unexpected outcomes, raising concerns among researchers. With initial funding and a goal of securing $40 to $100 million, Lambert and Zick aim to foster a more nuanced dialogue in the AI community. They believe that embracing the scientific method amid the current polarized views on AI development is key to responsibly advancing the technology.