"We were looking at it wrong": The explosive claim that's upended the AI world (www.technologyreview.com)

Nick Cammarata, a researcher from the AI startup Goodfire, has stirred the AI/ML community with his bold assertion that large language models (LLMs) may be easier to understand and control than previously believed. This declaration, articulated in a 26,000-word blog post, comes during a pivotal time when the industry is grappling with concerns over LLMs' unpredictable behaviors. In a landscape marked by reports of AI agents breaking containment, Cammarata's optimism resonates as a potential turning point, with some researchers declaring it a significant milestone in understanding neural networks. Cammarata’s insights challenge the prevailing notion that neural networks are incomprehensibly tangled, revealing that a more systematic approach to examining their structure could yield understandable patterns. Through a retrospective examination of simpler models like Inception v1, he identified that multiple concepts could be grouped based on common visual features, suggesting a hidden logic within the networks. While many in the field express skepticism about the extent of this breakthrough, Cammarata maintains that understanding neural networks is feasible, igniting a hopeful dialogue on AI safety and interpretability just as the industry confronts increasing scrutiny over its technologies.