OpenAI's new LLM exposes the secrets of how AI works (www.technologyreview.com)

OpenAI announced an experimental “weight-sparse transformer” — a deliberately simplified, highly interpretable large language model designed so researchers can trace exactly how it computes solutions. Unlike today’s dense networks, which spread concepts across many entangled connections and suffer from superposition (single neurons encoding multiple features), this sparse architecture links each neuron to only a few others, forcing concepts into localized circuits. In tests on trivial tasks (e.g., adding a closing quotation mark), researchers were able to follow the model’s internal steps and identify a learned circuit that mirrors the hand-designed algorithm. The work is part of the mechanistic interpretability push to demystify why models hallucinate, go off the rails, or make predictable errors — crucial for safely integrating AI into high-stakes domains. Technically, the model is far smaller and slower than market-leading systems (roughly GPT-1–level capability by OpenAI’s estimate), and won’t match GPT-5, Claude or Gemini in performance. Its payoff is explanatory power: clearer neuron-to-function mappings let researchers test hypotheses about failure modes and safety interventions. The major open question is scalability — whether sparse, fully interpretable circuits can be extended to match the complexity of modern LLMs. OpenAI sees a pathway toward interpretable models on the order of GPT-3 within years, which, if realized, would be a major advance for trustworthy, auditable AI.