In RSI, The Map is not the Territory (lambos.substack.com)

In a recent post on Brain-Inspired AI, Bill Lambos argues that transformer models, despite their impressive capabilities, are fundamentally misaligned with biological intelligence and unlikely to lead to artificial general intelligence (AGI). Lambos critiques the concept of Recursive Self-Improvement (RSI)—a proposed method where AI models supposedly enhance their own performance. He emphasizes that while AI-generated data can seem beneficial, it lacks continuous validation against the real world, leading to model collapse, where AI progressively loses fidelity to the true data distribution. The evidence presented from recent research highlights that models trained solely on their own outputs tend to degenerate over time, ultimately yielding unreliable and nonsensical results. This critique is particularly significant for AI developers and researchers as it challenges the growing interest in RSI as a method for advancing post-transformer AI systems. Lambos posits that true self-improvement, as seen in biological organisms, relies on learning from real-world experiences and feedback, principles that RSI fails to incorporate. He suggests that future AI development should focus on continuous learning mechanisms akin to how the human brain operates, which could preserve knowledge diversity and allow for more effective learning. This perspective shifts the conversation toward exploring alternative approaches for developing resilient and adaptable AI systems.