On the Limits of Self-Improving in Large Language Models: The Singularity Is Not Near Without Symbolic Model Synthesis (arxiv.org)

Recent research has challenged the notion of recursive self-improvement in Large Language Models (LLMs) and the anticipated AI Singularity, arguing that reliance on self-generated data can lead to model collapse. The study mathematically formalizes this process as a discrete-time dynamical system, revealing two key failure modes: Entropy Decay and Variance Amplification. These phenomena arise when the external, real-world data used to ground models diminishes, causing performance degradation instead of enhancement. This finding highlights the limitations of mainstream AGI narratives that assume LLMs can autonomously improve over time without external grounding. To address these challenges, the authors propose a neurosymbolic approach that integrates algorithmic probability and program synthesis, enabling LLMs to generate synthetic knowledge beyond mere data interpolation. The Coding Theorem Method (CTM) is suggested as a mechanism to identify generative processes rather than just correlations, distinguishing it from traditional statistical learning methods. This shift not only redefines how we understand the capabilities of current generative AI but also emphasizes the necessity of incorporating structured reasoning into model design to prevent degenerative dynamics and drive meaningful advancements toward true artificial general intelligence.