The Ainex Limit: Geometric Proof of LLM Collapse via Recursive Loops (github.com)

A new study has highlighted alarming findings regarding the degradation of language generation in large language models (LLMs), specifically the GPT-2 Small, after 20 iterations of a recursive synthetic dataset. Researchers discovered a significant 66.86% loss in semantic integrity by Generation 20, rendering the model functionally "brain-dead" with persistent hallucinations accepted as truth. This stark decline is characterized by a two-phase collapse: an initial 85% implosion in variance, followed by a gradual drift into a "logic-less topology," where the model generates increasingly nonsensical outputs. Significantly, this study introduces a novel metric, the Ainex Integrity Score ($\mathcal{A}$), to assess semantic reality, contrasting with traditional perplexity measurements that focus on confusion rather than meaning. By employing geometric metrics based on the convex hull of the embedding space, the researchers have provided new insight into the concept of model creativity and hallucinations. The results underscore the critical challenges in LLM development, especially concerning their ability to maintain semantic coherence in recursive learning environments, raising important questions about the future safety and reliability of AI systems. The full repository is available for others in the community to reproduce and explore these findings further.