The Curse of Recursion: Training on Generated Data Makes Models Forget (2024) (arxiv.org)

Recent research highlights a significant concern in the field of AI/ML, termed "Model Collapse," where training models using generated data leads to the irreversible loss of key features from the original content distribution. As large language models (LLMs) like GPT and image generators such as Stable Diffusion proliferate, the introduction of model-generated content into training datasets results in notable degradation of model performance. This occurs not only in language models but also in other generative frameworks, including Variational Autoencoders and Gaussian Mixture Models, raising alarm bells about the sustainability of benefits derived from large-scale online training data. The implications of this phenomenon are profound for the AI community, suggesting that reliance on self-generated text may compromise the richness and variability essential for effective model training. As the ecosystem increasingly utilizes output from LLMs, the originality and authenticity of training data sourced from genuine human interaction will become increasingly critical. This research underscores the need for careful curation of training datasets to mitigate the effects of recursive training and ensure the continued effectiveness of generative models, setting the stage for a reevaluation of best practices in data collection and model development.