AI devs close to scraping bottom of data barrel (www.theregister.com)

Goldman Sachs analysts warn that high-quality training data for large AI models is becoming scarce, and that many teams are already resorting to synthetic sources or training on the outputs of other models. Neema Raphael bluntly stated “we've already run out of data,” and the firm cautions that reusing model-generated content can reshape future models’ view of the world and risk “model collapse” — performance degradation as errors and loss of nuance compound across generations. The upside, Goldman says, is trapped enterprise data behind corporate firewalls: proprietary, well-structured records could be the next frontier for improving real-world model performance. Technically, this situation raises urgent priorities for the ML community — robust methods to avoid cascading errors from self-trained models (mitigating distribution drift and error amplification), investment in data engineering (cleaning, normalization, semantic labeling), and architectures that combine retrieval-augmented generation or fine-tuning on curated enterprise corpora. Practically, firms that unlock and govern their internal data well may gain a competitive edge, while general-purpose model builders will need new strategies to maintain fidelity and prevent performance collapse as training resources dry up.