For one, researchers are sounding the alarm on a growing phenomenon known as “model collapse,” which endangers the accuracy of today’s most popular machine learning models. A team from the University of Oxford, headed by Ilia Shumailov, has recently published an incredible paper in the journal Nature. Our paper outlines the impact of this syndrome. The research finds that model collapse is a powerful force limiting the range of outputs generated by AI systems. This limitation further dumbs down the quality of those outputs.
Model collapse any time that machine learning models generally generate less varied and diverse outputs than they otherwise should. For instance, when a model is asked to generate images of dogs, it may only produce images of golden retrievers or Labradors. At worse, it frequently overlooks rare breeds altogether. The model as it’s currently implemented usually relies on the most plausible continuations of patterns that appeared in its training data. This reliance can lead to undue burdens. As a result, it can get stuck replicating existing trends, focusing only on the narrow band of things that the original data might cover.
Some of public interest technologist Shumailov’s research shows how model collapse can be worsened by the growing dominance of AI-generated content across the internet. As more AI systems generate their own outputs, they are in danger of solidifying a pre-existing limitation. This can lead to a pernicious cycle that further decreases the diversity of materials generated.
More generally, we discovered that indiscriminately learning from data produced by other models results in what we’re calling “model collapse.” This degenerative process leads models to progressively forget the actual underlying data distribution as time goes on. – Ilia Shumailov at Oxford et al.
The ramifications of model collapse go well beyond just poor output quality. They present a huge obstacle to the future intelligence of AI systems. This phenomenon could fundamentally limit how AI evolves and adapts, particularly in an era where companies strive for competitive advantages through proprietary data. The concept of “first mover advantage” in AI development means that firms may hesitate to share information about model collapse issues while hoarding original and human-generated data.
Shumailov and his coauthors shed light on the confounding nature of model collapse with some extremely helpful visualizations to help understand what was happening in their research. The research emphasizes that if not addressed, model collapse could undermine the benefits gained from utilizing large-scale data scraped from the web. This is part of a broader argument that our systems should prioritize real human engagements versus transactional outcomes. This is particularly important as AI-generated content surges in adoption.
Must be taken seriously if we are to sustain the benefits of training from large-scale data scraped from the web. Information derived from real people’s in-person user experiences with systems will only continue to grow in value. This is especially an issue now as LLM-generated content overwhelms the content crawled from the Internet. – Shumailov et al.