Your bookmarks

Loading your bookmarks

Latest updates

New courses, skills, book chapters, and site improvements will appear here.

Create free accountSign upDashboardDashboard

Model collapse

  • Failures
  • Research term
3 steps

Model collapse is a training failure in which models trained, generation after generation, on AI-generated data lose parts of what the original data covered and amplify errors.

Every model is an imperfect copy of its training data. It tends to overweight common patterns and underweight unusual ones, and any sample of its output can miss an uncommon case. A model trained on that sample can't learn what isn't there, and it adds slips of its own for the next model to inherit.

Where you’ll meet it

In a team meeting about which AI tools to adopt, a coworker says the web is filling up with AI-written text, so each new chatbot trained on it will be dumber than the last. They call it model collapse and suggest the team not plan around these tools for long.

What to do

Treat it as a documented risk stretched into a forecast. Ask whether the experiments behind the term were set up the way companies gather and filter training data. Then ask whether newer chatbots have scored lower on tests than older ones, as the forecast predicts.

Easy to mix up

Mode collapse narrows the answers one model gives. Model collapse narrows the data that later models learn from, as each generation trains on the last one's output.

Check yourself

Two teams train a new model each year. One replaces last year's data with the old model's output. The other adds the output on top of all the human-written data it already has. Which plan is more likely to collapse?

Go deeperExperiments show the failure clearly, and checked synthetic data can improve models

Ilia Shumailov and colleagues tested the idea in a 2024 Nature paper, “AI models collapse when trained on recursively generated data.” They fine-tuned a small language model on text written by the previous round's model. By the ninth round, a prompt about medieval church architecture drew text that wandered into a list of jackrabbits with differently colored tails.

Synthetic data, examples a model generates for training, also improves models when something checks it, such as a filter that drops weak examples or a program that confirms math answers. Checking adds information, while unchecked output mostly repeats what the model already does. Careful teams record which model made each example and keep a human-made test set out of training.

Practice Modern AI

Or take the learning quiz on a mix of every set