Your bookmarks

Loading your bookmarks

Latest updates

New courses, skills, book chapters, and site improvements will appear here.

Create free accountSign upDashboardDashboard

Mode collapse

  • Failures
  • Standard term
2 steps

Mode collapse is when a generative model covers only a narrow slice of the variety it should produce. People also use the term loosely for AI output that keeps repeating the same patterns.

A model picks each next word from a spread of likely choices. Tuning it toward answers people rate highly tends to pile that likelihood onto a few safe-seeming choices. A low randomness setting, a fixed template, or a handful of examples narrows the spread further, so repetition alone doesn't say which cause is at work.

Where you’ll meet it

You ask an AI for twenty taglines for your team's new product. Most come back as “Your work, simplified” or “Meet the new way to plan,” and every one follows the same two-part rhythm.

What to do

Ask for a few taglines of each kind you want, such as a question, a promise, a joke, or a single word. If the repeats come back, start a fresh chat with no earlier taglines in it, because the model also copies patterns already on the page.

Easy to mix up

Model collapse affects later models trained on earlier models' text. Mode collapse shows up in the one model you are using, whose answers keep circling the same few patterns.

Check yourself

You ask for a name for your podcast in ten fresh chats and get ten different names, each a two-word pun on “pod.” How much variety did the model show?

Go deeperThe term began with image generators and loosened when applied to chatbots

It began with generative adversarial networks (GANs), image generators trained against a second network that tries to tell their pictures from real ones. A generator can find a few kinds of picture that pass and keep producing them, covering a few peaks, or modes, of the data and missing the rest. There it is a measurable training failure.

The chatbot sense is harder to measure, because each answer read alone can look fine. Studies draw many answers to the same request and count how many distinct openings, structures, or ideas appear. Counting ideas takes a judgment about when two answers say the same thing in different words, so results depend on who or what makes that call.

Practice Modern AI

Or take the learning quiz on a mix of every set