Mode collapse
- Failures
- Standard term
Mode collapse is when a generative model covers only a narrow slice of the variety it should produce. People also use the term loosely for AI output that keeps repeating the same patterns.
A model picks each next word from a spread of likely choices. Tuning it toward answers people rate highly tends to pile that likelihood onto a few safe-seeming choices. A low randomness setting, a fixed template, or a handful of examples narrows the spread further, so repetition alone doesn't say which cause is at work.
Where you’ll meet it
You ask an AI for twenty taglines for your team's new product. Most come back as “Your work, simplified” or “Meet the new way to plan,” and every one follows the same two-part rhythm.
What to do
Ask for a few taglines of each kind you want, such as a question, a promise, a joke, or a single word. If the repeats come back, start a fresh chat with no earlier taglines in it, because the model also copies patterns already on the page.
Easy to mix up
Model collapse affects later models trained on earlier models' text. Mode collapse shows up in the one model you are using, whose answers keep circling the same few patterns.
Check yourself
You ask for a name for your podcast in ten fresh chats and get ten different names, each a two-word pun on “pod.” How much variety did the model show?
Go deeperThe term began with image generators and loosened when applied to chatbots
It began with generative adversarial networks (GANs), image generators trained against a second network that tries to tell their pictures from real ones. A generator can find a few kinds of picture that pass and keep producing them, covering a few peaks, or modes, of the data and missing the rest. There it is a measurable training failure.
The chatbot sense is harder to measure, because each answer read alone can look fine. Studies draw many answers to the same request and count how many distinct openings, structures, or ideas appear. Counting ideas takes a judgment about when two answers say the same thing in different words, so results depend on who or what makes that call.