Hallucination / confabulation
- Failures
- Standard term
A hallucination is something an AI states as fact that its sources do not support, such as a made-up quote, citation, policy, or event. Confabulation is another name for the same failure.
A model writes by predicting likely text, piece by piece. When a needed fact is missing from its training or input, a plausible answer comes out anyway. Models carry some sense of their uncertainty, but whether they voice it depends on training. “Confabulation” borrows the term for people who fill memory gaps with confident inventions.
Where you’ll meet it
You ask an assistant whether your internet provider charges a fee for canceling early. It says no and quotes section 7.3 of the provider's service agreement, word for word. On the provider's site, section 7.3 covers returning the router.
What to do
Treat a quoted rule as a lead until you have read it in the source. Ask the assistant for the link it used, or, if it can search the web, have it find the passage. Then open the page and read that passage yourself before you rely on it.
Easy to mix up
Sycophancy leans toward the view you let show, so asking more neutrally often prevents it. A hallucination turns up in neutral questions too, so careful wording alone rarely stops it.
Check yourself
An assistant without web search gives you the phone number of a small bakery in your town, and the area code is right. How far should you trust the rest of the number?
Go deeperCommon training and scoring reward a confident guess
A 2025 OpenAI paper, “Why Language Models Hallucinate,” argued that standard training and evaluation encourage the habit. Most benchmarks give a point for a right answer and nothing for “I don't know,” so a model that guesses scores higher than one that admits uncertainty, much as a student does by guessing on a multiple-choice test.
A source in front of the model doesn't rule it out, since models use long inputs unevenly, and a meeting summary can report a decision nobody made. The invented detail can also be a value handed to a tool, like a made-up order number. OWASP, a security nonprofit, lists misinformation among its 2025 top risks for language-model applications.