Why does ChatGPT make things up instead of saying I don't know?
By Christophe Paka · Facts checked on 23 October 2026
You ask a chat AI for a court case that backs your argument, or the author of an obscure paper. A neat, confident answer arrives. You had even told it to admit when it does not know. The case does not exist.
The error is not the worst part. The voice is: the wrong answer arrives in the same calm prose as the right ones, and it can end up in a customer letter. So why does ChatGPT make things up? The figure below puts the same student in two exam rooms, and shows the one change that turns a bluff into an answer you can check.
How to read this figure
Read the closed book room. A plain model answers from memory, its training. No sources are allowed, and it was trained to fill every line. On most questions its memory serves it well.
Find where the bluff happens. When memory runs out, it writes a plausible guess in perfect prose. Bluffed lines look like known ones, and nothing flags the guesses. That bluffed line is a hallucination: a confident, fluent, wrong answer.
Cross to the open book room. Same student, but a grounded model now has trusted documents on its desk. Grounded means it answers from sources you gave it, not only from memory.
Read what changes. Answers point to a page. It may say not in the book. The grader can check claims, and when the book is silent the model can say so and cite what it found.
Note where the picture breaks. A student knows when she is unsure. A model's confidence does not track whether it is right, which is why the room, not the tone, is what you can change.
Key takeaways
- Check niche facts, exact numbers, citations, links and recent events at their source, whatever the tone.
- Add if you are not sure, say I don't know to your prompts; it lowers the odds of a bluff.
- Paste the source document when a fact must be right, so the model sits the open book exam.
Try this: catch a hallucination on purpose
Use any chat AI and a niche topic you know cold, such as your hometown or a hobby. Nothing here is for real work.
- Ask: list three published articles about your topic, with author, date and a link.
- Open each link and search each title. On a truly niche topic you will usually find at least one invented article, wrong author or dead link.
- Ask again, adding: only include articles you are certain exist. If you are not sure, say I don't know.
Expected result: The second answer is shorter and more careful, sometimes a plain I don't know. You watched one sentence change the odds of the next line.
Questions readers ask
Why does ChatGPT make things up rather than admit to not knowing it?
It answers one question over and over: what text is likely to come next? It never asks whether the text is true, because no list of facts sits inside the model. Where its training data ran thin, it still writes the likeliest sounding text.
If I tell it not to make things up, will it stop?
It helps, but it cannot stop it. Your instruction sits on the desk and makes I don't know a likelier next line. It cannot add the missing fact or a checking step. Anthropic's own guide says its techniques reduce hallucinations but do not eliminate them.
Is it getting better, and are paid plans safer?
Vendors promise fewer errors, not a rate you can plan on, for any plan. A score on someone else's test says little about your field. Test it on questions from your own work where you know the answers.
Sources: Google Cloud: What are AI hallucinations; Anthropic: Reduce hallucinations.
This figure is from AI Made Crystal Clear, chapter 5.
The chapter builds the idea step by step, with recipes you can follow, a Try it to check yourself, and every source.
Get the book
