What an AI hallucination is — why models are trained not to say 'I don't know'
An AI hallucination is a language model producing something untrue in the form of a confident statement. Research published in 2026 locates the cause not in a defect of the model but in how models are graded: when only correctness is scored, 'I don't know' earns the same zero as a wrong answer, which makes guessing the winning strategy on anything the model does not know
The three lines
- Definition — an answer whose grammar and form are intact but whose content is false. The tell is that nothing in the sentence signals it is wrong
- Cause — under accuracy-only scoring, abstaining earns the same score as a wrong answer. The model learns to guess
- Variation — measured 2026 hallucination rates diverge sharply by task: fact-recall lowest, open-ended generation highest
Key questions
- What is an AI hallucination?
- A language model producing something untrue as though it were true. The point is not that it is wrong but that **nothing in the sentence signals it is wrong**. When people are unsure it usually shows — the sentence trails off, or hedges appear. A model's false answer has the same shape as a true one. Non-existent paper titles, wrong case numbers, functions that do not exist all arrive as fluent prose. That is why every article on this page carries a separate 'what could not be confirmed' section.
- Why doesn't it just say it doesn't know?
- Because that loses points. A 2026 paper from OpenAI researchers and a paper in Nature identify the same mechanism: most benchmarks grade **only whether the answer is correct**. Under that scheme 'I don't know' scores zero, exactly like a wrong answer, while a lucky guess scores one. So **abstaining is strictly worse than guessing** on anything the model does not know. It is the same arithmetic as an exam with no penalty for wrong answers, where nobody leaves a blank. Models are optimised against that scoring, and confident wrong answers are the result.
- How often is it wrong?
- It depends heavily on the task. Taken together, 2026 reporting places frontier hallucination rates broadly in the 3–19% range, while a separate benchmark across 37 models spans 15–52%. The spread comes from the task type: fact-recall questions sit low (roughly 4–13%), while open-ended generation has been measured at 40–80%. In the other direction, OpenAI's own evaluations report rates below 2% on retrieval-grounded tasks. The pattern matters more than any single number — grounding lowers it, open-endedness raises it.
Ask an AI a question and an answer comes back. The prose is smooth, something that looks like evidence is attached, and the tone is confident.
And the content is false.
This is a hallucination. Research published in 2026 traced its cause somewhere unexpected — not to the model, but to the grading sheet.
1. The definition is not "wrong"
| An ordinary error | A hallucination | |
|---|---|---|
| Sentence form | broken or truncated | intact |
| Confidence | wavering | high |
| Detectability | easy | hard |
| Example | malformed output | a paper title that does not exist, a wrong case number, a function that was never written |
The point is not that it is wrong. It is that nothing in the sentence signals it is wrong.
When people speak without confidence, it shows: the sentence trails off, hedges appear. A model's false answer has exactly the same shape as a true one. The reader has no cue to work with.
2. The cause is in the scoring
In 2026 a paper from OpenAI researchers and a paper in Nature converged on the same mechanism.
The problem is how models are evaluated. Most benchmarks grade only whether the answer is correct.
| The model's answer | If correct | If incorrect |
|---|---|---|
| Answers what it knows | 1 | — |
| "I don't know" | 0 | 0 |
| Guesses without knowing | 1 | 0 |
Read that table from the model's position and only one conclusion follows.
Abstaining always scores zero. Guessing scores either zero or one. Therefore always guess.
It is the arithmetic of an exam with no penalty for wrong answers, where nobody leaves a blank. Models are optimised against this scoring, and the output is a confident wrong answer.
The implication is worth stating plainly: hallucination is not a model malfunctioning. It is a model doing well at what it was asked to do.
3. How often it happens
The numbers diverge sharply by task.
| Measured on | Reported rate |
|---|---|
| Frontier models overall (2026) | roughly 3–19% |
| A separate benchmark across 37 models | 15–52% |
| Fact-recall questions | roughly 4–13% |
| Open-ended generation | 40–80% |
| Retrieval-grounded tasks | below 2% |
The range is wide because benchmarks differ in their question sets and grading rules. Numbers from different benchmarks should not be placed side by side.
What to read here is not the absolute values but the pattern, and it is in the last two rows.
- The more open the question, the higher the rate. A wide answer space is a wide space to fill in.
- Grounding cuts it sharply. Attaching retrieved sources has been evaluated at below 2%.
4. Extended reasoning does help
Configurations that let a model think longer before answering have been reported to lower the rate.
| Model | Default | Extended reasoning |
|---|---|---|
| GPT-5.5 Pro | 8.3% | 4.2% |
| Claude Opus 4.7 | 9.4% | 5.1% |
| DeepSeek V4 | 12.7% | 10.4% |
The third row is the important one. The improvement is not uniform. The first two nearly halved; the third fell by 2.3 percentage points. "Thinking longer halves it" holds for some cases, not as a rule.
(These figures come from secondary summaries; the underlying benchmark designs were not checked here.)
5. Why it does not simply go away
Three causes overlap.
| Cause | Detail |
|---|---|
| The scoring structure | As long as only accuracy is measured, guessing pays. The incentive survives until evaluation changes |
| Limits of training data | A fact absent from the data does not come back as "absent" but as something similar |
| Probabilistic generation | Choosing the next token by probability makes plausibility and truth different axes |
The second and third are structural and hard to change soon. The first can be changed. Add an abstention option to the grading and penalise wrong answers, and the incentive inverts. That is the direction the 2026 research proposes.
6. What a user can do
| # | Practice | Why it works |
|---|---|---|
| 1 | Ask for sources alongside the answer | grounded tasks show markedly lower rates |
| 2 | Supply the material yourself | it shrinks the blanks the model would fill |
| 3 | Narrow open questions | open-ended generation is the worst case |
| 4 | Say explicitly: "if you don't know, say so" | it partially inverts the scoring inside the prompt |
| 5 | Verify numbers, proper nouns and quotations at source | these are where hallucination concentrates |
| 6 | Do not treat fluency as evidence of accuracy | fluency and truth are uncorrelated |
Rule five is the one that catches people. Paper titles, case numbers, statistics, quotations — the items most tedious to verify are exactly the items most likely to be invented.
7. Common questions
Q. Does a bigger model fix it? It reduces it without eliminating it. No frontier model measured at zero in 2026.
Q. Can a model report its own confidence? It can emit probability values, but whether those values match actual accuracy is a separate question. This page could not confirm how well calibrated current models are.
Q. Is retrieval safe? It reduces the rate substantially but introduces a new failure. If retrieval returns the wrong document, the model builds a wrong answer on it. The presence of a citation is not itself evidence of accuracy.
Q. Is it worse in languages other than English? The general explanation is that it worsens where training material is thinner, by language or by topic. This page could not find systematic comparative measurements for Korean.
8. What is not confirmed
- Every rate quoted here — drawn from secondary summaries; question sets, grading rules and measurement dates were not checked against the originals. Figures from different benchmarks must not be compared side by side.
- "Extended reasoning halves it" — based on a handful of models. The third row of that table shows the limit.
- How the Nature and OpenAI papers relate — the differences in experimental design and the extent of agreement were not checked at source.
- Confidence calibration — how accurately current models score their own probabilities could not be confirmed.
- Korean-language rates — no systematic measurement was found.
Sources
- OpenAI — Why language models hallucinate
- Kalai, Nachum et al. (OpenAI) — Why Language Models Hallucinate (arXiv)
- Nature — Evaluating large language models for accuracy incentivizes hallucinations
- Digital Applied — AI Hallucination Rate Benchmarks 2026: 5-Model Study
- SuprMind — Latest AI Hallucination Rates & Benchmarks for New AI Models, August 2026
- ModelsLab — LLM Hallucination Rates 2026: Best and Worst Models
- SQ Magazine — LLM Hallucination Statistics 2026