What is an AI Hallucination?
An AI hallucination is when a large language model generates text that sounds confident and coherent but is factually wrong or entirely fabricated. The model cites a paper that doesn’t exist, produces a legal case with the wrong outcome, invents a statistic, or describes a feature of a product that was never built. It doesn’t know it’s wrong. There’s no internal alarm that fires when the model is making something up — it generates the next token based on probability regardless of whether the output corresponds to reality.
The term “hallucination” is anthropomorphic and slightly misleading — the model isn’t confused in any human sense. A better technical frame is “confabulation”: the model produces a plausible continuation of the text rather than a verified factual statement. Whether that continuation happens to be true is a separate question the model isn’t equipped to answer.
Why LLMs Hallucinate
Language models are trained to predict the next token given the previous tokens. That training objective optimizes for fluency and coherence, not factual accuracy. The model learns statistical patterns in language: how concepts relate, how sentences are structured, what kinds of claims appear near other kinds of claims. It doesn’t learn a database of verified facts; it learns a compressed representation of patterns in text.
When a model is asked something outside its training distribution, or asked for specific details its training data underspecified, it does the same thing it always does: generate a plausible-sounding continuation. Since the training data is full of text that looks like authoritative statements, the confabulated content also looks authoritative. The model has no mechanism for expressing uncertainty proportional to its actual uncertainty — a gap that makes hallucination particularly dangerous in professional contexts.
Hallucination rates vary by task type. Models are more reliable on reasoning tasks, text transformation, and summarization of provided content. They’re less reliable on recall of specific facts, numbers, dates, citations, and names — the exact things operators often want to use AI for in business contexts.
Types of Hallucination
Not all hallucinations look the same:
- Factual hallucination: The model states something false with confidence. “The FDA approved X in 2019” when it wasn’t approved until 2022, or wasn’t approved at all.
- Citation hallucination: The model invents a plausible-sounding paper, case, or reference. The author, journal, year, and title all look real. The paper doesn’t exist.
- Instruction drift: When given a long document to summarize or analyze, the model gradually drifts away from what the document actually says and toward what seems plausible, especially toward the end of the context window.
- Self-referential hallucination: The model makes incorrect claims about its own capabilities, training data, or behavior. “I don’t have access to the internet” is accurate; “My training data goes through X date” is sometimes not.
Mitigating Hallucination in Production
The most effective mitigation is retrieval-augmented generation (RAG): rather than relying on the model’s internal weights for factual content, you retrieve the relevant documents at inference time and instruct the model to answer based on that retrieved content. This constrains the model to a verified source rather than pattern-matched memory. It doesn’t eliminate hallucination entirely — models can still misread or misrepresent the retrieved content — but it dramatically reduces the frequency and makes errors traceable.
Other mitigations include grounding instructions in the system prompt (“only answer based on the provided context; if you don’t know, say so”), adding confidence elicitation (“how confident are you in this answer, and what would change it?”), and human review workflows for outputs in high-stakes domains. Building AI systems without any of these mitigations and deploying them in legal, medical, financial, or compliance contexts is a specific category of risk that organizations regularly underestimate until it fails visibly.
Related Terms and Concepts
LLM, Retrieval-Augmented Generation, Prompt Engineering, Context Window, Agentic AI