What are AI hallucinations, and how do you stop them?

This is a plain explainer of what hallucinations are and how to reduce them. For the buyer-and-due-diligence angle - what unmanaged hallucinations do to a business’s value in a sale - see our piece on hallucinations, governance and the trust discount.

What a hallucination actually is

A hallucination is a confident falsehood. The model invents something that sounds right and presents it without any hint of doubt. In a business setting that looks like a made-up figure in a report, a citation that does not exist, a product feature or policy you never had, or a plausible but wrong step in an instruction.

The problem is not that AI makes mistakes - every tool does. The problem is the delivery. A hallucination arrives polished, fluent and certain, with no flashing light to warn you. A busy person reads it, it sounds reasonable, and it goes out the door.

Why it happens

To handle hallucinations you have to understand one thing about how these models work: a language model does not “know” facts. It predicts likely text. Given everything written so far, it works out the most probable next words and produces them. Most of the time the most probable text is also true - that is why it is useful at all. But when the model hits a gap, a detail it was never trained on or a question it cannot answer, it does not stop. It keeps predicting the most plausible-sounding continuation, and that continuation can be a confident invention.

OpenAI’s own research goes further, arguing that the way models are trained and scored actively rewards guessing over admitting uncertainty - much like a student who gets more marks for putting down an answer than for leaving a hard question blank. The model has, in effect, been taught that a confident guess beats an honest “I’m not sure.” That is the behaviour you are managing.

Why an owner should care

Hallucinations are the main trust risk in using AI - not the only one, but the one that bites quietly.

Wire AI into customer-facing replies, quotes, advice or anything with a legal or financial edge, and a single confident falsehood can do damage before anyone notices. Because the output looks competent, the error slips past the usual smell test. Get this right and AI is a force multiplier. Ignore it and you have automated a confident liar into your workflow.

How to reduce it

You cannot switch hallucinations off, but you can drive the rate down a long way. Five levers, roughly in order of impact.

• Ground answers in your own documents (RAG). Retrieval-augmented generation means the model answers from your material - your policies, products and data - instead of its general memory. You retrieve the relevant passages, hand them over, and the model quotes them. This is the single biggest reduction for questions about your own business.
• Ask for sources or citations. Require the model to show where a claim came from. A traceable claim can be checked; an untraceable one is a flag.
• Keep a human in the loop on anything that matters. Any output that touches money, law, safety or a customer commitment gets a person between it and the outcome. The model drafts; a human signs off.
• Use evals. An eval is a small set of test questions with known good answers. Run it before you trust a tool, and re-run it when anything changes, so you measure how often it is wrong rather than hope.
• Pick low-stakes tasks first. Start where a mistake is cheap - internal drafts, first-pass summaries, brainstorming - and build confidence before you point AI at anything high-stakes.

Honest limits

Here is the part the marketing skips: you reduce hallucinations, you do not eliminate them.

• RAG narrows the gap, it does not close it. If retrieval misses, or the right passage is not in your library, the model can still fall back on a confident guess. Grounding helps enormously; it is not a guarantee.
• Anyone promising zero hallucinations is overselling. The behaviour is baked into how these models work. The right mental model is risk management, not a cure.
• Confidence is not a signal. The model sounds exactly as sure when it is wrong as when it is right, which is precisely why the human check exists.

The practical rule: decide which tasks can tolerate a rare confident error and which cannot, put checks on the ones that cannot, and never wire AI straight into a high-stakes action - a payment, a legal commitment, a published claim - with no human in between.

The one-line version

A hallucination is when AI states something false but plausible, confidently, because it predicts likely text rather than knowing facts - and OpenAI’s research suggests models are trained in a way that rewards guessing. It is the main trust risk in AI. You reduce it by grounding answers in your documents, asking for sources, keeping a human in the loop, running evals and starting on low-stakes tasks. You manage it; you never cure it.