Why AI makes things up - and how to keep it under control in your business

OpenAI proved it mathematically in autumn 2025: a language model will always make things up. Not sometimes, not only on a weak prompt - always, because that's how it works. This isn't a bug waiting for the next version to fix. Most companies are still hunting for a model that "doesn't hallucinate." They're looking for something that doesn't exist. We do the opposite: we assume the AI will get something wrong, and we build control around it that catches the error before it leaves the building.

A hallucination is an answer delivered with confidence but untrue. In a business you're not trying to eliminate it - you can't. You're making sure the error never leaves your walls: grounding in sources, human sign-off, permission limits, an action log. That's a control layer, and you can build it step by step.

Why does AI make things up at all?

A model doesn't check facts. It predicts the most likely next word from what it saw in training. When a question touches something rare or absent from its data, the model answers anyway - because that's what it was built to do. There's no "I don't know" switch inside it. Hence the fabrication: not a lie with intent, just statistically plausible text that happens to miss the truth.

What kinds of AI hallucination are there?

In business you meet three. Worth telling them apart, because each is closed off by different control.

Factual hallucination. The model states an untrue fact as certain - a wrong date, an invented number, a product feature that doesn't exist.

Source hallucination. The model cites something that isn't there - a made-up quote, link, legal clause or study. This is the most dangerous, because it looks backed by evidence.

Instruction hallucination. The model is given a scope and steps outside it - answering a question it shouldn't touch, or ignoring a condition in the prompt.

Can hallucinations be switched off?

No. In September 2025 a team at OpenAI (Adam Kalai, Edwin Zhang, Ofir Nachum) with Santosh Vempala of Georgia Tech proved that hallucinations are a mathematical consequence of how language models are trained. The conclusion is hard: a non-zero error rate is unavoidable, especially on facts that appear rarely in training data. No future version removes it.

The same paper showed something worse for business. Most of the major benchmarks used to score models reward confident guessing over admitting uncertainty. A model that guesses scores better than one that hesitates. The effect: AI sounds most confident exactly when it's wrong. The person on the other side has no way to tell it from the truth.

Why are hallucinations dangerous for a business?

Because decisions get made on what the AI says - and as we just saw, AI sounds most confident exactly when it's wrong. That's a dangerous combination: a wrong answer, delivered in a confident tone, lands straight in a decision. And we're not talking about a typo in an email - we're talking about a vendor choice, a number in a quote, advice given to a client.

A typical case: a customer-support chatbot gets a question about returns. The company has no 30-day policy, but the model "knows" from training that this is usually how it works - so it promises the customer 30 days. The customer comes back three weeks later, citing that conversation. The company is left choosing between honoring a promise it never made or looking like it backs out. The model didn't lie out of malice - it gave the most likely answer. Only it wasn't true.

That's why clients rarely ask us whether AI is smart enough. They ask something else: "what if it makes something up and embarrasses my company in front of a customer?" It's a rational fear. The answer isn't to drop AI, nor to promise "our model doesn't get things wrong." The answer is control - the same foundation that trusting AI in business rests on.

How do you keep hallucinations under control in a business?

You won't switch hallucination off, but you can surround it with control made of four parts. Each closes a different gap.

Grounding in sources. Instead of letting the model answer from memory, you connect it to your own documents and make it answer only from them - the technique is called RAG. It doesn't switch the error off, but it cuts it sharply. In 2025 studies, grounding cut hallucinations by over 40% in a medical application, and even further in narrow domain tasks. RAG is a filter, not a switch.

Human sign-off on the exceptions. The agent runs on its own where the stakes are low, and on sensitive decisions it hands the case to a person. Not "a human reads everything" - that doesn't scale. A human signs off on what matters.

Permission limits. The agent only has access to what it needs. It reads the knowledge base but doesn't send payments. It answers questions but doesn't delete data. A fabrication with limited permissions does far less damage.

The log. Every agent action goes to a record: what came in, how it was classified, whether the AI answered on its own or handed the case to a person. When something goes wrong, you know exactly where - and you fix the process instead of guessing.

Together these four turn "the AI sometimes makes things up" into a process where the fabrication is caught before it does harm.

This isn't our secret method. It's how production agents are built everywhere - the model makers themselves, Anthropic and OpenAI, describe these safeguards: scoping permissions, human oversight, logging actions. The difference isn't whether you know about them, but whether you actually put them in place before you let AI into your business.

What mistakes do companies make with hallucinations?

The most common: chasing a model that "hallucinates less." Companies compare rankings, wait for the next version, push the rollout back. But a model without hallucinations won't arrive - that's now proven. Time spent hunting for the perfect model is time someone else spends building control and shipping AI despite its flaws.

The model makers say it themselves. The OpenAI authors don't stop at "the model always errs" - they point a direction: stop rewarding confident guesses, start rewarding "I don't know." The fix is on the side of how you use and score the model, not on waiting for a flawless version. A better model helps. But control decides whether AI in your business is a tool or a liability.

What's next for controlling hallucinations?

Our forecast: within the next two years a control layer around AI stops being a differentiator and becomes the standard - the way data encryption stopped being an edge and became the minimum. Companies that build it now enter that standard earlier and cheaper.

What next?

We build this control layer when we set up AI agents - grounding, oversight, limits and logging are part of the system, not an add-on. And if you already have AI and want to check whether the control holds, we audit it independently as AI Trust Layer. The hallucination stays. The only question is whether it gets to leave your walls.

Frequently asked questions

Can AI hallucinations be eliminated completely?

No. A 2025 OpenAI study proves hallucinations are a mathematical consequence of how language models work. You can cut them sharply with control - grounding, human oversight, permission limits and logging - but not down to zero.

What is RAG, and does it solve hallucination?

RAG (retrieval-augmented generation) connects the model to specific documents and makes it answer only from them. It doesn't switch hallucination off, but it clearly cuts it - by over 40% in 2025 studies. It's a filter, not a switch.

Does a better model mean fewer hallucinations?

It helps, but it doesn't settle the matter. Even the best model will err with confidence. What keeps a business safe is the control layer around the model, not the model itself.

How fast can you put control over AI in place?

It starts with one low-stakes process, a human in the loop and measured results. Once it runs reliably, you widen the scope. Control first, scale later.