A language model can tell you with complete certainty that a law came into effect on a date that doesn’t exist, that a scientific study concluded something it never said, or that a person said something they never said. It does so without hesitation, without any signs of uncertainty, in the same tone it would use to give you entirely accurate information.
This isn’t a bug that will eventually be fixed. It’s a structural feature of how these models work. And understanding it changes how you should use AI in any context where data matters.
What Is a Hallucination and Why It Happens
Language models don’t “know” things the way you do. They don’t have an internal database they consult to verify facts. They generate text token by token, choosing at each step the most probable continuation given the context.
When the context points toward a particular type of response — say, one that includes statistical data, citations, or dates — the model generates text that has the shape of that data, even if it doesn’t have accurate information available. The result is a response that sounds precise, has the right format, and is plausible in structure but may be entirely fabricated in content.
The model doesn’t know it’s lying. It has no capacity to distinguish between what it knows and what it’s making up.
This happens most often with questions about very specific data (statistics, figures, exact dates), bibliographic references, names of people and their statements, and detailed legislation or regulations.
The Patterns That Repeat Most Often
Over time, I’ve learned to recognize situations where the probability of hallucination goes up.
Citations and references. If you ask for the source of a claim, the model may invent a study with a plausible title, real authors from a related field, and a journal that exists but never published that paper. The result looks completely valid until you try to find it.
Statistics without a source. “67% of employees…” “According to a 2023 study…” If there’s no verifiable source attached to that number, treat it as approximate until you can confirm it.
Regulations and legislation. Models know general legal frameworks but can invent specific articles, law numbers, or effective dates that don’t match reality. For any legal matter with real consequences, verification is mandatory.
People and their words. Quotes attributed to real people are especially problematic. The model knows the thinking style and general positions of many authors and can fabricate quotes that sound authentic because they’re consistent with what that person might actually have said.
How to Verify: A Step-by-Step Process
This isn’t about verifying everything. It’s about knowing what to verify and how.
Identify the critical claims. Before acting on information from the AI, ask yourself: what would happen if this were wrong? If the answer is “nothing important,” you probably don’t need to verify. If the answer is “I’d make a wrong decision” or “I’d share incorrect information,” verify.
Find the primary source. Don’t ask the model if what it told you is correct. It already said it was. Find the original source: the study, the law, the book, the statement. If you can’t find the primary source, the information is unverified.
Ask questions that force uncertainty. Instead of “what’s the statistic on X?”, ask “how confident are you in this number? Do you have a specific source or is this an approximation?” Some models respond better to this framing and flag their own uncertainty.
Use the model to find counterexamples, not to confirm. If you want to know whether something is true, don’t ask “is it true that…?” Ask “what arguments exist against this?” or “are there exceptions or cases where this doesn’t apply?” It’s harder for the model to fabricate plausible counterexamples than to confirm what you’re already assuming.
When Not to Trust It Even If It Sounds Right
There’s a cognitive bias working against you when you use AI: if the answer has the right format, includes specific details, and is well-written, your brain evaluates it as more likely to be true. The appearance of credibility activates the same heuristic you use with people: “if they sound this confident and precise, they probably know what they’re talking about.”
This is exactly backwards from how hallucination probability works. The more specific and detailed the claim, the higher the probability that some detail is invented.
Questions with long, well-structured answers are more likely to contain specific errors than questions with short answers. Confidence in tone doesn’t correlate with accuracy in content.
The practical rule I use: the more important the information and the more specific the claim, the more I need to verify it outside the model. Not because AI is bad. But because it’s not designed to be a repository of verified facts — it’s designed to produce coherent and useful text. Sometimes those two things coincide. Not always.
Knowing that is half the work.
