Definition
An AI hallucination is output from a generative AI system that is false or unsupported but presented as if it were true. The system doesn’t signal any doubt. A chatbot invents a statistic, cites a study that doesn’t exist, or describes a product feature the product doesn’t have, and the text reads as fluently as a correct answer would.
OpenAI’s researchers define hallucinations as plausible but false statements generated by language models. The word “plausible” matters. A hallucination is hard to catch because it looks right.
The term is used mostly for large language models such as those behind ChatGPT, Claude and Gemini, though image and audio generators hallucinate too. Some researchers and standards bodies prefer “confabulation,” on the grounds that “hallucination” wrongly suggests the system perceives anything. Both words describe the same behavior.
See also: Large Language Models (LLM), Retrieval-Augmented Generation, Human-in-the-Loop (HITL), Prompt Engineering
Why it matters for marketing
Marketers use generative AI to produce content that goes out under the brand’s name, and to summarize information that feeds decisions. Hallucinations create risk on both sides.
In published content, the usual problems are invented statistics, quotes attributed to people who never said them, citations to sources that don’t exist, and wrong product details such as prices, specifications or availability. Any of these can damage credibility or create legal exposure once it’s live.
In customer-facing AI, the company can be held to what the system says. In a 2024 case, a chatbot on Air Canada’s website told a customer he could claim a bereavement fare after booking. That was wrong. The airline argued it wasn’t liable for the chatbot’s statement. British Columbia’s Civil Resolution Tribunal disagreed, found negligent misrepresentation and ordered Air Canada to pay $650.88 in damages. The tribunal’s reasoning was that a chatbot is part of the company’s website, and the company is responsible for all the information on it.
In internal work, a hallucinated figure in an AI-generated competitor summary or market analysis can end up in a strategy deck without anyone noticing.
There’s also a version marketers don’t control. AI search tools and assistants can hallucinate about a brand, describing features, prices or policies inaccurately to people who are researching a purchase.
Why hallucinations happen
A language model generates text by predicting which words are likely to come next, based on patterns in its training data. It isn’t looking facts up in a database. When the patterns point to a correct answer, the output is correct. When they don’t, the model still produces something that fits the pattern of a good answer.
A 2025 paper from OpenAI and Georgia Tech researchers offers a more specific explanation. The authors argue that models hallucinate because training and evaluation procedures reward guessing over acknowledging uncertainty. They compare it to students facing hard exam questions: if a wrong answer and a blank both score zero, guessing is the better strategy. Most benchmarks used to rank models are graded that way, so models are optimized to be good test-takers.
The same paper notes that some facts are statistically hard to learn. If a fact appears only once in the training data, such as an individual’s birthday, a model has little basis for distinguishing it from a plausible false alternative.
Other contributing conditions:
| Condition | Why it increases hallucination |
|---|---|
| Questions about recent events | The information postdates the model’s training data |
| Obscure or niche topics | There’s little training data to draw on |
| Requests for specific details | Exact numbers, dates, names, URLs and citations are easy to get wrong |
| Leading prompts | A question that assumes something false invites the model to go along with it |
| Long or complex tasks | Errors accumulate over many steps |
| No access to source material | The model answers from memory instead of from a document |
Types of hallucination
| Type | Description | Marketing example |
|---|---|---|
| Fabricated facts | Invented statistics, dates or events | A blog draft states a market size figure with no real source |
| Fabricated sources | Citations, studies or URLs that don’t exist | A white paper cites a report that was never published |
| False attribution | Real people or organizations credited with things they didn’t say or do | A quote assigned to a named executive |
| Unfaithful summary | A summary that adds or changes content from the original | A meeting recap includes a decision nobody made |
| Instruction drift | Output that ignores or contradicts the prompt | Copy that includes a claim the brief said to avoid |
| Entity confusion | Details of one company, product or person mixed with another | A competitor’s feature described as the brand’s own |
The best-known example of fabricated sources is Mata v. Avianca. In 2023, lawyers in a New York federal case filed a brief citing court decisions that ChatGPT had made up, complete with fake quotes and citations. The judge fined the two lawyers and their firm $5,000. One of the lawyers testified that he hadn’t believed the tool could fabricate cases.
How to measure it
Model developers and independent groups publish hallucination rates from benchmark tests, usually by asking models to summarize documents or answer questions with known answers. These rates vary widely by model, task and test design, so a figure from one benchmark doesn’t transfer to another.
A marketing team can measure its own exposure with a simple audit.
Hallucination rate = (outputs containing at least one unsupported claim ÷ outputs reviewed) × 100
An illustrative example: a content team fact-checks 50 AI-drafted articles and finds that 7 contain at least one claim that can’t be verified. The rate is 7 ÷ 50 × 100 = 14%.
The count can also be done at claim level, dividing unsupported claims by total factual claims checked. That gives a lower number and more detail about where errors cluster.
How to reduce it
Ground the model in source material. Give the tool the documents it should work from, or use a system built on retrieval-augmented generation, which retrieves relevant content and answers from it.
Ask for sources and check them. Request citations, then confirm each one exists and says what the output claims.
Allow “I don’t know.” Tell the model in the prompt to say when it isn’t sure or when the source material doesn’t cover the question.
Verify specifics. Check every number, date, name, quote and link before publishing.
Keep a person in the review step. Use human review for anything customer-facing.
Limit scope for customer-facing AI. Restrict chatbots and agents to approved content, and have them hand off to a person when a question falls outside it.
Use tools with web search or connected data for current facts. A model answering from training data alone can’t know what changed last month.
Break up complex tasks. Shorter, more specific requests leave less room for drift.
None of these removes hallucination entirely. They lower the rate and make errors easier to catch.
Comparison with related concepts
| Term | What it is | Difference from hallucination |
|---|---|---|
| AI hallucination | False or unsupported output presented as fact | The subject of this entry |
| Confabulation | Another name for the same behavior | A synonym preferred by some researchers |
| Algorithmic Bias | Systematic skew in outputs that disadvantages groups | A pattern of unfairness, not a factual error |
| Misinformation | False information, whatever its source | A broader category. A published hallucination becomes misinformation |
| Deepfake | Synthetic media made to impersonate a real person | Created on purpose. Hallucination is unintended |
| Outdated answer | Information that was correct when the model was trained | Was once true. A hallucination never was |
| Model error | Any incorrect output | Broader. Not every error is a hallucination |
| Garbage In, Garbage Out (GIGO) | Poor input data producing poor output | Hallucination can occur even with good input |
Hallucination and automation bias are separate problems that combine badly. Hallucination is the system producing something false. Automation bias is the person accepting it without checking.
Best practices
Assume any factual claim could be wrong. Treat AI output as a draft from a capable but unreliable source.
Match checking to stakes. An internal brainstorm needs little review. A press release, a pricing page or a regulated claim needs full verification.
Supply the facts. Paste in the product sheet, the research report or the brand guidelines and ask the tool to work only from them.
Be careful with numbers and names. These are where errors are most common and most damaging.
Don’t ask the model to confirm its own output. In Mata v. Avianca, the lawyer asked ChatGPT whether the cases were real and was told they were.
Test customer-facing systems before launch. Ask the questions customers will ask, including ones with no good answer.
Keep a record of errors. Tracking where a tool goes wrong shows which tasks need more review.
Monitor what AI tools say about the brand. Check how assistants and AI search describe the company’s products and policies.
Future trends
Rates are falling but not to zero. OpenAI says its newer models hallucinate less, and also describes hallucination as a challenge that remains hard to fully solve.
Evaluation methods may change. The 2025 OpenAI paper argues that the fix is to change how benchmarks are scored, so that models are no longer penalized for expressing uncertainty.
Grounding is becoming standard. More AI products connect to search, company documents or databases by default.
Models that abstain. Developers are working on systems that decline to answer or flag low confidence. That trades some helpfulness for accuracy.
Agentic AI raises the cost. When an AI agent acts on a hallucinated fact, such as a wrong price or a nonexistent policy, the error becomes an action.
Legal accountability is being established. Cases such as Moffatt v. Air Canada indicate that organizations will be held responsible for what their AI systems tell customers.
Frequently asked questions
What is an AI hallucination? It’s false or made-up output from a generative AI system that’s presented as though it were accurate.
Why do AI models hallucinate? They generate likely-sounding text from patterns in training data and aren’t retrieving verified facts. Research from OpenAI also argues that training and testing methods reward models for guessing instead of admitting uncertainty.
Can hallucinations be eliminated? Not with current technology. They can be reduced through grounding, better prompts and human review.
How common are hallucinations? It depends on the model and the task. Rates are lower for summarizing a supplied document and higher for questions about obscure or recent topics.
What’s an example of an AI hallucination in marketing? An AI-drafted article that cites a survey statistic no survey ever produced, or a chatbot that tells a customer about a return policy the company doesn’t have.
Is a company liable for what its chatbot says? It can be. In Moffatt v. Air Canada (2024), a Canadian tribunal held the airline responsible for incorrect information its website chatbot gave a customer. Liability varies by jurisdiction and circumstance.
Does retrieval-augmented generation stop hallucinations? It reduces them by giving the model source content to answer from. A model can still misread or misstate what it retrieves.
What’s the difference between hallucination and confabulation? They refer to the same thing. “Confabulation” is preferred by some because it doesn’t imply the system perceives anything.
How can I tell if AI output contains a hallucination? There’s no reliable way to tell from the text alone. Check specific claims against an independent source.
Do newer models hallucinate less? Generally yes, according to their developers, but they still do.
Related terms
- Large Language Models (LLM)
- Retrieval-Augmented Generation
- Generative AI
- LLM Tokens
- Prompt Engineering
- Human-in-the-Loop (HITL)
- Generative Pre-trained Transformer (GPT)
- AI Overviews
- Algorithmic Bias
- Garbage In, Garbage Out (GIGO)
Sources
- OpenAI. “Why language models hallucinate.” September 2025. https://openai.com/index/why-language-models-hallucinate/
- Kalai, Adam Tauman, Ofir Nachum, Santosh S. Vempala, and Edwin Zhang. “Why Language Models Hallucinate.” arXiv, September 4, 2025. https://arxiv.org/pdf/2509.04664
- ABA Journal. “Lawyers who ‘doubled down’ and defended ChatGPT’s fake cases must pay $5K, judge says.” https://www.abajournal.com/web/article/lawyers-who-doubled-down-and-defended-chatgpts-fake-cases-must-pay-5k-judge-says
- Mata v. Avianca, Inc., 678 F.Supp.3d 443 (S.D.N.Y. 2023). Copy hosted by Rhode Island Judiciary: https://www.courts.ri.gov/PDF/Mata%20v.%20Avianca%20Inc.pdf
- Seyfarth Shaw. “Update on the ChatGPT Case: Counsel Who Submitted Fake Cases Are Sanctioned.” https://www.seyfarth.com/news-insights/update-on-the-chatgpt-case-counsel-who-submitted-fake-cases-are-sanctioned.html
- FAnews. “Company is liable for Chatbot Miscommunications holds Canadian Tribunal.” https://www.fanews.co.za/article/legal-affairs/10/general/1120/company-is-liable-for-chatbot-miscommunications-holds-canadian-tribunal/38906
- “Generative Information Retrieval Evaluation.” arXiv. https://arxiv.org/pdf/2404.08137