AI Hallucination: Why AI Sounds Sure When It Is Wrong
AI can sound sure and still be wrong. What hallucination is, why it happens, and four real cases, from Google's Bard to a Deloitte report.
On February 8, 2023, Google published an ad for its Bard chatbot. In it, Bard said the James Webb Space Telescope took the first pictures of a planet outside our solar system. The answer was wrong: the first such pictures were taken in 2004 by the European Southern Observatory's Very Large Telescope. According to Al Jazeera, Alphabet's shares fell 8% that day and the company lost more than $100 billion in market value.
The wrong answer read as fluently and confidently as a right one would have. This behavior is called hallucination, or AI making things up.
What hallucination is
Chatbots such as ChatGPT, Claude, and Gemini are large language models: they answer by writing text that fits your question. Hallucination happens when that text contains false or nonexistent information presented as fact. What gets invented can be a number, a quote, a court case name, or a link.
The word "hallucination" is borrowed. The AI has no intent to deceive you. It writes text that fits the question, so an invented answer can sound exactly like a correct one, which makes hallucination hard to spot.
Why AI makes things up
When the model has the facts, it usually writes the right answer. When facts are missing, it still writes text that fits the question, and that text can be wrong.
A September 2025 paper by Adam Tauman Kalai and colleagues gives a plain explanation. Models hallucinate because training and evaluation procedures reward guessing over admitting uncertainty, like students facing hard exam questions. In a test where a blank answer earns nothing, guessing beats staying silent.
Anthropic's documentation describes the same problem from the user side: even the most advanced models can sometimes produce text that is factually incorrect, and the techniques that reduce this do not eliminate it. A fluent answer is not proof that it is correct. The article Is Prompt Engineering Dead? How to Structure Prompts for Modern LLMs makes a related point: models invent answers when they feel pressured to respond where data does not exist.
Four common forms
The table below is my own grouping of the cases in this article, not an official classification.
| Form | What it looks like | Case in this article |
|---|---|---|
| A plausible but false fact | The AI states something that is not true | Bard and the Webb telescope |
| Sources and quotes that do not exist | Case names, articles, or citations look real but are not | Mata v. Avianca, the Deloitte report |
| A wrong detail inside mostly correct text | Most of the text is accurate, but one footnote or quote is wrong | The Deloitte report |
| A rule or policy that does not exist | The AI describes a rule the organization does not apply | Moffatt v. Air Canada |
Four real cases
Google Bard and the Webb telescope (2023)
This is the example from the opening. A Google spokesperson said the error highlights the importance of a rigorous testing process, and the company announced a Trusted Tester program to combine external feedback with internal testing. One wrong answer in an ad was enough to make news at many outlets.
Two New York lawyers and the fake cases (2023)
Two lawyers filed a court brief that cited cases that did not exist. ChatGPT had produced the cases, complete with fake quotes and citations, and it assured the lawyers that the cases were real. On June 22, 2023, Judge P. Kevin Castel fined the lawyers and their firm $5,000 and ordered them to notify the judges whose names appeared on the fake opinions. The assurance the lawyers relied on came from the same tool that invented the cases.
Air Canada's chatbot and a refund policy that did not exist (2024)
In Moffatt v. Air Canada (2024 BCCRT 149), the airline's chatbot told a customer to book a regular ticket and apply for a bereavement refund within 90 days. Air Canada employees told him the airline did not allow retroactive applications. The British Columbia tribunal found that the airline had not taken reasonable care to make sure its chatbot was accurate, called the error negligent misrepresentation, and ordered a partial refund of CAD 650.88. The chatbot may not have been a large language model, so this is a case of a chatbot giving false information, not necessarily hallucination in the technical sense.
Deloitte's report for the Australian government (2025)
A report published in July 2025 for Australia's Department of Employment and Workplace Relations contained wrong footnotes and invented references and quotes. A judge's name was misspelled, a quote was attributed to paragraphs that do not exist in a 2019 judgment, and an article was credited to a real academic although it was never published. Chris Rudge, a welfare academic at the University of Sydney, found the errors after seeing more than a dozen invented claims attributed to his colleagues, and Australian Financial Review journalist Paul Karp reported them first.
Deloitte confirmed it used a generative AI toolchain based on Azure OpenAI GPT-4o to produce the report. It has not confirmed that AI caused the errors, and it said they did not affect the report's findings. According to Accounting Times, the report was worth about A$440,000 and the firm will repay part of that amount.
What the cases have in common
In all four cases the wrong answer looked like a right one, and it reached the public before anyone caught it. The parties involved were a technology company, two lawyers, an airline, and a large consulting firm.
The risk grows when an AI agent acts on your behalf. The article From Chatbot to Agentic AI: What Actually Changes & How to Use It Daily lists blind trust among its pitfalls. One way to reduce hallucinations in agents is to ground them with a domain glossary, covered in Why AI Agents Need a Domain Glossary in High-Stakes Systems. How to reduce the risk when you need accurate text will get its own article later.
Sources
- Why Language Models Hallucinate (arXiv:2509.04664): Kalai, Nachum, Vempala, and Zhang, September 2025. Argues that training and evaluation reward guessing over admitting uncertainty.
- Reduce hallucinations (Anthropic documentation): States that even the most advanced models can produce false text and that mitigation techniques do not eliminate it.
- Google shares tank 8% as AI chatbot Bard flubs answer in ad (Al Jazeera): Report on Bard's error, Alphabet's share drop, and Google's response, February 8, 2023.
- Court imposes sanctions on lawyers who filed bogus cases after relying on ChatGPT (LawNext): Report on the June 22, 2023 sanctions order in Mata v. Avianca.
- Mata v. Avianca, Inc. (Wikipedia): Case background, including the court's inability to find the cited cases and ChatGPT's assurance that they existed.
- AI Gone Wild: Airline Has to Honor a Refund Policy Its Chatbot Fabricated (Manatt): Law-firm summary of Moffatt v. Air Canada, 2024 BCCRT 149.
- Deloitte to refund government after using AI in $440k report (Accounting Times): October 9, 2025 report on the errors, who found them, and Deloitte's response.
