Hallucination

The AI gave you a confident, detailed, well-structured answer. It cited a case. It named a judge. It referenced a court filing with a docket number.

None of it existed.

This is hallucination. Not a glitch, not a bug, not an edge case the developers forgot to fix. It is a fundamental characteristic of how large language models work, and understanding it changes how you should use every AI tool you interact with.

A language model does not retrieve facts the way a search engine does. It does not look things up. It predicts. Given a sequence of words, it calculates the most statistically likely next word, then the next, then the next. It has been trained on an enormous amount of text and has learned the patterns of how information tends to be expressed. When you ask it a question, it generates an answer that looks like the kind of answer that question usually gets, based on everything it learned during training.

Most of the time this works. The patterns are right. The answer is accurate. But the model has no mechanism for knowing when it is wrong. It has no internal flag that fires when it crosses from knowledge into confabulation. It generates text that sounds confident because confident-sounding text is what it learned to produce. The hallucination looks exactly like the correct answer. That is what makes it dangerous.

A lawyer in New York submitted a legal brief citing six cases generated by ChatGPT. None of the cases were real. The citations were invented. The docket numbers were invented. The judges' names were invented. The lawyer had not verified any of them. The brief was filed in federal court.

The cases sounded real. That was enough.

Hallucination does not mean the AI is lying. It means the AI is completing a pattern without access to the ground truth that would tell it whether the pattern is correct. It is not deceptive by intent. It is wrong by design, and the wrongness is invisible until someone checks.

The check is your job. It has always been your job. AI does not change that. It just makes the cost of skipping it much higher.