Hallucination

Hallucination is one of the most widely known limitations of language models: they sometimes state false information with the same fluent confidence as accurate information. A model might cite a paper that doesn’t exist, give an incorrect date for a historical event, or invent details about a real person. This happens because models generate text by predicting what words come next based on patterns in training data — they don’t “look things up” or verify claims against a database of facts. For behavior architects, hallucination is a failure mode with serious real-world consequences in domains like health, law, and finance, making it a primary concern for both evaluation design and system architecture (for example, deciding whether to ground responses in retrieved facts).