Because a language model generates text by predicting likely words rather than looking up verified facts, it can state wrong information with complete confidence. This is called hallucination.
It happens most with specific details the model was not reliably trained on: fake citations, invented statistics, non-existent product features, or wrong dates. The output reads fluently, which makes errors easy to miss.
For example, an AI might confidently cite a court case or research paper that does not exist. The practical lesson is to treat AI output as a helpful draft, not a source of truth, and to verify anything important — especially numbers, quotes and facts that affect real decisions.