Hallucination

The main danger of hallucination is its tone: the model usually states a wrong answer with the same confidence as a correct one. The user has no cue for telling them apart. In distribution that can look like a fluent claim about last month’s sales, a SKU’s return rate, or a campaign’s effect that the invoices do not support.

Ways to reduce the risk in organizational systems: tie the answer to real data instead of relying on the model’s knowledge; use a deterministic query for numbers instead of letting the model generate them; validate the output against the data before display; a critique layer that rejects unsupported claims; and requiring the system to declare confidence level and data gaps.

**A practical point for evaluating a product:** ask the vendor in what situation the system says “I don’t know.” If there is no clear answer, that situation probably does not exist.