Extrinsic Hallucinations in LLMs

Lil'Log · 2 years ago · 38

Researchers distinguish between in-context hallucination (output inconsistent with provided context) and extrinsic hallucination (output not grounded in the model's pre-training data or world knowledge). Extrinsic hallucination occurs when large language models fabricate information rather than admitting knowledge gaps. To reduce extrinsic hallucination, LLMs must generate factual outputs while explicitly acknowledging when they lack information about a topic.