Extrinsic Hallucinations in LLMs
Lilian Weng
Researchers distinguish between in-context hallucination (output inconsistent with provided context) and extrinsic hallucination (output not grounded in the model's pre-training data or world knowledge). Extrinsic hallucination occurs when large language models fabricate information rather than admitting knowledge gaps. To reduce extrinsic hallucination, LLMs must generate factual outputs while explicitly acknowledging when they lack information about a topic.
Why it matters
Hallucination in large language models usually refers to the model generating unfaithful, fabricated, inconsistent, or nonsensical content. As a term, hallucination has been somewhat generalized to cases when the model makes mistakes. Here, I would like to narrow down the problem of hallucination to cases where the model output is fabricated and not grounded by either the provided context or world knowledge. There are two types of hallucination: In-context hallucination: The model output should be consistent with the source content in context. Extrinsic hallucination: The model output should be grounded by the pre-training dataset. However, given the size of the pre-training dataset, it is too expensive to retrieve and identify conflicts per generation. If we consider the pre-training data corpus as a proxy for world knowledge, we essentially try to ensure the model output is factual and verifiable by external world knowledge. Equally importantly, when the model does not know about a f
