Correcting Visual Blur Induced by Attention Distraction to Reduce Hallucinations: Algorithm and Theory

arXiv cs.AI · 18 hours ago

Researchers identified that hallucinations in multimodal large language models stem from attention distraction mechanisms similar to human visual perception under divided focus, manifested as spatial inconsistency in multi-head attention and temporal fading of attention to image tokens. The proposed AFIP method addresses this through cross-head attention enrichment and dynamic historical attention enhancement, with effectiveness demonstrated across multiple benchmarks without requiring additional model training. This work provides both theoretical understanding of how attention dispersion increases model complexity and a practical approach to reduce object hallucinations in MLLMs.