Knowledgeless Language Models: Suppressing Parametric Recall for Evidence-Grounded Language Modeling

arXiv cs.CL · 18 hours ago

Researchers introduced Knowledge-Less Language Models (KLLMs) that anonymize named entities during pretraining to reduce reliance on parametric knowledge and improve evidence-grounded reasoning. The models achieved up to 20-25% relative performance gains over standard language models on retrieval-grounded tasks with imperfect evidence. KLLMs show improved calibration and reliability by shifting toward using external context rather than internal parameters for factual claims.