Tracing LLM Behavior to the Training Data with Empirical Next-Token Distributions

arXiv cs.AI · 6 hours ago

Researchers studied how large language models' next-token predictions match the empirical distributions found in their training data, finding significant agreement for many inputs but divergence in others. The alignment between LLM outputs and empirical next-token distributions improves with larger model scale and more training compute. The work suggests mechanistic interpretability research should focus on understanding how model behaviors originate from training data rather than solely examining learned weights.