LLMs Are Not (Consistently) Bayesian: Quantifying Internal (In)consistencies of LLMs’ Probabilistic Beliefs
Apple Machine Learning Research ● Covered by 2 sources
The article proposes treating large language models as information processing rules and measuring how far their belief updates deviate from Bayes updates. It introduces the “information processing gap” as the quantitative way to track these internal probabilistic (in)consistencies. As a result, the work frames LLM uncertainty behavior in terms of measurable Bayes-update errors rather than assuming consistent Bayesian reasoning.
Why it matters
Modern AI systems are being deployed in complex domains such as medicine, science, and law, where there is often not a single correct answer given the observed evidence. Such systems must be able to represent and update uncertain beliefs about the world as new evidence arrives to make rational decisions. We introduce the novel technique of studying LLMs as information processing rules and utilize the information processing gap—the deviation from Bayes updates—to study the internal (in)consistencies of how LLMs update their probabilistic beliefs from evidence. Our extensive experiments evaluate…