What happens when information theory accounts for reasoning?

IBM Research has a fresh take on a very old question: what if communication theory cared about reasoning, not just transmission? In a paper in Proceedings of the National Academy of Sciences, Luis Lastras and a group that includes Jonathan Lenchner, Barry Trager, Mark Squillante, Chai Wah Wu, Ronald Fagin, Wojciech Szpankowski, and Alexander Gray argue that a message’s value depends on what a receiver can deduce from it.

The starting point is a classic Feynman thought experiment. If humanity’s knowledge were about to disappear, what single sentence would you leave behind? Feynman’s answer was a sentence about atoms, because so much else could be rebuilt from it. IBM Research’s point is that this kind of value has always sat outside the clean, elegant world Claude Shannon built in 1948, where the job was to measure how efficiently bits move from sender to receiver.

Shannon’s model was brilliant because it ignored meaning and still solved the engineering problem. Networks don’t need to know whether they’re moving a cat photo or a medical diagnosis. But people do care, and the researchers say that gap matters. A bit from a broken sensor is one thing. A bit from an autonomous vehicle telling it to brake is another entirely. The difference is not in the bit itself. It’s in what follows from it.

Lastras and his collaborators extend the sender-receiver setup by giving the receiver a reasoning ability. They introduce a quantity they call logical semantic entropy, which is meant to capture the communication limits when deduction is part of the story. Their claim is blunt: communication can become more efficient when the receiver can infer more from less, because the sender doesn’t have to spell out every fact one by one.

The paper then pushes that idea into some awkward corners. In the “No Need to Know” result, a sender who doesn’t know exactly what the receiver already knows still faces essentially the same fundamental limit. In the “Less Is More” paradox, Alice can use broad shorthand to save bits, but that same shorthand can leak extra context to Bob. And when the receiver is not merely uninformed but wrong, the cost of correction can blow up dramatically as those mistaken beliefs get more specific.

The project also sounds like the kind of thing that starts as a side quest and then gets serious. Lastras began it with a small team and kept at it over several years, spending time each week on a problem that has sat around since the early days of information theory. That makes the paper feel less like a neat tweak and more like an invitation to rethink a foundation. With AI systems increasingly doing both transmission and reasoning, that’s not a small ask.