Making LLMs more accurate by using all of their layers
Google Research
Google Research built a decoding trick called SLED that makes LLMs check their own earlier layers, not just the last one, before answering. It cuts hallucinations without extra data or retraining, for a roughly 4% speed cost.
Based on reporting by Google Research — read the original for the full story.
Summary, retelling and take written by AI under human oversight; images are AI-generated illustrations. How we work · Report an error
Every LLM has a bad habit: it commits to an answer based on whatever its final layer spits out, even when earlier layers were quietly waving a flag. Google Research's new method, SLED (Self Logits Evolution Decoding), is built around a simple bet — that those earlier layers actually know something useful, and ignoring them is leaving accuracy on the table.
The mechanics are less mysterious than the name suggests. Transformers process text through a stack of layers, and each layer can generate its own guess, or "logits," about the next token, using the same final projection matrix the last layer relies on. Normally only the last layer's guess counts. SLED instead pulls guesses from every layer, weights them, and blends them into a final probability distribution. No retraining, no external database, no fine-tuning — just smarter use of information the model already computed.
The team's toy example is the clearest illustration: a model asked to price six toys at 10 tokens each, with a 10% discount for buying four or more, tends to blurt out "6 x 10 = 60" because that arithmetic pattern is overrepresented in training data. It skips the discount. But when Google's researchers looked at the intermediate layers, a decent chunk of them had already flagged "x" as the more likely next token instead of "=," meaning they'd correctly anticipated the discount step. SLED surfaces that signal and nudges the model toward the right calculation, 54 tokens instead of 60.
Across tests on Gemma 3, GPT-OSS, and Mistral, spanning multiple-choice benchmarks like TruthfulQA and FACTOR, plus open-ended and chain-of-thought tasks, SLED beat both the unmodified models and DoLa, the previous best factuality-decoding method, with accuracy gains up to 16% on some datasets. It works on base and instruction-tuned models alike, and it stacks with other decoding tricks rather than competing with them.
The catch is modest: inference takes about 4% longer than DoLa because SLED has to process every layer's logits instead of just the last one. Google has released the code on GitHub and is already eyeing extensions into fine-tuning, visual question answering, code generation, and long-form writing — suggesting this is meant as infrastructure, not a one-off paper trick.
My take — AI-written commentary, not fact-checked reporting
This is the kind of fix I actually want to see more of: cheap, open-source, and aimed at the boring but crucial problem of models confidently lying. No new data pipeline, no bolted-on retrieval system, just squeezing more honesty out of weights the model already has. If it holds up outside Google's benchmarks, expect it to quietly show up in a dozen open-weight releases within a year, because a 4% latency tax for fewer hallucinations is a trade almost anyone running production LLMs would take without blinking.
Read more about this at: Google Research