Making LLMs more accurate by using all of their layers

Google Research · 9 months ago

Researchers introduced Self Logits Evolution Decoding (SLED), a method that improves LLM factual accuracy by leveraging information from all neural network layers during text generation instead of just the final layer. SLED achieved up to 16% accuracy improvement on multiple benchmarks including TruthfulQA and FACTOR across models like Gemma, GPT-OSS, and Mistral, with only a 4% increase in inference latency. The method requires no external knowledge base or fine-tuning and can be combined with other factuality-improvement techniques.