Diverse reasoning traces teach LLMs to make better decisions

Amazon Science · 1 month ago

Researchers introduced a method for training large language models on multiple diverse reasoning paths for the same problem, using global forking tokens and set-supervised fine-tuning to prevent mode collapse where different tokens produce identical outputs. Performance improved 5% to 7% on standard benchmarks (with gains of 6.84% on AIME 2025 Pass@1 compared to baseline methods), demonstrating that enabling models to learn distinct reasoning strategies and select the appropriate one per question directly improves accuracy. This approach enables LLMs to solve problems through multiple qualitatively different strategies like algebraic manipulation or geometric reasoning, rather than collapsing to a single solution method.