Entropy-Preserving Supervised Fine-Tuning via Adaptive Self-Distillation for Large Reasoning Models

arXiv cs.CL · 18 hours ago

Researchers propose CurioSFT, a supervised fine-tuning method that preserves model exploration capability through self-distillation and adaptive temperature adjustment, rather than reducing diversity during training. The method improves in-distribution mathematical reasoning performance by 2.5 points and out-of-distribution performance by 2.9 points compared to standard fine-tuning, with downstream reinforcement learning gains averaging 5.0 points. By maintaining exploration during fine-tuning, the approach provides reinforcement learning with a broader solution space to optimize from.