Policy of Thoughts: Scaling Test-Time Training for LLM Reasoning via Online Policy Evolution

arXiv cs.AI · 18 hours ago

Researchers introduced Policy of Thoughts (PoT), a framework that enables language models to refine their reasoning strategies during test time by updating adapter weights based on execution feedback from failed attempts. A 4-billion-parameter model using PoT achieved 49.71% accuracy on LiveCodeBench, surpassing GPT-4o and DeepSeek-V3 while being 50 times smaller. The approach allows models to dynamically adapt their reasoning process for individual problems rather than relying on a frozen policy.