Improving mathematical reasoning with process supervision

OpenAI Blog · 3 years ago

OpenAI trained a model using process supervision, which rewards individual reasoning steps rather than just final answers, to improve mathematical problem solving. The approach achieved state-of-the-art performance on mathematical benchmarks by incentivizing correct intermediate steps in reasoning chains. This method also improves model alignment by ensuring the generated reasoning is endorsed by humans rather than just producing correct outputs through potentially opaque means.