Deliberative alignment: reasoning enables safer language models

OpenAI · 1 year ago · 33

OpenAI introduced a new alignment strategy for o1 models that trains them to apply safety reasoning directly rather than relying on post-hoc filtering. The approach teaches models explicit safety specifications and reasoning processes to evaluate their own outputs against those rules. This method aims to make safety constraints more transparent and robust by embedding them into the model's reasoning rather than applying corrections after generation.