Proximal Policy Optimization (PPO)

Hugging Face Blog · 3 years ago

Proximal Policy Optimization is a reinforcement learning algorithm that stabilizes agent training by restricting policy updates through a clipped probability ratio. The clipping constrains the policy ratio to a range of [1−ϵ, 1+ϵ], with epsilon typically set to 0.2 in implementations. This mechanism prevents destructive large weight updates and allows the algorithm to update policy weights only when the ratio stays within bounds or when advantage estimates push toward the acceptable range.