Curriculum for Reinforcement Learning
Lilian Weng
Curriculum learning trains machine learning models more efficiently by ordering training samples from simple to complex, a technique proposed by Jeffrey Elman in 1993 that has since been applied extensively to reinforcement learning. Key concrete approaches include task-specific curricula that sort examples by difficulty metrics, teacher-student frameworks where one agent selects training tasks for another, and self-play methods where competing agents create natural curriculum progression through interaction. Adopting curricula can accelerate convergence speed and improve generalization, though poorly designed curricula may harm performance, making it essential to balance task difficulty with agent capability during training.
Why it matters
[Updated on 2020-02-03: mentioning PCG in the “Task-Specific Curriculum” section. [Updated on 2020-02-04: Add a new “curriculum through distillation” section.
