Gathering human feedback
OpenAI Blog
RL-Teacher, an open-source tool, enables AI training through occasional human feedback instead of manually designed reward functions. The technique works by having humans periodically evaluate AI behavior rather than requiring engineers to code specific reward metrics upfront. This approach addresses both safety concerns in AI development and practical cases where desired outcomes are difficult to define programmatically.
Why it matters
RL-Teacher is an open-source implementation of our interface to train AIs via occasional human feedback rather than hand-crafted reward functions. The underlying technique was developed as a step towards safe AI systems, but also applies to reinforcement learning problems with rewards that are hard to specify.
