Learning from human preferences

OpenAI Blog · 9 years ago

Researchers at Anthropic and DeepMind have developed an algorithm that learns human preferences by comparing pairs of proposed behaviors rather than requiring explicit goal functions to be written. The system evaluates which of two actions a human prefers, using this feedback to infer the underlying objective. This approach reduces risks from misaligned proxy goals or incorrectly specified objectives in AI systems.