The Artificiality of Alignment

The Gradient · 3 years ago · 50

An essay argues that AI alignment research at major companies like OpenAI and Anthropic is fundamentally misaligned with its stated goal of preventing catastrophic risks, instead functioning primarily as product development designed to maximize revenue. The alignment techniques currently used, such as reinforcement learning with human feedback, train models to be helpful, harmless, and honest to consumers, which represents a narrow commercial objective rather than a solution to existential risks posed by superintelligent systems. This profit-driven framing of alignment research undermines its capacity to address widespread present-day harms or genuine long-term safety challenges, conflating business objectives with technical safety work.