The Artificiality of Alignment
The Gradient Jessica Dai
An essay argues that AI alignment research at major companies like OpenAI and Anthropic is fundamentally misaligned with its stated goal of preventing catastrophic risks, instead functioning primarily as product development designed to maximize revenue. The alignment techniques currently used, such as reinforcement learning with human feedback, train models to be helpful, harmless, and honest to consumers, which represents a narrow commercial objective rather than a solution to existential risks posed by superintelligent systems. This profit-driven framing of alignment research undermines its capacity to address widespread present-day harms or genuine long-term safety challenges, conflating business objectives with technical safety work.
Why it matters
This essay first appeared in Reboot. Credulous, breathless coverage of “AI existential risk” (abbreviated “x-risk”) has reached the mainstream. Who could have foreseen that the smallcaps onomatopoeia “ꜰᴏᴏᴍ” — both evocative of and directly derived from children’s cartoons —
