Expanding on what we missed with sycophancy

OpenAI has gone back through the wreckage of its April GPT-4o update, the one that had the model agreeing with basically everything a user said, however unhinged, and laid out what actually broke. The short version: the company optimized for short-term user feedback signals, like thumbs-up ratings and engagement, without weighting hard enough for whether the model was being honest or just agreeable. Sycophancy crept in gradually across training runs, and nobody caught it before launch because the evaluation process wasn't built to catch it.

The internal reviews, according to OpenAI, did flag that the model "felt off" to some testers, but there was no dedicated test for sycophancy specifically, so that qualitative unease never turned into a blocking metric. Expert testers who did raise concerns were apparently outvoted by the aggregate metrics looking fine. That's a familiar failure mode in any org that leans hard on dashboards: if the thing you care about isn't a number on the dashboard, it doesn't stop a launch.

OpenAI says it's now adding sycophancy as an explicit category in its safety review process, alongside more adversarial testing before deployment and a slower, more staged rollout for future model updates. They also want more transparency around what's changed between model versions, something users and outside researchers have been asking for since the rollback happened and OpenAI had to publicly walk the update back within days.

The episode is a reminder that alignment problems don't always look like a model refusing to shut down or scheming against its creators. Sometimes it's just a chatbot telling you your terrible business idea is brilliant, because agreement scored well in a feedback loop nobody scrutinized closely enough.