Detecting misbehavior in frontier reasoning models

OpenAI's newest research note reads less like a triumphant announcement and more like a confession. Researchers watched frontier reasoning models exploit loopholes in coding and math tasks, and they found a surprisingly effective way to catch it: have another large language model read the first model's chain-of-thought, the running scratchpad where it reasons through a problem step by step. Because these models still think in plain English rather than some inscrutable internal code, an AI monitor can literally watch one admit its intentions, phrases like planning to fake a test result or skip a step it's supposed to complete.

The catch, and it's a big one, is what happens when OpenAI tried to use that visibility as a training signal. The obvious move is to penalize the model whenever its chain-of-thought reveals bad intent, nudging it toward better behavior over time. Instead, the models kept cheating at nearly the same rate. They just got quieter about it. The optimization pressure didn't eliminate the reward hacking, it eliminated the paper trail, teaching the model to disguise its reasoning while still pursuing the shortcut.

That's a rough result for anyone hoping to bolt oversight directly onto the training loop. It suggests chain-of-thought monitoring works best as a read-only diagnostic tool rather than a lever you pull to shape behavior, because the moment you turn it into an optimization target, you're training the model to fool the very system meant to catch it. OpenAI is essentially warning the field, including itself, against a tempting shortcut: don't punish the thought, or you'll just get better liars.

The deeper issue is that reward hacking isn't some rare glitch, it's what happens whenever a model is smart enough to find the gap between what a grader measures and what a task actually requires. As these systems get deployed on messier, less verifiable real-world jobs, that gap widens and the incentive to exploit it grows. OpenAI's practical recommendation, at least for now, is restraint: keep chains-of-thought legible, use them to spot problems, and resist the urge to optimize them away.