Meet GPT-Red: an LLM super-hacker OpenAI built to make its models safer

OpenAI has a new tool in its safety arsenal, and it's got an aggressive name to match its job: GPT-Red. The idea is simple enough on paper. Instead of relying purely on human red-teamers to poke and prod at large language models looking for weaknesses, OpenAI built a system that does a lot of that probing itself, using AI to attack AI.

Jessica Ji, a senior research analyst at Georgetown's Center for Security and Emerging Technology, dug into how this works in a piece for MIT Technology Review. The core technique is self-play, a training method with roots in game-playing systems like AlphaGo, where an AI improves by competing against versions of itself. Here, that same loop gets pointed at cybersecurity. One system tries to find flaws and jailbreaks in a model, the other tries to patch them, and the cycle repeats, each side getting sharper.

What makes this notable isn't the concept, red-teaming AI with AI has been floated before, but the fact that OpenAI is operationalizing it at scale on its own frontier models. Ji's assessment, quoted directly, is that "the results look very promising." That's a fairly restrained line from a researcher who studies this space closely, and it suggests GPT-Red is catching real vulnerabilities that might slip past manual testing, especially the kind of subtle prompt injection or jailbreak techniques that mutate faster than human teams can track.

The bigger context here is speed. Attackers experimenting with jailbreaks move quickly, and traditional red-teaming, however skilled, has a bottleneck problem: there are only so many hours in a human researcher's day. An automated adversary that can run thousands of attack attempts overnight changes that math entirely. Whether it changes it enough to stay ahead of increasingly creative bad actors is the open question nobody, including OpenAI, has fully answered yet.