Hoax: Fake Russian “troll” error message

OpenAI's latest threat report includes a small but telling case: someone used ChatGPT to manufacture a fake error message designed to look like the system had flagged the user as a Russian troll. It reads like something built for a screenshot, not for actual moderation. The message was never something ChatGPT would generate on its own — it was fabricated, then apparently shared as if it were a genuine product output.

OpenAI says the account behind this hoax likely traces back to the United States, not Russia, which flips the expected script. Rather than a foreign influence operation dressing up propaganda, this looks like a domestic actor manufacturing evidence of foreign interference, or at least the appearance of AI moderation catching it. That distinction matters a lot for how these incidents get reported and reacted to.

The company pulled the account once it identified the activity, treating it as a violation of usage policies around impersonation and deceptive content. No technical vulnerability was exploited here — this wasn't a jailbreak or a model failure. It was social engineering aimed at an audience, using ChatGPT's branding and the current anxiety around foreign influence campaigns as raw material.

What's notable is how little effort it takes to produce a believable fake in this space right now. A convincing-looking error screen, a plausible narrative about Kremlin-linked trolls, and a receptive audience primed by years of stories about disinformation — that's enough to generate attention, regardless of whether any of it reflects what the AI actually did.