Operation “Sponsored Discontent”: Influence activity

OpenAI · 1 year ago · 17 ● 4 sources

OpenAI removed accounts likely linked to China that were using its AI models to generate English social media posts and Spanish-language articles for influence operations. The operation involved dozens of accounts coordinating across multiple platforms to spread content at scale. As a result, OpenAI strengthened detection of coordinated inauthentic behavior and increased monitoring of large-scale content generation for influence campaigns.

Read the full story →

OpenAI AI Security Content Moderation

Copied

Romance-baiting scam: AI-assisted pig butchering workflows

OpenAI · 1 year ago · 40 ● 2 sources

OpenAI removed accounts based in Cambodia that used its AI tools to generate and translate deceptive romance and investment scam messages. The banned accounts appeared coordinated for what is known as pig butchering fraud, where scammers build fake romantic relationships to manipulate victims into financial schemes. The action illustrates how generative AI tools are being weaponized for large-scale fraud and the moderation challenges platforms face in detecting such abuse.

Read the full story →

OpenAI AI Security Content Moderation

Copied

Task scam: AI-assisted fake review jobs

OpenAI · 1 year ago · 55 ● 2 sources

OpenAI shut down accounts based in Cambodia that used AI translation tools to run fake-review job scams requiring victims to pay upfront fees. The scammers specifically leveraged OpenAI's models to translate messages, making the scheme harder to detect across language barriers. Platforms will likely face pressure to better detect AI-assisted fraud patterns and add stricter verification for gig-work recruitment.

Read the full story →

OpenAI AI Security Content Moderation

Copied

Operation “Peer Review”: AI-assisted surveillance planning

OpenAI · 1 year ago · 39 ● 4 sources

OpenAI removed accounts believed to originate from China that were using its AI models to draft surveillance system proposals, analyze documents, and write code. The accounts were part of an operation called 'Peer Review' and were banned from the platform. The action reflects OpenAI's enforcement of policies against using its tools for surveillance purposes.

Read the full story →

OpenAI AI Security Content Moderation

Copied