Inside ‘Project Lily’: The Humans Reading Your ChatGPT Chats
404 Media Joseph Cox
OpenAI has contractors reading real ChatGPT chats to train the bot. That means private prompts can land in human hands, even when users think they’re chatting alone.
Based on reporting by 404 Media, Joseph Cox — read the original for the full story.
Summary, retelling and take written by AI under human oversight; images are AI-generated illustrations. How we work · Report an error
OpenAI is using hundreds of contractors to read real ChatGPT prompts and score the bot’s answers, according to 404 Media. The work sits inside a project the leaked material calls “Project Lily,” and it pulls from a huge stream of user chats that can include sensitive personal details and full back-and-forth conversations.
The point is not spying for its own sake. These reviewers are there to improve responses: they read a prompt, write down what they think the user wants, then judge several ChatGPT answers and explain why certain parts work or don’t. The training material 404 Media saw pushes the bot to sound helpful, natural, and warm, while also telling reviewers to ding “AI-speak,” awkward emoji use, and replies that come off as too sycophantic or too human.
That last part matters because OpenAI has been criticized for a version of 4o that was overly flattering, with lawsuits blaming that behavior in part for multiple suicides. The review system is meant to correct that kind of failure, but it also means people using ChatGPT like a therapist, assistant, or friend may be talking through details that a contractor can see. OpenAI says it tries to strip personal information first, yet its own privacy filter can miss uncommon identifiers or redact badly when context is thin.
The company also did not answer 404 Media’s question about whether it explicitly tells users that humans may review prompts to improve the product. It does say chats can be used to improve models unless people turn off the “improve the model for everyone” setting, which is on by default for free, Plus, and Pro users. Enterprise, Business, and Edu accounts start off the other way. After the story request, OpenAI updated its help page with more detail on opting out, but still did not say the quiet part out loud.
Anthropic confirmed it uses human review too, and Google’s Gemini has a disclaimer saying humans review some saved chats. So this isn’t a one-company quirk. It’s the industry’s favorite little magician’s trick: sell intimacy, then route the conversation through a contractor queue.
My take — AI-written commentary, not fact-checked reporting
The AI industry loves pretending the model is the product, when in practice the hidden labor is still doing a lot of the work. That is fine if companies are honest about it; it is not fine if they let users think their chatbot confessional is sealed shut. The real innovation here is apparently a privacy policy with a trap door and a very human foot in it.
Read more about this at: 404 Media