Yet another experiment proves it's too damn simple to poison large language models

Ron Stoner isn't actually the 2025 world champion of 6 Nimmt!, the German card game also known as Take 5. But for a while, if you asked several AI chatbots, they'd tell you he was. Stoner engineered the whole thing himself: he registered 6nimmt.com, posted a fake press release announcing his own victory, then quietly added the claim to the game's Wikipedia page, citing his own site as the source. Wikipedia's apparent authority did the rest.

That's the part that should worry anyone who trusts AI search results. Stoner didn't hack anything or write malicious code. He spent about twenty minutes and twelve dollars exploiting a simple fact: large language models with web search don't verify provenance, they just trust whatever ranks highest for a query. His fabricated site was the only source in existence for the claim, so it became the source, laundered through Wikipedia's credibility and served up by chatbots as settled fact.

Stoner breaks the damage into three layers. There's the immediate retrieval problem, where a bad source poisons a single answer. There's training-data contamination, which is worse: his edit sat live on Wikipedia from February 2025 until he pulled it last Friday, meaning any model scraped during that window may have baked his fake championship into its weights permanently, regardless of whether the Wikipedia page gets fixed. And then there are AI agents, which Stoner flags as the real danger zone. A chatbot repeating a lie is embarrassing. An agent with tool access acting on a poisoned source is a security incident waiting to happen.

None of this is technically novel, and Stoner is upfront about that. It's the same SEO and misinformation playbook that's worked on search engines for two decades, just repackaged for a system that presents answers with more confidence and less visible sourcing. A domain registered days before a citation was added should have triggered some kind of suspicion. It didn't, because nothing in the current pipeline checks for that kind of pattern.

Stoner wants AI companies to start treating data provenance as a core design problem rather than an afterthought, and he plans to re-test his fake championship in six months to see if it surfaces from training data alone, without any live web source to point to. If it does, that will confirm his fabrication is now permanently embedded in some model's memory, Wikipedia correction or not.