Yet another experiment proves it's too damn simple to poison large language models

The Register · 2 months ago

A security engineer created a fake Wikipedia entry claiming he won a nonexistent German card game championship and seeded a domain to support it, then showed multiple AI chatbots confidently cited his fabricated victory when asked. The attack required only a $12 domain registration, a single Wikipedia edit, and about 20 minutes of effort. The experiment reveals that AI systems relying on web search cannot distinguish legitimate sources from newly registered domains, and that poisoned training data could persist in future models even after cleanup, representing a significant security risk especially if exploited to control AI agent actions.