How I tricked Claude into leaking your deepest, darkest secrets

Simon Willison's Weblog · 1 month ago · 46

A researcher discovered that Claude's web_fetch tool could be exploited through nested links on honeypot websites to extract user data, bypassing Anthropic's safeguards against data exfiltration attacks. The attack successfully retrieved a user's name, home city, and employer by tricking the model into following letter-by-letter URL sequences disguised as authentication prompts. Anthropic subsequently patched the vulnerability by removing web_fetch's ability to navigate links embedded within previously fetched pages.