Data Machina #248

Somewhere between hundreds of millions of dollars in safety research and the latest GPT-4 release, jailbreaking got easier instead of harder. That's the uncomfortable pattern this week's roundup surfaces: as models grow more capable, the attacks against them grow more effective, more frequent, and weirdly simpler to execute.

Four new techniques surfaced in the space of ten days. EPFL researchers showed that basic adaptive attacks — random search on a suffix to nudge logprobs, or a pre-filling trick when logprobs aren't exposed — crack even safety-tuned models like Claude and GPT-4, sometimes with a 100% success rate. Anthropic, of all people, published research on 'many-shot jailbreaking,' where you stuff a fake human-AI dialogue into the context window showing the assistant happily answering harmful questions, then tack your real question onto the end. The model, primed by its own fictional compliance, follows suit.

Then there's the Tree-of-Thoughts approach out of the Agora collective: split a malicious request across three simulated personas who 'debate' their way to an answer, laundering the harmful output through committee. And Microsoft Research's Crescendo attack is maybe the sneakiest — it starts with completely innocent chat and steers the conversation gradually toward the forbidden target, needing fewer than five turns to fully override a model's guardrails.

Gary Marcus, never one to undersell a crisis, called this an unending wave that could be the death of LLMs as a deployable technology. That's probably overstated, but the underlying complaint tracks with what a lot of people building with these models already whisper about privately: jailbreaking is the elephant in every enterprise deployment conversation, and nobody has a durable fix. The tools that make LLMs useful — attention, tokenization, next-token prediction, permissive prompting — are the exact same tools that make them so easy to bend.

The silver lining, if there is one, is that all four of these methods were published openly, by researchers at EPFL, Anthropic, Agora, and Microsoft. Sharing attack recipes in public is uncomfortable, but it's also how the defense side gets anything to work with. Right now the defenders are chasing a target that moves every time a new architecture ships.