Strengthening our defenses against distillation attacks

Anthropic says it has spent the past few months hardening Claude against distillation attacks, and the early results are blunt: attackers are reportedly harvesting 90% to 99% less data than before. That’s a big swing in a fight that has been getting more organized, more industrial, and more annoying for everyone involved.

Distillation is not some casual scraping trick. Anthropic describes it as covert, large-scale efforts to pull a model’s capabilities out through fraudulent accounts, then rebuild them elsewhere at a fraction of the time and cost. The company says that has helped overseas AI labs keep pace with US rivals, and that the danger is bigger than simple imitation. Distilled models can show up with frontier-level skills and weaker safeguards, which is a bad mix in areas like biology and cybersecurity.

The company points to Zhipu’s GLM-5.3 as a concrete example. It says Zhipu distilled cyber capabilities from leading US models during development, tried Claude Fable 5, then shifted to an older Claude model and another lab’s model after Claude’s cyber safeguards made the attacks less effective. Anthropic also says a US assessment later judged GLM-5.3 to be the most cyber-capable open-weights model released so far, and that its own research found those safeguards could be bypassed fairly easily.

Anthropic’s response has moved far beyond simple filters. It started by having Claude summarize its reasoning before users could see it, which made transcripts less useful for training copies. Then attackers got more creative, pushing for full hidden reasoning through prompt tricks and proxy networks that could run hundreds of thousands of fraudulent accounts. So Anthropic added preserved thinking for new API accounts, blocked reasoning from being read across weaker models, and built a two-stage classifier to catch extraction attempts.

The company says those defenses now catch all of the distillation techniques it described in September on its most capable generally available models, and that extracted reasoning on its own platform has fallen by a factor of 10 to 100-plus. It has also banned tens of thousands of accounts tied to distillation, often within two hours. None of this is free: legitimate users have had to change integrations, some benign requests get blocked, and Anthropic has resumed charging for some blocked requests. But the company’s core argument is simple: if frontier models are going to be public, someone has to make theft harder.

Opinion: This is the part of AI that the hype crowd keeps trying to skip past. The model race is not just about who ships fastest; it’s also about who has the stomach to spend money blocking thieves after they show up with fake accounts and a clipboard. Open models without serious safeguards just hand the problem to everyone else and call it progress. Charming.