AI Models Overthink Problems—and It’s a Security Risk

Reasoning models are supposed to be the smart ones—the ones that pause, think step by step, and work through hard math or code before answering. Turns out that habit of thinking things through is also a lever attackers can pull. Researchers from Zhejiang University and Alibaba, presenting at ICML 2026 in Seoul, built a way to deliberately trigger "overthinking": feeding models prompts that look like real problems but are logically broken, so the model spirals into long, fruitless reasoning loops trying to solve something unsolvable.

The method itself is a genetic algorithm. Take 940 problems from three math benchmark datasets, break each into its component premises and final question using an LLM, then mutate them—swap premises between problems, delete some, add extras, or swap questions across unrelated problem sets. After each round, the team checks how many words a target model spits out and how often it uses telltale overthinking language like "but," "wait," "maybe," or "alternatively." The mutations that produce the longest, most confused responses survive to the next generation. Run this for five generations and you get prompts precision-tuned to make a model choke.

And it worked broadly. The attack hit DeepSeek-R1, Alibaba's own Qwen3-Thinking, OpenAI's GPT-o3, and Google's Gemini 2.5 Flash, all producing dramatically bloated responses. The worst case was DeepSeek-R1 on the MATH dataset—a set of high school competition problems—where the longest output ran 26.1 times longer than the model's longest response to an unmodified question. The team also tried the technique on coding, scientific reasoning, and dialogue tasks, not just math, and saw the same kind of blowup each time.　　

What makes this more than a lab curiosity is that it doesn't need access to a model's internals—querying the target from the outside is enough, which means closed commercial services aren't safe just because their weights are locked away. There's a catch: crafting these prompts against expensive reasoning models takes a lot of repeated querying, which isn't cheap. But the researchers showed that prompts generated using a smaller, cheaper model still transferred effectively to bigger targets, which is the detail that should worry providers most, since it lowers the cost of mounting this kind of attack considerably.

Wei Cao, a master's student on the team, was careful to frame this as a demonstration of a vulnerability rather than a ready-made denial-of-service weapon—pricing models, rate limits, context windows, and existing defenses would all shape how damaging a real attack could be. But the point stands: overthinking isn't a quirk of one company's model, it's a shared weakness baked into how reasoning models work, and that's exactly the kind of attack surface providers now need to account for.