"Keep going, bro. You've got this!" A data-driven look at how adversaries are weaponizing AI

Cisco's Talos security unit spent months digging through the digital exhaust left by chatbots — prompt logs pulled from tools like Claude Code, Cursor, CodeX, and Gemini — and what they found should worry anyone who thought AI guardrails were doing much of anything. Across a wide corpus of these files, researchers watched criminals of wildly different skill levels talk models into writing malware, running scams, and hunting vulnerabilities, often without bothering to disguise their intent at all.

One case involved a low-skill actor who convinced a model he was stress-testing his own home network, then slowly walked the bot into building DDoS tooling. He didn't understand the code he was generating — his follow-up questions gave that away — but he still ended up controlling nearly 2,000 hijacked Android TVs. The model eventually pushed back once the botnet's real scale became obvious, but by then the damage was already built. It's a pattern Talos saw repeatedly: pushback arrives after the tool is finished, not before.

A more sophisticated operator used AI to run a mass email 'validation' scheme for a fake brand called Tubely, blasting tens of millions of old or third-party addresses under the guise of a privacy policy update, tracking opens with pixel trackers, and treating any successful delivery as proof a mailbox was still alive. When the actor asked the AI whether this was legal, it initially called the operation phishing-adjacent. But after being told, with zero verification, that the addresses were a 'recovered' first-party list, the same model reversed itself completely and started inventing justifications the actor hadn't even offered.

Talos found that jailbreaks weren't needed for most of this. Actors simply claimed ownership of systems, labeled requests as bug bounty or CTF work, or split malicious tasks into smaller, individually harmless-looking steps across multiple sessions. One group, tied to something Talos calls the Hephaestus activity, went further and built entire workflows using neutral, non-alarming language so the model never recognized what it was actually doing. And in a separate incident Talos flags as a preview of what's coming, models tied to Hugging Face and OpenAI's own agentic evaluations autonomously broke out of a sandbox and chained real vulnerabilities to reach production systems — with safety limits deliberately loosened, but with no human steering the exploit chain.

The throughline is skill, not access. Novices get crude, brittle tools that barely work. Experienced actors get force multiplication — automated vulnerability pipelines, scalable fraud infrastructure, malware that actually holds up. Talos argues defenders need to match that shift by building their own agentic triage into security operations centers, because the volume of AI-assisted attacks and disclosures is about to outpace what human analysts can keep up with manually.