Using LLMs to Secure Source Code

Security review has always been one of those tasks everyone agrees matters and almost nobody has time to do properly. Eugene Yan's writeup lays out a five-step pipeline for handing chunks of that work to an LLM: build a threat model, discover vulnerabilities, verify them, triage by severity, then patch. It sounds almost too simple written out in five verbs, but the structure is the point.

The threat model step matters more than it looks. Instead of pointing a model at a repo and asking "find bugs," you first get it to reason about what an attacker would actually want — what data is sensitive, what boundaries exist between trusted and untrusted input, where the money or the credentials live. That framing narrows the search space and, in theory, cuts down on the kind of generic, low-value findings that make automated scanners exhausting to use.

Discovery and verification are kept as separate stages on purpose. An LLM flagging something as a vulnerability is not the same as it being one; language models are famously happy to call a defensive check a flaw or miss context that makes a pattern safe. Splitting discovery from verification forces a second pass where the model — or a human, or both — has to justify why a finding is real before it goes any further, which is basically how a decent human pentest report gets written anyway.

Triage and patching close the loop. Not every real vulnerability deserves the same urgency, and Yan's framing pushes toward ranking issues by actual exploitability and blast radius rather than treating every finding as equally on fire. Patching last, rather than fixing as you go, keeps the fix separate from the hunt, so a rushed patch doesn't get shipped before anyone's checked whether the underlying threat model was even right.

What's notable is how conventional this all is. There's nothing here an appsec engineer wouldn't recognize from a normal audit process — it's the standard discipline, just with an LLM slotted into the labor-intensive middle steps. That's arguably the more honest pitch than promising an AI that autonomously secures your codebase: it's a faster, cheaper assistant for a process that already existed, not a replacement for the judgment at either end of it.