Building an early warning system for LLM-aided biological threat creation

OpenAI just published its first real attempt at measuring something that sounds like science fiction: can a chatbot make it easier for a regular person to create a biological weapon. The company ran an evaluation pulling together both biology experts and students, comparing what people could figure out with GPT-4's help versus using regular internet search alone. The headline finding is almost anticlimactic. GPT-4 gave at most a mild boost in accuracy for tasks tied to biological threat creation, nowhere near enough to call it a decisive risk.

But the point of this exercise was never really about GPT-4 itself. It's about building the measurement tools before a future model actually deserves the alarm. Biological weapons sit in a strange category for AI safety researchers, because unlike, say, generating malware, the failure mode here is catastrophic and largely untestable through normal red-teaming. You can't exactly let volunteers attempt a real anthrax synthesis to see how much a language model helped. So OpenAI is trying to construct a blueprint, a repeatable evaluation methodology, that can be pointed at GPT-5 or whatever comes next and produce something more rigorous than vibes.

What's notable is who OpenAI brought into the room. Actual biology experts and students, not just internal safety staff, tried to complete threat-relevant tasks with and without model assistance. That structure matters because it starts to separate two very different questions: does the model know dangerous things, and does the model make a person meaningfully faster or more accurate at using that knowledge. Right now, for GPT-4, the answer to the second question is a shrug. Mild uplift, inconclusive uplift, the kind of result that doesn't justify panic but also doesn't let anyone declare the problem solved.

OpenAI is explicit that this is a starting point, not a verdict. Which is the responsible way to frame it, honestly. Biosecurity researchers have warned for years that frontier models could eventually compress the tacit knowledge gap that keeps bioweapons out of reach for most people, the kind of practical know-how that used to require actual wet-lab experience. This evaluation doesn't prove that risk is imminent. It proves OpenAI now has a rough instrument to keep measuring it, model release after model release, instead of guessing.