Estimating worst case frontier risks of open weight LLMs

OpenAI has a problem that closed-model labs don't: once you publish the weights, you can't take them back. So before shipping gpt-oss, its open-weight release, the company ran an experiment that amounts to attacking its own model on purpose. The method is called malicious fine-tuning, or MFT, and the idea is simple in concept even if it's expensive in practice — take the base model and push it, through targeted fine-tuning, toward the highest capability it can reach in two specific danger zones: biology and cybersecurity.

The logic here is that a determined bad actor with access to open weights isn't going to politely ask the model to help synthesize a pathogen or write exploit code. They'll fine-tune around whatever guardrails exist. So OpenAI's researchers essentially became that bad actor, using their own compute and expertise to see how far the model could be pushed before deciding whether releasing gpt-oss was too risky. This is a genuinely different posture than the usual red-teaming exercise, where testers poke at a model with clever prompts. MFT assumes the attacker isn't limited to prompting at all.

The paper doesn't just describe the technique, it treats this as a template for how the frontier-risk conversation around open weights should work going forward. If a lab wants to argue an open release is safe, the argument has to survive the assumption that someone will retrain the thing. That's a much higher bar than showing a chatbot refuses harmful requests in its default configuration, and it's the kind of scrutiny that open-weight advocates have been slow to apply to their own releases.

What's notable is less the specific numbers from the biology and cybersecurity domains — OpenAI is characteristically selective about publishing exact capability uplift figures for obvious reasons — and more that a major lab is formalizing worst-case testing as a prerequisite for open releases at all. Given how fast open-weight models from Meta, Mistral, and Chinese labs have been catching up to frontier capability, this kind of methodology, if it becomes standard practice industry-wide, would matter more than any single model's individual safety score.