We have Mythos at Home: GLM 5.2 beats Claude in our Cyber Benchmarks

Semgrep · 3 months ago · 13

Semgrep benchmarked GLM 5.2, an open-weight model from Zhipu AI, against Claude Code and other models on IDOR vulnerability detection, the same task used to evaluate frontier coding agents. GLM 5.2 achieved a 39% F1 score, beating Claude Code's 32% while costing roughly $0.17 per vulnerability found—one-sixth the price of comparable frontier models. The finding shows that vulnerability detection performance depends heavily on the harness scaffolding around a model, though GLM 5.2's strong performance as a bare-prompt open-weight model demonstrates teams need not rely on a single expensive frontier LLM for security tasks.