We have Mythos at Home: GLM 5.2 beats Claude in our Cyber Benchmarks
TLDR Dev
Semgrep benchmarked GLM 5.2, an open-weight model from Zhipu AI, against Claude Code and other models on IDOR vulnerability detection, the same task used to evaluate frontier coding agents. GLM 5.2 achieved a 39% F1 score, beating Claude Code's 32% while costing roughly $0.17 per vulnerability found—one-sixth the price of comparable frontier models. The finding shows that vulnerability detection performance depends heavily on the harness scaffolding around a model, though GLM 5.2's strong performance as a bare-prompt open-weight model demonstrates teams need not rely on a single expensive frontier LLM for security tasks.
Why it matters
Semgrep's recent tests showed that the GLM 5.2 open-weight model outperformed Claude Code in detecting Insecure Direct Object References (IDORs) despite lacking the advanced harnessing capabilities used by Semgrep's multimodal pipeline.
