Sakana AI’s LLM Peer Review System Catches 73% of Core-Claim Errors
MarkTechPost 4 days ago 21
Sakana AI published the TMLR paper Beyond Imitation, introducing an LLM-assisted peer review system called Multi-Layered Review that aims to catch inserted mistakes rather than mimic human reviews. It reported catching 73.43% of core-claim errors in its Contradiction Benchmark using 4 reviews, versus 14.81% for the best baseline. The release changes peer review evaluation by providing a planted-error benchmark plus a multi-agent, 3-pass workflow that improves error detection but still struggles with exact matches and can be affected by hidden prompt injection.