EDINET-Bench: A Japanese Financial Benchmark Using Securities Reports

Sakana AI

Sakana AI developed EDINET-Bench, a Japanese financial benchmark for evaluating large language models on tasks like accounting fraud detection using securities reports from the Financial Instruments Exchange. The benchmark dataset contains approximately 41,000 securities reports spanning 10 years with about 600 labeled fraud cases, and was accepted to ICML 2026. Evaluation showed that state-of-the-art LLMs achieved only 0.7 ROC-AUC on fraud detection—comparable to classical logistic regression—revealing the difficulty of the task, though including textual information from reports improved performance.

Why it matters

EDINET-Bench: 有価証券報告書を用いた日本語金融ベンチマークの公開