EDINET-Bench: A Japanese Financial Benchmark Using Securities Reports
Sakana AI 51
Sakana AI developed EDINET-Bench, a Japanese financial benchmark for evaluating large language models on tasks like accounting fraud detection using securities reports from the Financial Instruments Exchange. The benchmark dataset contains approximately 41,000 securities reports spanning 10 years with about 600 labeled fraud cases, and was accepted to ICML 2026. Evaluation showed that state-of-the-art LLMs achieved only 0.7 ROC-AUC on fraud detection—comparable to classical logistic regression—revealing the difficulty of the task, though including textual information from reports improved performance.
