Sakana AI released a reasoning benchmark based on Sudoku puzzles to test and improve AI models' logical reasoning capabilities. The benchmark includes thousands of curated traditional and modern Sudoku puzzles, with data extracted from thousands of hours of reasoning explanations from YouTube channel Cracking The Cryptic, where world-championship-level solvers narrate their step-by-step solving process. Current state-of-the-art models struggle significantly, with only OpenAI's o3 achieving a 5% success rate on the easiest puzzles, highlighting the gap between human-like reasoning and contemporary AI approaches.
Sakana AI developed EDINET-Bench, a Japanese financial benchmark for evaluating large language models on tasks like accounting fraud detection using securities reports from the Financial Instruments Exchange. The benchmark dataset contains approximately 41,000 securities reports spanning 10 years with about 600 labeled fraud cases, and was accepted to ICML 2026. Evaluation showed that state-of-the-art LLMs achieved only 0.7 ROC-AUC on fraud detection—comparable to classical logistic regression—revealing the difficulty of the task, though including textual information from reports improved performance.
Researchers introduced LVSum, a benchmark with 72 annotated videos averaging 16 minutes to evaluate how well multimodal large language models summarize long-form video content with temporal grounding. Testing showed that transcripts significantly outperform visual frames for summarization quality, with a substantial gap remaining between AI-generated and human summaries. Current MLLMs struggle with temporal grounding and cross-modal coherence, revealing systematic weaknesses in understanding the temporal sequencing of events in extended videos.
Every AI story that matters,
in your inbox by 8am.
TLDRocket reads all relevant sources, removes duplicate coverage, and summarises the
day in two minutes. Follow companies and topics for alerts, or get the
briefing in Slack. Free, no spam, unsubscribe anytime.
Reading TLDRocket needs no cookies, and the readership counts we rely on come from
our own cookieless analytics. Google Analytics is the exception: it sets cookies and
reports to Google, so it stays switched off until you allow it. You can change your
mind any time from “Cookie settings” in the footer.
Strictly necessary
Session security and form protection (tldrocket-session,
XSRF-TOKEN, 2 hours). The site cannot work without them,
so they need no consent.
Always on
Google Analytics 4 (_ga,
_ga_<id>, up to 2 years). Measures which
stories and sections readers use. Google acts as a third-party processor and may
store the data outside the EU. No advertising, no profiling, no data sold.
