Cross-Dataset Generalization in Urdu Fake News Detection: An Empirical Study with XLM-RoBERTa and a Length Confound Analysis

arXiv cs.CL · 6 hours ago

Researchers conducted the first cross-dataset generalization study for Urdu fake news detection using XLM-RoBERTa on two public datasets containing 10,083 and 13,388 articles. Transfer from Notri-Fact to Ax-to-Grind achieved macro F1 of 0.771, but reverse transfer collapsed to F1 of 0.005 because fake articles in Ax-to-Grind averaged 117 words versus 35 for real articles, a 3.4x length asymmetry causing the model to exploit this shortcut. The study demonstrates that identifying such dataset confounds through bidirectional transfer testing is essential for developing robust multilingual fake news detection systems.