Robust Explanations for User Trust in Enterprise NLP Systems

arXiv cs.AI · 18 hours ago

Researchers developed a black-box evaluation framework to test whether natural language model explanations remain stable when text is altered through perturbations like deletion and back-translation. The framework measured top-token flip rates across six models including BERT, RoBERTa, and Llama variants on three benchmark datasets with 64,800 test cases. Decoder-based large language models showed 73% lower explanation instability rates than encoder models on average, with stability improving 44% when scaling from 7 billion to 70 billion parameters, enabling organizations to select models with appropriate robustness-cost tradeoffs before deploying in regulated applications.