arXiv cs.CL
·
18 hours ago
Researchers developed G-SHARE, a structured reasoning framework that applies the CNNP nine-step diagnostic guideline to analyze human-factor events in nuclear power plants by combining evidence extraction, stepwise reasoning, and consistency validation. The system was evaluated on a dataset of real human-factor event reports from Chinese nuclear facilities annotated by domain experts, achieving substantially higher accuracy and macro-F1 scores than one-shot language model prompting and traditional machine learning approaches. The framework enables explicit use of evidence and logical validation, demonstrating that operationalizing expert guidelines into auditable workflows improves diagnostic reliability in safety-critical environments.
arXiv cs.AI
·
18 hours ago
Researchers introduced Filtered Reasoning Score (FRS), a metric that evaluates the quality of reasoning in Large Language Models by analyzing their most-confident reasoning traces rather than just final accuracy. The metric assesses reasoning along dimensions including faithfulness, coherence, utility, and factuality, using only the top-K% most confident traces to avoid averaging over coincidental correct answers. Models with identical accuracy scores showed significant differences when evaluated with FRS, and models with higher FRS on one benchmark performed better on other reasoning benchmarks, suggesting FRS captures transferable reasoning capabilities beyond outcome-based evaluation.
arXiv cs.AI
·
18 hours ago
EZSMTV3, a new constraint answer set programming framework, combines answer set programming with constraint processing and satisfiability solving to handle complex combinatorial problems. The system uses existing SMT solvers like CVC5, YICES, and Z3 rather than implementing custom search procedures, and was benchmarked against competing CASP systems including CLINGCON and CLINGO variants. The framework's more expressive input language and support for optimization via weak constraints enable researchers to tackle mixed-domain constraint problems involving both integers and reals more readily.
arXiv cs.AI
·
18 hours ago
Researchers developed interventional grounding audits, a testing method that substitutes predicates in language model reasoning steps to check whether conclusions actually depend on stated premises. When applied to GPT-4o on 50 ProntoQA problems, the method achieved F1=0.806 in detecting proof-tree dependencies, outperforming a self-consistency baseline at F1=0.343. The work reveals that 66% of correctly-solved problems contain at least one reasoning step insensitive to direct dependencies, exposing cases where models reach right answers through faulty reasoning that passive evaluation methods miss.
