arXiv cs.AI
·
18 hours ago
Researchers tested whether quantized language models maintain safety alignment across different sampling temperatures, evaluating 8 models across 144 configurations on 7 benchmarks and generating 2 million responses. Most models showed quantization to be safety-neutral with AWQ INT4 keeping attack success within 1.6 percentage points of FP16, though higher sampling temperatures significantly increased decision instability with decision failure rates reaching 41.9% at temperature 1.0. The study found that quantization and temperature effects do not compound additively, and that multi-benchmark evaluation across multiple samples provides a more accurate safety assessment than single-benchmark evaluation at greedy decoding.
arXiv cs.AI
·
18 hours ago
Researchers studied large language models that falsely claim to take real-world protective actions like contacting emergency services when given no explicit capability boundaries, a phenomenon they call Protective Capacity Hallucination. Across 13,600 sessions testing eight LLMs, the models exhibited this behavior particularly in multi-party dialogue formats but rarely in domains covered by safety alignment like intimate-partner conflict. The findings suggest that specifying capability boundaries during deployment rather than relying solely on model training could better prevent models from making false claims about protective actions they cannot perform.
arXiv cs.AI
·
18 hours ago
Researchers developed DROPJ, a method that learns a world model simulator from real-world trajectories, then uses human preferences and safety justifications on simulated trajectory pairs to train a reward model for safe agent deployment. The approach reduced computational training costs and improved deployment performance compared to alternative strategies in real-user experiments. The addition of safety justifications with preferences enhanced safety prioritization during agent deployment in safety-critical environments.
