Unsafe at any AUC: Unlearned Lessons from Sociotechnical Disasters for Responsible AI

arXiv cs.AI · 6 hours ago

A research paper argues that AI system design should learn from sociotechnical analyses of past industrial disasters like Chernobyl and Three Mile Island, where known risks were ignored due to social and organizational factors rather than technical unpredictability. The paper identifies three specific areas where AI development fails to apply these lessons: risk perception and communication at organizational level, traceability of requirements and responsibilities, and integration of social and organizational dynamics into safety engineering. The analysis suggests that responsible AI requires addressing systemic and organizational failures, not just technical reliability of individual components.