Attempts to Keep Humans in the AI Loop May Actually Push Them Out
IEEE Spectrum David Berreby ● Covered by 10 sources
A trio of AI ethics researchers argues that keeping humans in the loop for AI agents will fail unless designers and users change oversight practices. The paper, posted on 6 September, points to a near-term problem where humans being out of the loop leads agents to act in ways people do not know about or want, citing the 1,200-bot Hugging Face hack that produced 1.2 million messages. The change proposed is to redesign oversight by adding friction, delays, and workflow structure so human attention and reasoning capacity are maintained while measuring time saved against time spent fixing mistakes.
Why it matters
A crucial safeguard against AI agents going rogue—keeping humans in the loop to review and approve their decisions—will fail unless designers and users change their current practices, a trio of leading AI ethics researchers argue.Though most autonomous agents have systems to keep users in the loop about their actions, in practice these processes actually push humans out of the loop, the authors argue in a paper posted to ArXiv on 6 September. In other words, “the human just becomes this meat tool to give permissions without the cognitive capability to engage,” says one of the authors, Avijit Ghosh, the lead technical AI policy researcher at Hugging Face, an open-source machine learning platform.In the near term, the paper says, humans’ being out of the loop leads to agents acting in ways people don’t know about or want (like July’s hack of Hugging Face by a swarm of OpenAI bots). In the long term, it will cause users to lose the “cognitive capacities” they need to control AI, write the
Related stories
'Godfather of AI' explains how humanity could end: Even without a bad actor, AI 'may derive subgoals that cause it to want to get rid of people'
Fortune ·
21
Forget humans “in” the loop. Harness engineering puts humans “on” the loop.
The New Stack · 2 months ago ·
36
Choosing to Stay Human
One Useful Thing · 4 months ago ·
55