Visual Language Models Train Robots to Read Human Emotions
IEEE Spectrum AI Michelle Hampson
Researchers at Monash University trained a vision language model based on Gemini 2.5 to help robots recognize human emotions from facial expressions and contextual cues during collaborative tasks. The VLM achieved an emotion recognition score of 0.86 compared to 0.77 for conventional facial analysis systems, and 31 of 40 human participants preferred emotionally adaptive robot apologies over scripted ones. However, the study found that robots' emotional capabilities matter far less to humans than actual task competence, as participants' trust decreased regardless of how well the robot apologized after failing its physical responsibilities.
Why it matters
This article is part of our exclusive IEEE Journal Watch series in partnership with IEEE Xplore. As robots advance in terms of dexterity and other physical capabilities, it becomes more likely that humans may find themselves working alongside them. If that happens, how will robots’ emotional capabilities need to advance for them to successfully work with people?In a recent study, researchers trained collaborative robots to read human emotions by not only accounting for facial expressions, but also contextual factors in the interactions as well. Through experiments with 40 volunteers, the researchers then evaluated how a robot’s ability to read human emotions and adjust its behavior in turn impacted a human’s perception of the robot and its capabilities as the two collaborated on tasks. The results—which show that the emotional capabilities of robots only go so far with humans—were published 18 May in IEEE Robotics and Automation Letters.Seung Chan Hong led the study as part of his unde
