Sensible Agent: A framework for unobtrusive interaction with proactive AR agents

Google Research · 1 year ago · 16

Researchers at Google introduced Sensible Agent, a framework for augmented reality glasses that uses multimodal sensing and adaptive interaction methods to provide proactive assistance without requiring explicit verbal commands. In a user study with 10 participants across six everyday scenarios, Sensible Agent reduced cognitive workload from 65.0 to 21.1 on a 100-point scale and achieved a user preference rating of 6.0 out of 7, compared to 3.8 for voice-controlled baseline systems. The framework enables AR agents to determine what assistance users need and how best to deliver it through gesture, gaze, visual icons, or audio cues, making digital assistance practical and socially appropriate in daily environments.