An Intelligent-Cloud Edge Multimodal Interaction System for Robots

arXiv cs.AI · 6 hours ago

Researchers developed a cloud-edge system for robots that combines an improved gesture detection model with language and vision models to enable human-robot interaction in complex environments. The enhanced YOLO detector achieved 98.9% precision on a public gesture dataset and 90.7% mAP@0.5, while the full system reached 95% success rate on single-action tasks. The architecture distributes computation between cloud servers and the TonyPi robot, allowing accurate gesture recognition and task planning despite limited onboard resources.