Reimagining Independence: How Meta’s AI Models Are Helping the University of Pittsburgh Transform Assistive Robotics
Meta AI Blog
The University of Pittsburgh's RAMMP project is integrating Meta's AI models (DINOv3 and SAM) into assistive robotic systems to enable users to control devices through natural language and vision, detecting objects like door buttons and cups in real-time. The team optimized these models to run efficiently on battery-powered edge devices, reducing memory footprint and using lower precision where needed while maintaining reliability in unpredictable everyday environments. This allows assistive robot users to interact with their surroundings more naturally without complex interfaces or network connectivity, improving their ability to perform daily activities independently.
Why it matters
For people relying on assistive devices, every second counts. The unpredictable nature of everyday environments, such as a child darting across a sidewalk, the sudden appearance of a curb, or a dropped set of keys, requires an immediate reaction. Processing camera images and sensor data directly on the device, known as edge computing, empowers robotic mobility platforms to function as responsive tools. These tools must be robust and consistent across the wide range of dynamic environments in which people live.
