StreetReaderAI: Towards making street view accessible via context-aware multimodal AI

Google Research · 8 months ago

Researchers developed StreetReaderAI, a prototype system that uses multimodal AI to make street view imagery accessible to blind and low-vision users through audio descriptions and conversational AI. In a lab study with 11 blind users who explored over 350 panoramas and made over 1,000 AI requests, the system answered 86.3% of questions correctly with a median usefulness rating of 7 on a 7-point scale. Future work aims to add autonomous exploration capabilities, full route planning, and richer 3D audio interfaces to expand accessibility.