Reimagining the mouse pointer for the AI era

Google DeepMind · 3 months ago

Google is developing an AI-powered mouse pointer that understands context from what users point at, allowing them to request actions across any application without switching windows. The system works in Chrome starting today and will arrive in Googlebook laptops soon, using Gemini to interpret visual and semantic context from a user's pointer position. This eliminates the need for detailed text prompts, replacing them with simple gestures and natural speech like "Fix this" or "Show me directions."