Reimagining the mouse pointer for the AI era
Google DeepMind
Google DeepMind is testing a smarter mouse pointer that uses Gemini to understand what you're pointing at, not just where. Point at something, say what you want, and skip the typing — this could kill the copy-paste dance between apps and AI chat windows.
Based on reporting by Google DeepMind — read the original for the full story.
Summary, retelling and take written by AI under human oversight; images are AI-generated illustrations. How we work · Report an error
The mouse pointer hasn't really changed since it showed up on screens decades ago. It moves, it clicks, it occasionally spins to tell you something's loading. Google DeepMind researchers Adrien Baranes and Rob Marchant think that's a missed opportunity, and they're using Gemini to turn the pointer into something that actually understands context — not just coordinates on a screen.
The pitch is simple: right now, using AI means dragging your work into some separate chat window, typing out what you want, and hoping the model gets the context you didn't fully explain. DeepMind wants to flip that. Point at a photo of a building, say "show me directions," and the system figures out the rest because it can already see what you're pointing at. No copy-pasting, no describing the image in text, no switching windows.
The team built this around four principles. Keep users in their flow instead of forcing an "AI detour" — point at a PDF, ask for a summary, paste it straight into an email. Let the system see what matters instead of demanding a wall of text — hover over a chart, ask for a pie version, done. Let people talk the way they actually talk to each other, using "this" and "that" instead of full sentences, since pointing and shared context fill in the rest. And treat what's on screen as real entities — dates, places, objects — rather than dumb pixels, so a blurry photo of a handwritten note becomes an actual clickable to-do list.
This isn't just a research paper sitting on a shelf. Google's already shipping pieces of it. Gemini in Chrome now lets you point at part of a webpage and ask about it directly, comparing products or previewing a couch in your living room instead of typing a paragraph describing what you see. A feature called Magic Pointer is coming to the new Googlebook laptop, and there's a public demo running in Google AI Studio right now if you want to poke at it yourself. Google Labs' Disco project will apparently keep testing further variations.
What's notable here is the framing: DeepMind isn't selling this as a smarter chatbot, but as an attempt to make AI disappear into the interface rather than living in its own box. Whether that actually reduces friction or just adds a new layer of "is the AI understanding my point correctly" anxiety is the real open question.
My take — AI-written commentary, not fact-checked reporting
I like this more than most AI-interface announcements because it's attacking a genuinely boring, overlooked problem instead of promising another chatbot that writes your emails. The idea that AI should meet you inside your existing workflow rather than demanding you feed it context through a text box is obviously correct, and it's a little embarrassing it took this long for a major lab to bother. Still, Google controlling both the pointer and the model behind it is a closed-loop bet — I'd rather see this land as an open interaction standard than another feature locked to Chrome and a new laptop nobody asked for.
Read more about this at: Google DeepMind