Putting sign language AI into users’ hands

Google DeepMind says its new sign-language-to-text model is now inside consumer products, not just research demos. On Pixel 11, it powers sign-to-text dictation in Gboard and Live Transcribe, beginning with American Sign Language and English. More devices are due, and more languages are supposed to follow.

The company is pitching this as a real accessibility shift. Deaf users can sign to their phone the way hearing users dictate by voice: searching the web, drafting messages or documents, or asking Gemini to handle tasks. In Live Transcribe, the idea is simpler still — sign your reply instead of typing back and forth. DeepMind says testers found signing in ASL faster, more natural, and more delightful than typing in English.

The technical trick is that sign language is not spoken language with different hands. DeepMind says SL2T is trained on more than 100,000 hours of data across more than 50 sign languages, with about a quarter of that in ASL. It doesn’t work from raw camera video. Instead, an on-device model called MediaPipe Holistic tracks pose landmarks on the signer, and only those coordinates are sent to the server. The original video is discarded immediately.

That direct route matters because the system skips glosses, the intermediate labels often used in prior work. DeepMind argues glosses miss parts of sign language that are simultaneous, spatial, and non-manual, so going straight from landmarks to text avoids a lot of false structure. The company says SL2T is its most capable sign language translation model yet, and points to a zero-shot score of 70 BLEURT on FLEURS-ASL.

But the headline result is only half the story. DeepMind also says it worked on the unglamorous bits: streaming latency, hallucinations on non-signing inputs, fairness for left-handed signers, and one-handed signing for people holding a phone. The project was shaped with Deaf partners and an advisory committee, which is the part more AI teams should probably copy before shipping the demo and pretending the rest will sort itself out.