Control an Android Phone with Gemini 3.5 Flash Computer Use

Philipp Schmid · 3 months ago · 48 ● 3 sources

Google's Gemini 3.5 Flash model includes a native Computer Use feature that lets developers control Android devices by having the model analyze screenshots and issue commands like taps and text input through ADB. The implementation uses a loop where Gemini receives a screenshot and goal, returns structured function calls (click at y=300 x=500), the developer's code executes those actions via ADB, and sends back a new screenshot for the model to continue. This enables fully automated task completion on Android emulators or physical devices, with supported actions including app launching, typing, swiping, long-pressing, and key presses on a normalized 0-999 coordinate grid.