Accelerating Gemini Nano models on Pixel with frozen Multi-Token Prediction

Google Research · 2 weeks ago

Google announced a frozen Multi-Token Prediction architecture for Gemini Nano models on Pixel 9 and 10 devices that speeds up on-device text generation by attaching a lightweight prediction head to the existing model without retraining it. The approach achieves 50% or more speedup compared to standalone drafters and uses a zero-copy architecture that saves 130MB of memory per instance by leveraging the main model's cached computations. This enables faster execution of features like AI Notification Summaries and Proofread with reduced energy consumption and battery drain.