Workload-Driven Optimization for On-Device Real-Time Subtitle Translation

arXiv cs.CL · 6 hours ago

Researchers optimized a 0.6B-parameter language model for on-device English-to-Traditional-Chinese subtitle translation by replacing the 151k-token vocabulary with a 64k-token subtitle-specific tokenizer and fine-tuning the model. LocalSubs achieved a 59.2% win rate against Google Translate in pairwise evaluation and showed 1.63x faster inference on Apple M2 Metal compared to the original model. The optimization enables privacy-preserving, low-latency subtitle translation on consumer devices without relying on cloud services.