Compressing Streaming Neural Audio Encoders via Latent-Space Distillation

Apple Machine Learning Research · 3 weeks ago · 25

Researchers compressed streaming neural audio tokenizers by distilling a teacher’s continuous pre-quantizer latent into a student encoder instead of matching discrete tokens or output distributions. At 2.8× compression, the student stayed within 1.9% relative WER of the teacher on 5 of 6 pairs without fine-tuning. The result is a single distillation recipe that works across token interfaces and yields better accuracy than an independently trained tokenizer at the same capacity.