Improving quality and robustness in LLM-based text-to-speech systems
Amazon Science
Amazon researchers addressed three problems in LLM-based text-to-speech systems: accent leakage in multilingual voice cloning, limited expressiveness, and reliability issues like hallucinations and cutoffs. They used locale-specific fine-tuning with LoRA, classifier-free guidance for prosody, and chain-of-thought reasoning to predict phonemes and duration before generation, reducing critical errors to less than one second per hour on long-form text. These techniques improved speech quality scores by 5% to 20% across nine language locales compared to their previous model.
Why it matters
Low-rank adaptation, data augmentation, and chain-of-thought reasoning are among the techniques enabling accent-free polyglot outputs, improved expressiveness, and reliable synthesis.
