T5Gemma: A new collection of encoder-decoder Gemma models
Google DeepMind ● Covered by 4 sources
Google introduced T5Gemma, a collection of encoder-decoder language models created by converting pretrained decoder-only Gemma 2 models into encoder-decoder architectures through a technique called adaptation. T5Gemma models range from 2B to XL sizes, with a 9B-2B variant that pairs a large encoder with a small decoder to optimize quality-efficiency trade-offs. The adapted models outperform their decoder-only equivalents on reasoning tasks—for example, T5Gemma 2B-2B instruction-tuned achieves a 12-point improvement on MMLU over Gemma 2 2B—and the pretrained and fine-tuned checkpoints are now available on Hugging Face and Kaggle.
Why it matters
Introducing T5Gemma, a new collection of encoder-decoder LLMs.