Chat Templates: An End to the Silent Performance Killer

Hugging Face Blog · 2 years ago

Hugging Face has added a chat_template attribute to tokenizers that stores the exact formatting a chat model was trained with, preventing silent performance degradation when users accidentally use mismatched formats. The feature uses Jinja templates to convert conversation histories into correctly formatted strings, eliminating the need to manually code formatting logic or hunt through documentation. This shifts responsibility for format specification from the transformers library to individual model repositories, allowing developers maximum flexibility while ensuring users apply the correct preprocessing their models expect.