Getting Started with Transformers on Habana Gaudi

Hugging Face Blog · 4 years ago

Hugging Face and Habana Labs partnered to enable Transformer model training on Habana Gaudi accelerators, which deliver 40% better price performance than GPU-based Amazon EC2 instances. Fine-tuning a BERT model on the MRPC task completed in 2 minutes and 12 seconds with an F1 score of 0.9181 on a dl1.24xlarge instance. Users can now efficiently train Transformer models on Gaudi hardware at reduced cost, with Spot Instances further lowering expenses to $3.93 per hour compared to the standard $13.11 per hour rate.