Getting Started with Hugging Face Transformers for IPUs with Optimum

Hugging Face just widened the hardware playing field for its Optimum library, adding support for Graphcore's Intelligence Processing Unit. BERT is the first model to get the IPU treatment, with Graphcore engineers reworking it to run on their IPU-POD16 systems through the familiar Transformers interface.

The pitch here is accessibility. Most companies don't have the engineering headcount to hand-optimize a transformer model for exotic silicon, and Optimum is designed to paper over that gap. Instead of rewriting BERT from scratch for Graphcore's Poplar SDK, developers install optimum[graphcore], clone the optimum-graphcore repo, and point a script called run_qa.py at a question-answering task. The whole setup runs through Graphcloud, Graphcore's cloud platform, with PopTorch handling the PyTorch-to-IPU translation underneath.

The numbers from Hugging Face's own test run are worth a look. Fine-tuning bert-base-uncased on SQuAD 1.1 for three epochs took just over six minutes — 368 seconds — processing 88,524 training samples at a rate of nearly 721 samples per second. The resulting model hit an exact-match score of 80.66 and an F1 of 88.28 on 10,784 validation samples, numbers that land squarely in the range you'd expect from a properly tuned BERT base model on that dataset.

What matters more than the specific benchmark is the workflow. The IPUTrainer class slots into the same training loop Hugging Face users already know, just swapping the hardware target underneath. That's the real bet Optimum is making: that developers will adopt alternative AI accelerators faster if switching to them costs a config file change rather than a rewrite. Graphcore says BERT is just the opening act, with vision, speech, translation and text-generation models slated to get the same treatment in the months ahead.