TLDRocket
Sign in

Hardware & Infrastructure

261 summarised stories in Hardware & Infrastructure, each linking back to the original source. Browse all topics →

Thursday, 4 November 2021

Scaling up BERT-like model Inference on modern CPU - Part 2

Hugging Face 4 years ago 42

Intel's Ice Lake Xeon CPUs achieve up to 75% faster inference on NLP tasks compared to the previous Cascade Lake generation through hardware improvements like new instructions and PCIe 4.0 support. The performance gains come from software optimizations including Intel's oneAPI libraries (oneMKL, oneDNN, oneTBB) and framework-specific tuning of memory allocation, parallelization, and mathematical operators. Users can extract these improvements by enabling oneDNN in TensorFlow via an environment variable or through PyTorch's native integration, along with tuning parallelization settings like OpenMP configuration.

The daily briefing

Every AI story that matters, in your inbox by 8am.

TLDRocket reads all relevant sources, removes duplicate coverage, and summarises the day in two minutes. Follow companies and topics for alerts, or get the briefing in Slack. Free, no spam, unsubscribe anytime.