Profiling in PyTorch (Part 1): A Beginner's Guide to torch.profiler

Hugging Face Blog · 1 month ago

PyTorch's torch.profiler module provides tools to analyze performance bottlenecks by exporting profiler tables showing execution time statistics and profiler traces showing temporal operation sequences across CPU and GPU. Running a 64×64 matrix multiplication and addition operation showed the GPU idle 99% of the time due to launch overhead, but increasing to 4096×4096 matrices shifted the workload from overhead-bound to compute-bound, with GPU kernel execution time rising from 23 microseconds to 4.495 milliseconds. Understanding these profiling artifacts—tables for identifying hotspots and traces for investigating kernel dispatch chains and CPU-GPU synchronization gaps—enables developers to systematically identify and address performance constraints in PyTorch models.