Profiling in PyTorch (Part 1): A Beginner's Guide to torch.profiler

Hugging Face · 3 months ago · 12

PyTorch's torch.profiler module provides tools to analyze performance bottlenecks by exporting profiler tables showing execution time statistics and profiler traces showing temporal operation sequences across CPU and GPU. Running a 64×64 matrix multiplication and addition operation showed the GPU idle 99% of the time due to launch overhead, but increasing to 4096×4096 matrices shifted the workload from overhead-bound to compute-bound, with GPU kernel execution time rising from 23 microseconds to 4.495 milliseconds. Understanding these profiling artifacts—tables for identifying hotspots and traces for investigating kernel dispatch chains and CPU-GPU synchronization gaps—enables developers to systematically identify and address performance constraints in PyTorch models.