Block-sparse GPU kernels

OpenAI Blog · 8 years ago

Optimized GPU kernels for block-sparse neural networks have been released, enabling faster computation than standard libraries like cuBLAS and cuSPARSE. The kernels achieved state-of-the-art performance on text sentiment analysis and generative image and text modeling tasks. Networks using these kernels can execute orders of magnitude faster depending on their sparsity configuration.