AI and efficiency
OpenAI Blog
An analysis found that the compute required to train neural networks to fixed performance levels on ImageNet classification has halved every 16 months since 2012. Achieving AlexNet-level performance now requires 44 times less compute than in 2012, compared to an 11-fold improvement that Moore's Law would predict over the same period. The findings indicate that algorithmic improvements have driven more efficiency gains than hardware advances for heavily invested AI tasks.
Why it matters
We’re releasing an analysis showing that since 2012 the amount of compute needed to train a neural net to the same performance on ImageNet classification has been decreasing by a factor of 2 every 16 months. Compared to 2012, it now takes 44 times less compute to train a neural network to the level of AlexNet (by contrast, Moore’s Law would yield an 11x cost improvement over this period). Our results suggest that for AI tasks with high levels of recent investment, algorithmic progress has yielded more gains than classical hardware efficiency.
