AI and efficiency

OpenAI Blog · 6 years ago

An analysis found that the compute required to train neural networks to fixed performance levels on ImageNet classification has halved every 16 months since 2012. Achieving AlexNet-level performance now requires 44 times less compute than in 2012, compared to an 11-fold improvement that Moore's Law would predict over the same period. The findings indicate that algorithmic improvements have driven more efficiency gains than hardware advances for heavily invested AI tasks.