Is AI progress slowing down?

AI Snake Oil · 1 year ago

Three leading AI developers reported difficulties scaling up their next-generation models, causing industry leaders like Ilya Sutskever to pivot from declaring model scaling dead to promoting inference scaling as the path forward. The reports from OpenAI, Anthropic, and Google Gemini emerged within weeks of each other, with no new fundamental evidence that scaling had actually failed. Industry insiders' rapid narrative flip reveals they lack reliable foresight and are influenced by commercial incentives, while inference scaling shows promise for specific problem classes like coding and mathematics but faces fundamental limitations for tasks like writing and translation where improvement depends on training data rather than reasoning time.