39 Lessons on Building ML Systems, Scaling, Execution, and More
Eugene Yan
A practitioner shared 39 lessons learned from attending ML conferences in 2024, covering building effective ML systems, production scaling, team execution, and customer-focused product development. Key concrete points include that reward function engineering is half the battle for ML systems, LLM costs drop two orders of magnitude in 18 months, and each 10x increase in scale uncovers new operational issues. The lessons emphasize investing in robust evaluation frameworks, starting simple before adding complexity, and balancing big vision with careful attention to implementation details across cross-functional teams.
Why it matters
ML systems, production & scaling, execution & collaboration, building for users, conference etiquette.
