DataTalksClub - Building an ML System; Behind the Scenes

Eugene Yan · 5 years ago

An engineer presented a case study on building a machine learning system for a Southeast Asian hospital group, detailing the data pipeline from hospital servers through model training to production endpoints. The system used time-based validation splits rather than random splits, separate models for each hospital instead of a single unified model, and emphasized that machine learning comprises less than 20% of the effort while engineering practices and domain expertise provided the majority of impact. Key improvements came from consulting hospital staff and administrators, implementing proper version control and monitoring via ELK stack and Airflow, and focusing on understanding the problem before optimization.