Choosing Problems in Data Science and Machine Learning

Eugene Yan · 5 years ago

A data science leader discusses how to prioritize which problems a team should tackle by using cost-benefit analysis, considering extent and severity of problems, and evaluating whether solutions are quantifiable. Key metrics include assigning dollar values to benefits (like $1.2 million yearly revenue from a notification system), accounting for development costs (team compensation plus infrastructure), and recognizing that capabilities and learning exercises reduce future costs by acting as multipliers. Teams should balance incremental improvements with disruptive solutions, avoiding pitfalls like resume-driven development where engineers choose technologies based on personal career benefit rather than business value.