Mechanisms for Effective Machine Learning Projects

Eugene Yan · 3 years ago · 19

An engineer describes four organizational mechanisms for improving machine learning project success: assigning a pilot and copilot to each project where the copilot spends approximately 10% effort reviewing the pilot's work, conducting literature reviews before starting implementation, performing methodology reviews to catch data leaks and evaluation errors, and timeboxing project phases with constraints reduced by 10-20% from similar past projects. The copilot approach requires the copilot to spend roughly one hour per ten hours the pilot works on the project. These practices help prevent months-long off-track work and critical errors that would otherwise require costly rework.