Don't Mock Machine Learning Models In Unit Tests

Eugene Yan · 2 years ago · 43

A developer outlines best practices for unit testing machine learning code, arguing that ML models should often be tested with actual weights rather than mocked like traditional software dependencies. The author recommends using small sample data, testing against random or empty model weights when feasible, and marking critical tests against full models as slow-running to avoid loading billions of parameters during standard development. Unit testing ML code requires different approaches than conventional software because learned model logic cannot be easily isolated or mocked the way handcrafted code logic can be.