Data Science Project Quick-Start
Eugene Yan
A veteran data scientist shares hard-won rules for starting projects right. Skip them, and you pay in production headaches later.
Based on reporting by Eugene Yan — read the original for the full story.
Summary, retelling and take written by AI under human oversight; images are AI-generated illustrations. How we work · Report an error
Eugene Yan opens with a confession: early in his career, he'd jump straight into model training the moment someone handed him a problem. No context, no success metrics, no thought about reproducibility. The result wasn't speed — it was technical debt that showed up right when the project hit production and stakes got high.
His fix starts before any code gets written. Ask why the project matters now, and what "success" actually means to the business side. He walks through a recommendation-engine example: a product manager wants better "engagement," but whether that means clicks or purchases changes everything about how you label training data and what you optimize for. Context matters too — a feature store built for the whole org needs to be robust and documented, while one built to bridge a temporary gap can be scrappy and short-lived. Treating both requests the same wastes effort in one direction or under-delivers in the other.
Yan also pushes for constraints early, even unglamorous ones like latency budgets, GPU availability, or a hard cap on false positives in fraud detection. He frames this as liberating rather than limiting: clear boundaries narrow the search space so teams stop wasting cycles on solutions that could never ship anyway. Before locking in targets, he recommends digging into the data and even building a quick, rough baseline — sometimes in a day or two — because that's often when you discover a 95% recall target is fantasy when your first pass barely clears 60%.
The piece closes on the practical mechanics: talk to domain experts who already do the task manually, scavenge GitHub and Hugging Face for prior art instead of reinventing algorithms, and keep experiment pipelines clean using tools like Hyperopt, Optuna, MLflow, and Papermill. None of this is flashy. But Yan's argument is that the boring groundwork — intent, constraints, data sanity checks — is what separates a prototype that quietly dies after a demo from one that actually survives contact with production.
My take — AI-written commentary, not fact-checked reporting
This is the kind of advice that sounds obvious until you watch a team skip every step of it and wonder why their model works in the notebook and nowhere else. The real signal here is cultural, not technical: most data science failures aren't about algorithms, they're about nobody agreeing on what 'success' meant before the GPUs started spinning. If more teams treated a day of data-poking and a scrappy baseline as mandatory instead of optional, we'd waste a lot fewer research budgets chasing metrics nobody asked for.
Read more about this at: Eugene Yan