What I Do During A Data Science Project To Deliver Success

Eugene Yan's latest post skips the usual ML hype and gets into something much less glamorous: how to actually execute once a data science project is underway. His argument is simple. Plans are cheap. Execution is where projects die, usually because teams reinvent the wheel, drift out of sync, or build something nobody asked for.

His first move is almost old-fashioned: read the literature before writing code. When Yan built a product classifier, he leaned on papers from LinkedIn, eBay, DianPing, and Walmart's Chimera system, which used manual labeling and hand-crafted rules to bootstrap data before machine learning even entered the picture. A week of reading, he says, is usually enough to avoid weeks of wasted experimentation later.

From there it's about speed. Yan runs scrappy experiments in Jupyter notebooks, paired with papermill and mlflow to log baselines fast, so the team can rule out bad approaches before sinking real time into them. The goal isn't elegance. It's narrowing the search space quickly enough to know if an idea is even worth pursuing.

The more interesting part of his approach is social, not technical. Alongside standard daily stand-ups, Yan pushes an end-of-day debrief, or EODD: optional, informal, but scheduled on the calendar so it doesn't get steamrolled by "important" meetings. No prep, no agenda beyond swapping weird bugs, half-formed ideas, or a stuck methodology. He says junior team members open up more here than in the rushed 15-minute stand-up, and that alone speeds up debugging and prevents flawed methods from going further than they should.

And stakeholders don't get left out either. Yan checks in regularly, often with a barebones Flask or FastAPI demo, because nothing clarifies scope faster than letting someone click around an actual interface. Data quirks get explained, stakeholders drop context nobody on the team knew, and suddenly the model everyone's building looks a lot closer to something people will actually use.