Serendipity: Accuracy’s Unpopular Best Friend in Recommenders
Eugene Yan
Recommendation systems typically optimize for accuracy metrics like NDCG and recall, but serendipity—measuring surprising and novel recommendations—matters more for long-term user engagement and business health. The article reviews metrics for measuring diversity, novelty, and unexpectedness across 10+ research papers, with formulas like cosine similarity and point-wise mutual information used to quantify how different recommendations are from user history and population trends. Balancing serendipity with relevance prevents user boredom, improves long-tail product sales, and can outperform accuracy-focused approaches in A/B tests despite worse offline evaluation scores.
Why it matters
What I learned about measuring diversity, novelty, surprise, and serendipity from 10+ papers.
