System Design for Recommendations and Search

Eugene Yan · 5 years ago · 42

Industrial recommendation and search systems typically split work into offline batch processes (model training, embedding creation, index building) and online serving (candidate retrieval and ranking). Candidate retrieval uses embeddings with approximate nearest neighbor indexes to reduce millions of items to hundreds in under 100 milliseconds, while ranking then scores these candidates using deep learning models with richer features. Implementations from Alibaba, Facebook, JD, and DoorDash demonstrate this pattern across different retrieval methods (embeddings, graphs, decision trees), with practical optimizations like unified query embedding plus ANN lookup in single service instances to reduce latency.