Personalizing Incremental Video Search with Hybrid Text and ID Embeddings

Typing to search on a TV remote is miserable enough without the results being useless until you've spelled out half the title. Apple's ML research team tackled that specific pain point: incremental search, where the system has to rank results after every keystroke, often with just one to three characters of intent to work with. Their fix blends two different ways of understanding what a user might want — a multilingual text encoder fine-tuned on co-engagement data, and a separate collaborative filtering model built from ID-based interaction patterns. Both get folded into a pairwise XGBoost ranker alongside a user's recent watch history.

The headline offline numbers are respectable but not dramatic: a 2.99% bump in NDCG@10 and a 3.30% gain in MRR for sessions where the system has some user history to draw on. But the real story shows up when the researchers slice the data by query length. On those brutal one-to-three-character prefixes, where intent is still forming and generic ranking has almost nothing to grab onto, NDCG@10 jumped 8.63%. Once queries got longer and more specific, that lift shrank to just 1.46%. Personalization, in other words, earns its keep precisely in the moment when search is at its weakest.

A similar pattern held for watch history depth. Users with only one to five items in their history saw a 2.13% NDCG lift; users with fifty-one to one hundred items saw 4.37%. That's notable because the baseline ranker actually performs worse for those long-history users to begin with — NDCG@10 for that cohort drops from 0.733 to 0.680 without personalization. So the system isn't just rewarding people who already had good default results; it's clawing back ground exactly where the non-personalized baseline falls short.

Apple also ran this live, for three weeks, not just in offline simulation. Tap-through rate rose 1.14%, conversion rate rose 1.23%, and the rank position of the item users ultimately picked improved by 2.91% — all statistically significant. The team says they also probed the trade-off between the semantic text embeddings and the collaborative ID embeddings through ablation tests, and checked embedding quality using LLM-judged similarity labels specifically to sidestep the click and exposure bias that tends to creep into these evaluations.

None of these numbers are earth-shattering on their own. A few percentage points here and there rarely make headlines. But the pattern across every slice — ambiguous queries, deep watch histories, live traffic — points the same direction: personalization pays off hardest exactly where search infrastructure has traditionally struggled, at the ambiguous, half-typed, underspecified moment before intent crystallizes.