Design: Video Recommendations

Clipstream (the made-up video platform from module 35) wants a personalised home feed. Clarify:

• scale: 50,000,000 daily users, about 4 home-feed opens each; hundreds of millions of videos; thousands of hours uploaded daily;
• surfaces: the home feed (open-ended browsing) and 'up next' after a video (strongly shaped by what was just watched);
• latency: the feed must render within about 200 ms;
• goal: people find videos they value and come back; not just more minutes today;
• constraints: safety and policy rules; creators, especially new ones, need a fair chance at an audience.

The goal statement matters more here than anywhere: a recommender optimised for the wrong thing changes what millions of people watch.

• Home feed and 'up next' have different intents and usually get different models or features.

Candidate objectives, and what each rewards:

• click-through: easy to measure; rewards thumbnails that mislead;
• watch time: rewards content people actually watch; can favour long videos regardless of value;
• satisfaction: survey answers ('was this worth your time?'), likes, shares, and negative signals (dislikes, 'not interested', quick abandonment);
• long-term return: whether people come back over weeks; the real goal, but slow and noisy.

Most mature systems predict several of these and combine them (module 14).

Each step down the list is closer to real value but harder to measure.

• Predicting several targets and blending them lets the platform balance immediate engagement against satisfaction, and adjust the balance by experiment.

The system learns from logged behaviour:

• impressions with position and surface (module 5);
• watches: start, duration, completion share, rewatches;
• explicit signals: likes, dislikes, shares, subscriptions, 'not interested', 'don't recommend this channel';
• surveys: a small random sample of users asked to rate recommended videos;
• content: titles, transcripts, frame embeddings, categories (from module 35's pipeline).

Watch duration is plentiful but must be normalised: a 3-minute music video watched fully is a stronger signal than 3 minutes of a 60-minute documentary.

• Survey responses are rare but are the closest thing to a ground truth for satisfaction, so they are used to train a satisfaction model whose predictions become a ranking signal.

Several sources, merged (module 13):

• two-tower retrieval: a user tower over watch history, searches, demographics and context; a video tower over video ID and content; nearest neighbours in an ANN index;
• co-watch: videos often watched after videos the user just watched (strong for 'up next');
• subscriptions: new uploads from followed channels;
• fresh and trending: recent uploads gaining traction in the user's region and languages;
• exploration: a small quota of new creators and topics.

No single source covers everything.

• Two-tower retrieval generalises to unseen combinations.
• Co-watch captures sequences ('part 2').
• Subscriptions honour explicit choices.
• Exploration prevents the feedback loop from closing (module 22).

A video platform's best content is often its newest, but models trained on history are biased towards old videos that have accumulated data. Remedies:

• include the age of the training example as a feature during training, and set it to zero (now) at serving time, so the model learns how popularity changes over time and predicts for the present;
• a fresh index and fresh-upload candidate source;
• exploration traffic for new uploads to gather early signals quickly.

Without this, the model keeps recommending what was popular when its training data was collected.

• The 'example age' trick is a simple, published fix that lets a model separate 'this video is good' from 'this video was popular last month'.

The ranker scores a few hundred candidates with a multi-task model (module 12):

• shared layers learn from all signals;
• separate heads predict engagement (click, watch duration) and satisfaction (like, survey-predicted satisfaction, dismissal);
• a mixture-of-experts design lets heads share some experts and not others, so conflicting tasks interfere less;
• a small position tower learns how much position alone drives clicks during training, and is switched off at serving, to remove position bias;
• a final score combines the heads with weights tuned by experiment.

Engagement and satisfaction partly conflict; a shared model with separate heads learns both efficiently.

• Removing position bias inside the model is cleaner than correcting labels afterwards.

Before showing the feed:

• diversity: limit consecutive videos from one channel or topic;
• policy: remove or demote content that breaks rules or is borderline, regardless of predicted engagement;
• creator fairness: guarantee some exposure for new creators via exploration slots;
• freshness mix and already-watched filtering.

Item-level scores cannot express list-level goals.

• Policy decisions must be hard rules, readable and auditable, not patterns the ranker might learn.

Keeping the feed fast at this scale:

• precompute user embeddings and candidate pools in batch for active users, refreshed through the day;
• update session features live as the user watches (what was just watched matters most for 'up next');
• rank in one batched call on GPU-backed model servers with dynamic batching (module 19);
• fallback to precomputed candidates in retrieval order, then to regional trending.

Precomputation moves the heavy work off the request path; session features keep it current.

• At tens of millions of users, small per-request savings are large in total cost.

Measure at several horizons:

• offline: per-head AUC and calibration; NDCG on held-out sessions; recall of each candidate source; by user type (new, light, heavy);
• online: A/B tests on satisfied watch time, survey satisfaction, dismissals, and return visits; at least two weeks for novelty effects;
• long-term holdouts for the cumulative effect of many launches;
• ecosystem health: share of watch time going to new creators; diversity of topics per user over time.

A recommender shapes the data it learns from (module 22).

• Ecosystem metrics catch narrowing that per-session metrics reward.
• Long-term holdouts reveal whether short-term wins add up to people coming back.

Public work this design draws on:

• Covington, Adams and Sargin, 'Deep Neural Networks for YouTube Recommendations', RecSys 2016: two-stage design, next-watch prediction, example age, watch-time ranking;
• Zhao et al., 'Recommending What Video to Watch Next: A Multitask Ranking System', RecSys 2019: mixture-of-experts ranking and a position-bias tower;
• Yi et al., 'Sampling-Bias-Corrected Neural Modeling for Large Corpus Item Recommendations', RecSys 2019: two-tower retrieval with frequency correction;
• Chen et al., 'Top-K Off-Policy Correction for a REINFORCE Recommender System', WSDM 2019: learning from logged recommendations.

These are some of the most cited industrial recommendation papers; knowing their key ideas is expected in senior recommendation interviews.

The questions interviewers ask about the video recommendation design, from the first screening call to the senior round. Cover the answer, say yours aloud, then compare.

Reading an idea and explaining it are different skills.

• Each answer below is the two or three sentences an interviewer is listening for.
• The reasoning behind each one is in the cards above. If an answer surprises you, go back to that card.

A short build that uses this module's ideas. Write it yourself before looking anything up; the checks at the end tell you whether it works.

An idea you have only read fades in a week.

• Building it once, even in fifty lines, is what makes it yours.
• Interviewers can tell the difference between someone who has read about a loop and someone who has debugged one.