Design a recommendation system end to end. How would you generate candidates, rank them, evaluate quality and handle cold start?
Interview preparation resource from Gate Smashers.
I would use a multi-stage system: collect consented interaction and item data, generate a few hundred candidates using collaborative, content-based and popularity sources, filter ineligible items, rank candidates with a model using user, item and context features, then apply diversity, freshness and policy rules. I would evaluate retrieval and ranking offline, validate impact through controlled online experiments, and monitor latency, coverage, feedback loops and segment quality. Cold-start users and items need metadata, popularity, exploration and onboarding signals.
Recall@K measures how many relevant items were retrieved in the top K candidates.
NDCG rewards relevant items near the top of a ranked list and normalizes by the ideal ordering.
| Stage | Purpose | Typical methods |
|---|---|---|
| Candidate generation | Reduce a large catalog to hundreds of items | Two-tower retrieval, collaborative filtering, similarity, popularity |
| Eligibility filtering | Remove invalid or unsafe items | Availability, policy, block and history rules |
| Ranking | Estimate utility and order candidates | Learning-to-rank, classification or regression models |
| Re-ranking | Apply list-level quality and constraints | Diversity, freshness, deduplication, quotas |
Begin with the objective
Define the recommendation surface and success event before selecting an algorithm. Watch time, click-through rate, purchase probability, retention and user satisfaction lead to different systems. Include guardrails such as complaints, hides, cancellations, diversity and latency so the model does not optimize a narrow proxy at the expense of users.
Candidate generation
Retrieving from the full catalog at request time is usually too expensive. Candidate generators produce a manageable set using methods such as item-to-item similarity, collaborative filtering, two-tower embeddings, content similarity, recent popularity, followed creators and business-specific sources. Multiple generators improve coverage.
Filtering and ranking
Remove blocked, unavailable, already-consumed or policy-ineligible items. A ranking model then predicts utility for each user–item–context tuple. Features may include user history, item attributes, freshness, similarity, context and calibrated outputs from other models. A final re-ranking stage can enforce diversity, freshness, deduplication, creator limits and business constraints.
Feedback and training data
Log impressions, positions, scores, model versions and subsequent actions. Clicks are exposure-biased: an item cannot be clicked if it was never shown, and position affects the probability of interaction. Negative sampling, debiasing, exploration and carefully defined labels are essential.
Cold start
- New user: Use onboarding interests, location or context where appropriate, popular high-quality items and controlled exploration.
- New item: Use metadata and content embeddings, creator information and a small exploration budget.
- Sparse domain: Blend collaborative signals with content and popularity rather than relying on a single method.
Evaluation
Offline retrieval metrics include Recall@K and coverage. Ranking metrics include NDCG@K, MAP and calibrated task metrics. Offline improvements are not sufficient: run A/B tests with primary and guardrail metrics, and inspect new-user and long-tail segments.
Reliability and scale
Precompute embeddings and indexes, cache stable candidates, use approximate nearest-neighbour search, set latency budgets for each stage and provide a safe popularity fallback. Monitor catalog coverage, score drift, repeated content, delayed feedback, filter rates and online outcomes.