What is Recommendation Engine?

The three core approaches

Collaborative filtering uses behaviour patterns: people who interacted with similar things to you also liked this. It is powerful because it requires nothing about the items themselves and surfaces genuinely non-obvious connections. It fails completely on new users and new items, which have no interaction history.

Content-based filtering matches item attributes to a profile of what a user has liked. It handles new items immediately — a brand-new product still has a description and category, and therefore an embedding. Its weakness is a narrowing filter bubble, since it only ever recommends more of the same.

Hybrid approaches combine both and are what essentially every production system uses. The standard architecture is two-stage: a cheap candidate generation step narrows millions of items to a few hundred, then a more expensive ranking model orders those precisely. This exists for latency reasons — you cannot score a full catalogue per request — and it is worth designing for from the start.

Cold start, and the metric trap

Cold start has three variants and each needs a different answer. New user: fall back to popularity, use context such as device and location, or ask a few onboarding questions. New item: rely on content features until interaction data accrues, and deliberately allocate some exposure so it can accumulate any. New system: begin with content-based or popularity ranking and layer collaborative signals in as behaviour data arrives.

The more consequential trap is evaluation. Offline metrics — precision at k, NDCG, recall — are computed against logged historical behaviour, and that log only contains outcomes for items the previous system chose to show. A new model that would have recommended something better is scored as wrong, because no one ever saw it. Offline improvements therefore correlate weakly with live results, and this catches teams repeatedly.

Take offline metrics as a filter for obviously bad candidates, and treat online experimentation as the actual measurement. Interleaving and A/B tests measure what users do. And choose the target metric carefully: optimising click-through reliably produces clickbait, so pair engagement with retention, satisfaction, or long-run revenue.

Feedback loops and what to optimise

A recommender shapes the data it will later be trained on. Show an item more, it gets more interactions, it looks more popular, it gets shown more. Popularity bias compounds, the catalogue's long tail becomes invisible, and retraining encodes the bias more deeply each cycle — a specific instance of the loop problem described in model drift.

Counteracting this requires deliberate design rather than better accuracy. Reserve a share of impressions for exploration, whether through bandit-style allocation or simple randomisation. Log what was shown, not only what was clicked, so you can reason about exposure and correct for position bias. Measure catalogue coverage and diversity alongside accuracy.

Finally, keep business rules outside the model. Inventory limits, contractual promotions, content policy, and regional restrictions belong in a post-ranking layer where they are explicit, auditable, and changeable without retraining. Attempting to teach the model your commercial constraints produces a system nobody can adjust when those constraints change.