Back to work

Case study

Movie Recommendation

Pick a few films you like and get ranked recommendations that explain themselves, served as a static site.

Code on GitHub (opens in a new tab)Live demo (opens in a new tab)2.48× better top-10 ranking (NDCG@10) than recommending the most popular films
01

Problem

Recommend films from a handful of liked titles and say why each one was picked, without running a backend.

You search 9,742 films, pick up to 10 you liked, and their neighbour lists are added up in the browser. Every result says which of your films drove it, which genres and tags they share, and how much of its score came from co-ratings and how much from content. The selection lives in the URL fragment, so a result page can be shared, and nothing is sent anywhere[1]Source 1What it does, in the Movie Recommendation READMEOpen source 1 (opens in a new tab)All sources.

02

Data

The ratings are MovieLens ml-latest-small from GroupLens Research, last updated in September 2018. GroupLens recommends it for education and development, and says its latest datasets are not appropriate for reporting research results, so I treat the numbers here as a check on my own choices, not as a benchmark[2]Source 2MovieLens datasets at GroupLens ResearchOpen source 2 (opens in a new tab)All sources.

03

Constraints and approach

  • No backend. All modelling happens offline; the browser only adds up neighbour lists.
  • Every recommendation explains where it came from.
  • The offline numbers are measured on the same truncated, rounded lists the browser receives.
  • The TypeScript scorer has to give exactly the Python scorer’s results.

An offline Python pipeline tunes and evaluates a sentence-embedding model, item-kNN and a hybrid on MovieLens, then exports neighbor lists. The browser scores them, and a golden-file test keeps the TypeScript scorer identical to the Python one.

Three models and a baseline

For the content model each film becomes a short text built from its title, year, genres and up to eight of its most-applied user tags, with tags about viewing logistics such as “in netflix queue” dropped. That text is embedded with all-MiniLM-L6-v2, and each film keeps its nearest neighbours by cosine similarity. The collaborative model is item-kNN over who rated what, with a shrunk cosine, so two films that share two raters cannot look as similar as two that share two hundred; films with fewer than 3 ratings get no collaborative list. The hybrid gives each liked film’s co-rating list more weight the more ratings it has, and falls back to content for films with none. The baseline recommends the most-rated films the user has not rated. A golden file of expected results for several liked sets is exported by the pipeline and replayed by Vitest against the real export, so the two scorers cannot drift apart silently[3]Source 3How it works, in the Movie Recommendation READMEOpen source 3 (opens in a new tab)All sources.

Choices I measured

All 9,742 films with 10 neighbours each fit in 1.7 MB of JSON, about 0.7 MB gzipped, which removes the server and the hosting bill; the cost is a model frozen at build time. I searched the list length over 10, 15, 20 and 30 on validation, and 10 scored best for item-kNN while halving the export compared with 20. The hybrid weights came out close to a switch: a film with 3 ratings already puts 86% of its weight on co-ratings. MiniLM beat bge-small-en-v1.5 on validation (content NDCG@10 0.0219 against 0.0192) and is smaller. I chose item-kNN over ALS because its neighbour lists are the serving format and need no training loop[4]Source 4Design decisions and trade-offs, in the Movie Recommendation READMEOpen source 4 (opens in a new tab)All sources.

04

How I measured it

The split follows time. For each of the 610 users, the latest rating is the test item and the second-to-last is the validation item behind every tuning decision, so the test item is never seen while tuning. Models rank all 9,742 films minus those the user already rated, and the query is the user’s 5 most recent ratings, the setting closest to how the demo is used. One held-out item ranked against the full catalog, with no negative sampling, is a hard setup, which is why the absolute numbers are low. HitRate@10 comes with a 95% confidence interval. Coverage is the share of all films that appear in at least one user’s top 10. The tables reproduce byte for byte from a fresh download[5]Source 5Results and how to read them, in the Movie Recommendation READMEOpen source 5 (opens in a new tab)All sources.

05

Results

Hybrid NDCG@10 vs popularity baseline
2.48×
Hybrid HitRate@10 (popularity 0.043)
0.090
Films with a full list of 10
100%
Films in the catalog
9,742
NDCG@10 by modelQuery: each user’s 5 most recent ratings, 610 users. Higher is better.
NDCG at 10 of the popularity baseline, the content model, item-kNN and the hybrid on the temporal test split
SetupNDCG@10
Most popular films
0.019
Content (embeddings)
0.023
Item-kNN (co-ratings)
0.048
Hybrid
0.048
HitRate at 10 and catalog coverage by model
ModelHitRate@10 (95% CI)Coverage
Most popular films0.043 (0.028 to 0.059)1.2%
Content (embeddings)0.039 (0.025 to 0.054)27.4%
Item-kNN (co-ratings)0.088 (0.067 to 0.112)14.3%
Hybrid0.090 (0.067 to 0.113)16.2%

Co-ratings beat popularity by about 2.5× NDCG. Content alone barely beats popularity on accuracy, but it recommends far less popular films (16 ratings on average against 233) and covers 27.4% of the catalog. The hybrid and item-kNN are level; the hybrid’s gain is a full list of 10 for every film, against 51.1% of films for item-kNN alone. Using all of a user’s ratings as the query instead of the 5 most recent drops the hybrid to 1.60× popularity. The full evaluation output, with every one of the 214 tuning trials, is committed next to the code[6]Source 6Full evaluation output with every tuning trial (pipeline/eval/results.json)Open source 6 (opens in a new tab)All sources.

06

What did not work

The hybrid did not beat item-kNN. Their intervals overlap almost completely, so I report it as a coverage gain and not as a win. A user’s whole history also made a worse query than their latest few ratings, since a summed lifetime history drifts toward popular titles. Giving every liked film an equal say felt fairer, but on validation it cost about 8% NDCG@10 (0.0486 against 0.0528), so I kept raw scores. The first version of this project looked rows up by pandas label after filtering, and 45% of titles returned another film’s neighbours; now one sorted id array defines the rows, and tests check it on the exported files[7]Source 7Normalised scores and the id bug, in the README’s design decisionsOpen source 7 (opens in a new tab)All sources.

07

What I would do next

Two limits shape what comes next. In a mixed selection, a well-rated film’s co-rating scores tend to crowd out a rarely rated film’s content neighbours; rank-based fusion per liked film would fix that at some cost in offline accuracy, so it would need its own evaluation. And leave-last-out keeps other users’ later ratings in training, so the collaborative model sees some of the future, while 610 test users give wide intervals. After that I would try EASE or implicit ALS as the collaborative model, add a recency-weighted query and evaluate a cold-start slice on its own[8]Source 8Limitations and next steps, in the Movie Recommendation READMEOpen source 8 (opens in a new tab)All sources.

08

Sources