Netflix Prize (2009) – Finding the best algorithm to predict user film ratings (en.wikipedia.org)

The Netflix Prize (2006–2009) was an open competition to improve Netflix’s Cinematch collaborative-filtering recommender by predicting user movie ratings from a large anonymized dataset (100.48M ratings from 480,189 users on 17,770 movies). Entrants submitted predicted ratings for a 2.817M “qualifying” set (split into quiz and test subsets) and were scored by RMSE; the $1M grand prize required a 10% RMSE improvement over Cinematch (target test RMSE 0.8572). After years of progress prizes and thousands of teams, BellKor’s Pragmatic Chaos won in 2009 with a test RMSE of ~0.8567 (quiz RMSE ~0.8558), edging out a rival ensemble that had comparable quiz performance. Technically the contest drove major advances in large-scale recommender methods: matrix-factorization and temporal/implicit extensions, heavy use of ensembles/blending (winners combined dozens to hundreds of predictor models—earlier joint teams reported 207 predictors), careful data-conditioning, and strategies to avoid leaderboard overfitting via probe/quiz/test splits. The dataset’s scale, sparsity, and deliberate privacy perturbations also highlighted practical engineering challenges. Beyond the cash prize, the contest shaped research and industry practice by demonstrating the practical power of model ensembling, the importance of evaluation design (and RMSE’s limits for ranking quality), and the value of public, realistic benchmarks for advancing ML systems.