How to choose a recommender
Input Data
What is the input?
RePlay models differ by types of data they can process:
Collaborative use only user-item interaction logs.
Content-based use only user or item features.
Hybrid can use both intercations and features.
Are interactions explicit?
Our information can be either explicit e.g. ratings, or implicit e.g. view, number of play counts.
Some models transform any type of data to implicit (unary ratings).
Will there be new users?
Some models need to be completely retrained to give predictions for new users (not present in train data, but have interactions history) while others don’t.
Will there be new items?
The same goes for new items.
Algorithm |
Data |
Interactions |
New Users |
New Items |
---|---|---|---|---|
Popular Recommender |
Collaborative |
converted to unary ratings |
+ |
- |
Popular By Users |
Collaborative |
implicit feedback |
- |
- |
Wilson Recommender |
Collaborative |
binary ratings |
+ |
- |
UCB |
Collaborative |
binary ratings |
+ |
+ |
KL-UCB |
Collaborative |
binary ratings |
+ |
+ |
Random Recommender |
Collaborative |
converted to unary ratings |
+ |
+ |
K-Nearest Neighbours |
Collaborative |
converted to unary ratings |
+ |
- |
Alternating Least Squares |
Collaborative |
implicit feedback |
- |
- |
Neural Matrix Factorization (Experimental) |
Collaborative |
converted to unary ratings |
- |
- |
SLIM |
Collaborative |
unary ratings, explicit feedback |
+ |
- |
ADMM SLIM (Experimental) |
Collaborative |
unary ratings, explicit feedback |
+ |
- |
Mult-VAE (Experimental) |
Collaborative |
converted to unary ratings |
+ |
- |
Word2Vec Recommender |
Collaborative |
converted to unary ratings |
+ |
- |
Association Rules |
Collaborative |
converted to unary ratings |
+ |
- |
Cluster Recommender |
Hybrid |
converted to unary ratings |
+ |
- |
LightFM Wrap (Experimental) |
Hybrid |
+ |
+ |
|
Implicit Wrap (Experimental) |
Collaborative |
- |
- |
|
Two Stages Scenario (Experimental) |
Hybrid |
converted to unary ratings for second level |
|
|
*
- depends on base models.
Model requirements
Should recommendations be personalized?
Should cold users get recommendations? (without any interactions).
Should model recommend cold items? (that no one interacted with).
Should model be able to recommend unseen items?
Algorithm |
Personalized |
Cold Users |
Cold Items |
Recommends Unseen Items |
---|---|---|---|---|
Popular Recommender |
- |
+ |
- |
+ |
Popular By Users |
+ |
- |
- |
- |
Wilson Recommender |
- |
+ |
- |
+ |
UCB |
- |
+ |
+ |
+ |
KL-UCB |
- |
+ |
+ |
+ |
Random Recommender |
- |
+ |
+ |
+ |
K-Nearest Neighbours |
+ |
+ |
- |
+ |
Alternating Least Squares |
+ |
- |
- |
+ |
Neural Matrix Factorization (Experimental) |
+ |
- |
- |
+ |
SLIM |
+ |
- |
- |
+ |
ADMM SLIM (Experimental) |
+ |
- |
- |
+ |
Mult-VAE (Experimental) |
+ |
- |
- |
+ |
Word2Vec Recommender |
+ |
- |
- |
+ |
Association Rules |
+ |
- |
- |
+ |
Cluster Recommender |
+ |
+ |
- |
+ |
LightFM Wrap (Experimental) |
+ |
+ |
+ |
+ |
Implicit Wrap (Experimental) |
+ |
- |
- |
+ |
Two Stages Scenario (Experimental) |
+ |
|
|
|
*
- depends on base models.
More info on models.
Model Comparison
All metrics are calculated at $k=10$
MovieLens 1m
Model |
HitRate |
MAP |
MRR |
NDCG |
Coverage |
Surprisal |
fit_pred_time |
---|---|---|---|---|---|---|---|
ALS (Implicit) |
0.686 |
0.172 |
0.419 |
0.265 |
0.133 |
0.163 |
10.128 |
LightFM |
0.699 |
0.167 |
0.431 |
0.263 |
0.151 |
0.168 |
10.833 |
SLIM |
0.669 |
0.172 |
0.419 |
0.261 |
0.04 |
0.124 |
12.456 |
ItemKNN |
0.655 |
0.166 |
0.409 |
0.256 |
0.056 |
0.138 |
17.964 |
LightFM (w/ feats) |
0.691 |
0.16 |
0.412 |
0.255 |
0.232 |
0.195 |
86.704 |
PopRec |
0.645 |
0.157 |
0.39 |
0.244 |
0.034 |
0.118 |
12.3 |
MultVAE |
0.64 |
0.151 |
0.396 |
0.238 |
0.031 |
0.123 |
26.977 |
NeuroMF |
0.627 |
0.111 |
0.318 |
0.193 |
0.257 |
0.235 |
350.737 |
ADMM SLIM |
0.591 |
0.084 |
0.304 |
0.159 |
0.367 |
0.237 |
77.647 |
Word2Vec |
0.515 |
0.072 |
0.244 |
0.138 |
0.145 |
0.24 |
25.133 |
Wilson |
0.414 |
0.045 |
0.181 |
0.092 |
0.017 |
0.262 |
10.034 |
RandomRec (popular) |
0.382 |
0.028 |
0.142 |
0.069 |
0.654 |
0.318 |
6.827 |
RandomRec (uniform) |
0.183 |
0.009 |
0.068 |
0.026 |
0.961 |
0.537 |
7.898 |
ALS (Explicit) |
0.093 |
0.005 |
0.033 |
0.013 |
0.266 |
0.684 |
13.876 |