How to choose a recommender

Input Data

What is the input?

RePlay models differ by types of data they can process:

  • Collaborative use only user-item interaction logs.

  • Content-based use only user or item features.

  • Hybrid can use both intercations and features.

Are interactions explicit?

Our information can be either explicit e.g. ratings, or implicit e.g. view, number of play counts.
Some models transform any type of data to implicit (unary ratings).

Will there be new users?

Some models need to be completely retrained to give predictions for new users (not present in train data, but have interactions history) while others don’t.

Will there be new items?

The same goes for new items.

Algorithm

Data

Interactions

New Users

New Items

Popular Recommender

Collaborative

converted to unary ratings

+

-

Popular By Users

Collaborative

implicit feedback

-

-

Wilson Recommender

Collaborative

binary ratings

+

-

UCB

Collaborative

binary ratings

+

+

KL-UCB

Collaborative

binary ratings

+

+

Random Recommender

Collaborative

converted to unary ratings

+

+

K-Nearest Neighbours

Collaborative

converted to unary ratings

+

-

Alternating Least Squares

Collaborative

implicit feedback

-

-

Neural Matrix Factorization (Experimental)

Collaborative

converted to unary ratings

-

-

SLIM

Collaborative

unary ratings, explicit feedback

+

-

ADMM SLIM (Experimental)

Collaborative

unary ratings, explicit feedback

+

-

Mult-VAE (Experimental)

Collaborative

converted to unary ratings

+

-

Word2Vec Recommender

Collaborative

converted to unary ratings

+

-

Association Rules

Collaborative

converted to unary ratings

+

-

Cluster Recommender

Hybrid

converted to unary ratings

+

-

LightFM Wrap (Experimental)

Hybrid

depends on loss

+

+

Implicit Wrap (Experimental)

Collaborative

depends on model

-

-

Two Stages Scenario (Experimental)

Hybrid

converted to unary ratings for second level

*

*

* - depends on base models.

Model requirements

Should recommendations be personalized?

Should cold users get recommendations? (without any interactions).

Should model recommend cold items? (that no one interacted with).

Should model be able to recommend unseen items?

Algorithm

Personalized

Cold Users

Cold Items

Recommends Unseen Items

Popular Recommender

-

+

-

+

Popular By Users

+

-

-

-

Wilson Recommender

-

+

-

+

UCB

-

+

+

+

KL-UCB

-

+

+

+

Random Recommender

-

+

+

+

K-Nearest Neighbours

+

+

-

+

Alternating Least Squares

+

-

-

+

Neural Matrix Factorization (Experimental)

+

-

-

+

SLIM

+

-

-

+

ADMM SLIM (Experimental)

+

-

-

+

Mult-VAE (Experimental)

+

-

-

+

Word2Vec Recommender

+

-

-

+

Association Rules

+

-

-

+

Cluster Recommender

+

+

-

+

LightFM Wrap (Experimental)

+

+

+

+

Implicit Wrap (Experimental)

+

-

-

+

Two Stages Scenario (Experimental)

+

*

*

*

* - depends on base models.

More info on models.

Model Comparison

All metrics are calculated at $k=10$

MovieLens 1m

Model

HitRate

MAP

MRR

NDCG

Coverage

Surprisal

fit_pred_time

ALS (Implicit)

0.686

0.172

0.419

0.265

0.133

0.163

10.128

LightFM

0.699

0.167

0.431

0.263

0.151

0.168

10.833

SLIM

0.669

0.172

0.419

0.261

0.04

0.124

12.456

ItemKNN

0.655

0.166

0.409

0.256

0.056

0.138

17.964

LightFM (w/ feats)

0.691

0.16

0.412

0.255

0.232

0.195

86.704

PopRec

0.645

0.157

0.39

0.244

0.034

0.118

12.3

MultVAE

0.64

0.151

0.396

0.238

0.031

0.123

26.977

NeuroMF

0.627

0.111

0.318

0.193

0.257

0.235

350.737

ADMM SLIM

0.591

0.084

0.304

0.159

0.367

0.237

77.647

Word2Vec

0.515

0.072

0.244

0.138

0.145

0.24

25.133

Wilson

0.414

0.045

0.181

0.092

0.017

0.262

10.034

RandomRec (popular)

0.382

0.028

0.142

0.069

0.654

0.318

6.827

RandomRec (uniform)

0.183

0.009

0.068

0.026

0.961

0.537

7.898

ALS (Explicit)

0.093

0.005

0.033

0.013

0.266

0.684

13.876