LibRec Examples on Real Data Sets

& comparison with other recommendation libraries

Quick Jump
Remarks
  1. Comparative recommendation packages include MyMediaLite (v3.10, C#, MMLite for short) and PREA (v1.2, Java).
  2. General settings: rand.seed=1, evaluation.setup=cv -k 5 -p on --test-view all -bold-driver.
  3. The reported results may not represent the best performance an algorithm can achieve,
    while results with * symbol indicate the closeness to those reported by the reference paper.
  4. The reported results may not be updated when a new version of LibRec has been released.
  5. Generally, LibRec runs much faster than the others while achieving competitive performance.

Rating Prediction

MovieLens (1M)

Algorithm

 

 MAE RMSE Train Time (s) Test Time (s)
MMLite PREA LibRec MMLite PREA LibRec MMLite PREA LibRec MMLite PREA LibRec
GlobalAvg 0.934 0.934 0.934 1.117 1.118 1.117 00:00 00:00 00:00 00:00 00:03 00:00
UserAvg 0.829 0.828 0.829 1.035 1.035 1.036 00:00 00:00 00:00 00:00 00:03 00:00
ItemAvg 0.782 0.782 0.782 0.979 0.979 0.980 00:00 00:00 00:00 00:00 00:03 00:02
UserKNN 0.683 0.696 0.703 N/A 0.893 0.905 N/A 00:00 06:34 N/A (20:53)x5 19:18
neighbors=80, similarity=pcc, shrinkage=25 (PREA inapplicable); MMLite: throws an out of memory (OOM) exception
ItemKNN 0.681 0.677 0.688 0.869 0.863 0.876 01:22 00:00 02:15 01:18:04 (02:40:03)x5 05:10
neighbors=80, similarity=pcc, shrinkage=10 (PREA inapplicable); MMLite: reg_u=25, reg_i=10
SlopeOne 0.711 0.711 0.711 0.901 0.901 0.901 05:29 (00:30)x5 00:14 01:30 (00:15)x5 00:10
RegSVD 0.674 0.675 0.671 0.857 0.855 0.852 00:58 (02:33)x5 00:04 00:00 (00:03)x5 00:00
factors=10, reg=0.05, learn.rate=0.005, max.iter=75
PMF N/A 0.762 0.747 N/A 0.943 0.926 N/A (13:42)x5 00:13 N/A (00:03)x5 00:00
factors=10, reg=0.4, learn.rate=50, momentum=0.8, max.iter=200
NMF N/A 0.733 0.727 N/A 0.927 0.920 N/A (02:42)x5 01:08 N/A (00:04)x5 00:00
factors=100, max.iter=5
BPMF N/A 0.708 0.707 N/A 0.899 0.899 N/A (00:14)x5 02:02 N/A (00:03)x5 00:00
factors=2, max.iter=20; PREA only uses 1 iter for Gibbs where LibRec uses 2 iters per epoch
BPMF N/A 0.704 0.695 N/A 0.903 0.896 N/A (16:55)x5 07:56 N/A (00:04)x5 00:00
factors=20, max.iter=50; PREA uses sparse matrix/vector while LibRec uses dense versions
BiasedMF 0.676 N/A 0.671 0.854 N/A 0.854 04:50 N/A 00:17 00:00 N/A 00:00
factors=40, reg=0.06, learn.rate=0.07, max.iter=110; MMLite: biag_reg=0.001, bold_driver=true
BiasedMF 0.676 N/A 0.671 0.854 N/A 0.854 05:41 N/A 00:27 00:00 N/A 00:00
factors=60, reg=0.06, learn.rate=0.07, max.iter=100; MMLite: biag_reg=0.001, bold_driver=true
BiasedMF 0.676 N/A 0.671 0.853 N/A 0.853 07:08 N/A 00:35 00:00 N/A 00:00
factors=80, reg=0.06, learn.rate=0.07, max.iter=100; MMLite: biag_reg=0.001, bold_driver=true
BiasedMF 0.675 N/A 0.670 0.853 N/A 0.852 10:05 N/A 00:52 00:00 N/A 00:00
factors=120, reg=0.06, learn.rate=0.07, max.iter=100; MMLite: biag_reg=0.001, bold_driver=true
SVD++ 0.667 N/A 0.668 0.851 N/A 0.851 02:11:18 N/A 01:52:35 00:00 N/A 00:05
factors=10, reg=0.05, learn.rate=0.005, max.iter=80
SVD++ 0.668 N/A 0.668 0.853 N/A 0.852 04:23:45 N/A 02:45:03 00:00 N/A 00:06
factors=20, reg=0.05, learn.rate=0.005, max.iter=80
Algorithm MAE RMSE Train Test Configuration
URP 0.706 0.892* 04:40 00:00 factors=9, burn.in=300, max.iter=500, sample.lag=10, alpha=2, beta=0.5
GPLSA 0.723 0.921* 12:02 00:00 factors=10, burn.in=30, max.iter=100, sample.lag=10, q=10
LDCC 0.721 0.909 54:08 00:00 ku=kv=5, burn.in=1000, max.iter=2000
LDCC 0.718 0.903* 54:43 00:00 ku=kv=5, burn.in=1000, max.iter=2000 (evaluation=train-ratio -r 0.99)

MovieLens (100K)

Algorithm MAE RMSE Train Time (s) Test Time (s)
MMLite PREA LibRec MMLite PREA LibRec MMLite PREA LibRec MMLite PREA LibRec
GlobalAvg 0.945 0.949 0.945 1.126 1.128 1.126 00:00 00:00 00:00 00:00 00:00 00:00
UserAvg 0.835 0.838 0.835 1.041 1.043 1.042 00:00 00:00 00:00 00:00 00:00 00:00
ItemAvg 0.817 0.823 0.817 1.024 1.030 1.025 00:00 00:00 00:00 00:00 00:00 00:00
PD N/A N/A 0.794 N/A N/A 1.094 N/A N/A 00:00 N/A N/A 14:26
sigma=2.5
UserKNN 0.721 0.732 0.737 0.921 0.937 0.944 00:02 00:00 00:05 01:38 (00:20)x5 00:03
neighbors=60, shrinkage=25, similarity=pcc; MMLite: reg_u=12, reg_i=1
ItemKNN 0.703 0.716 0.723 0.899 0.914 0.924 00:03 00:00 00:05 02:00 (01:47)x5 00:06
neighbors=40, shrinkage=2500, similarity=pcc; MMLite: reg_u=12, reg_i=1
SlopeOne 0.739 0.740 0.739 0.939 0.940 0.940 00:15 (00:01)x5 00:00 00:03 00:00 00:00
RegSVD 0.741 0.730 0.730 0.949 0.932 0.936 00:07 (00:16)x5 00:00 00:00 00:00 00:00
factors=10, reg=0.05, learn.rate=0.005, max.iter=100
BiasedMF 0.724 N/A 0.722 0.918 N/A 0.918 00:06 N/A 00:00 00:00 N/A 00:00
factors=5, reg=0.1, learn.rate=0.07, max.iter=100; MMLite: bias_reg=0.1, reg_u=reg_i=0.1, bold_driver=true
BiasedMF 0.724 N/A 0.721 0.917 N/A 0.916 00:09 N/A 00:00 00:00 N/A 00:00
factors=10, reg=0.1, learn.rate=0.07, max.iter=100; MMLite: bias_reg=0.1, reg_u=0.1, reg_i=0.12, bold_driver=true
BiasedMF 0.720 N/A 0.718 0.911 N/A 0.911 00:40 N/A 00:06 00:00 N/A 00:00
factors=80, reg=0.1, learn.rate=0.07, max.iter=100; MMLite: bias_reg=0.003, reg_u=0.09, reg_i=0.1, bold_driver=true
BiasedMF 0.720 N/A 0.719 0.911 N/A 0.911 00:58 N/A 00:09 00:00 N/A 00:00
factors=120, reg=0.1, learn.rate=0.07, max.iter=100; MMLite: bias_reg=0.003, reg_u=0.09, reg_i=0.1, bold_driver=true
BiasedMF 0.720 N/A 0.720 0.910 N/A 0.911 01:17 N/A 00:20 00:00 N/A 00:00
factors=160, reg=0.1, learn.rate=0.07, max.iter=100; MMLite: bias_reg=0.003, reg_u=0.08, reg_i=0.1, bold_driver=true
BiasedMF 0.720 N/A 0.720 0.911 N/A 0.911 12:23 N/A 01:05 00:00 N/A 00:00
factors=320, reg=0.1, learn.rate=0.07, max.iter=500; MMLite: bias_reg=0.007, reg_u=reg_i=0.1, bold_driver=true
SVD++ 0.713 N/A 0.719 0.908 N/A 0.912 04:23 N/A 03:40 00:00 N/A 00:00
factors=5, reg=0.1, learn.rate=0.01, max.iter=100; MMLite: bias_learn_rate=0.07, reg=1, bias_reg=0.05, fre_reg=true
SVD++ 0.713 N/A 0.718 0.909 N/A 0.912 08:26 N/A 04:33 00:00 N/A 00:00
factors=10, reg=0.1, learn.rate=0.01, max.iter=100; MMLite: bias_learn_rate=0.07, reg=1, bias_reg=0.005, fre_reg=true
SVD++ 0.716 N/A 0.718 0.912 N/A 0.911 16:46 N/A 06:33 00:00 N/A 00:00
factors=20, reg=0.1, learn.rate=0.01, max.iter=100; MMLite: bias_learn_rate=0.07, reg=1, bias_reg=0.005, fre_reg=true
Algorithm MAE RMSE Train Test Configuration
UserCluster 0.945 1.126 00:01 00:00 factors=1, max.iter=5 (When factors=1, it is equivalent as GlobalAvg and ItemCluster)
UserCluster 0.839 1.048 03:05 00:00 factors=10, max.iter=30
ItemCluster 0.820 1.023 02:09 00:00 factors=10, max.iter=30
URP 0.805 0.998 01:35 00:00 factors=10, burn.in=1400, max.iter=2000, sample.lag=10, alpha=2, beta=0.5
URP 0.792 0.984 02:02 00:00 factors=10, burn.in=1400, max.iter=2000, sample.lag=10, alpha=1, beta=0.5
BUCM 0.870 1.056 03:07 00:00 factors=30, burn.in=1400, max.iter=2000, sample.lag=10, alpha=2, beta=0.5, gamma=0.5
BUCM 0.847 1.038 02:55 00:00 factors=30, burn.in=1400, max.iter=2000, sample.lag=10, alpha=1, beta=0.5, gamma=0.5
GPLSA 0.792 1.012 00:44 00:00 factors=10, burn.in=30, max.iter=100, sample.lag=10, q=5, validation=0.1
GPLSA 0.807 1.030 01:39 00:00 factors=20, burn.in=30, max.iter=100, sample.lag=10, q=5, validation=0.1
GPLSA 0.789 1.008 00:44 00:00 factors=10, burn.in=30, max.iter=100, sample.lag=10, q=10, validation=0.1
LDCC 0.735 0.928 59:16 00:00 ku=20, kv=19, burn.in=2000, max.iter=5000, sample.lag=500 (perplexity = 3.290148*)
LDCC 0.743 0.937 32:58 00:00 ku=10, kv=20, burn.in=2000, max.iter=5000, sample.lag=500 (perplexity = 3.308719)
LDCC 0.736 0.929 39:33 00:00 ku=20, kv=10, burn.in=2000, max.iter=5000, sample.lag=500 (perplexity = 3.285468)

Epinions (665K)

Algorithm MAE RMSE Train Time (s) Test Time (s)
MMLite PREA LibRec MMLite PREA LibRec MMLite PREA LibRec MMLite PREA LibRec
GlobalAvg 0.918 0.918 0.918 1.207 1.207 1.207 00:00 00:00 00:00 00:00 00:00 00:00
UserAvg 0.928 0.937 0.930 1.199 1.209 1.203 00:00 00:00 00:00 00:00 00:01 00:00
ItemAvg 0.825 0.898 0.825 1.094 1.163 1.094 00:00 00:00 00:00 00:00 00:01 01:04
BiasedMF 0.814 N/A 0.817 1.048 N/A 1.056 00:28 N/A 00:02 00:00 N/A 00:00
factors=5, reg=0.35, learn.rate=0.01, max.iter=30; MMLite: reg=3.5, bias_reg=0.01
SocialMF 0.897 N/A 0.825 1.239 N/A 1.070 19:40 N/A 52:18 00:00 N/A 00:00
factors=5, reg=0.001, reg.social=1, learn.rate=0.05, max.iter=200; MMLite: learn_rate=0.01 (as 0.05 cannot train)
SocialMF NaN N/A 0.825 NaN N/A 1.062 NaN N/A 52:39 00:00 N/A 00:00
factors=5, reg=0.001, reg.social=5, learn.rate=0.05, max.iter=200
SocialMF 0.903 N/A 0.826 1.243 N/A 1.082 31:52 N/A 01:13:22 00:00 N/A 00:00
factors=10, reg=0.001, reg.social=1, learn.rate=0.05, max.iter=200; MMLite: learn_rate=0.01, others as default
Algorithm MAE RMSE Train Test Configuration
PMF 0.979 1.290   00:00 factors=5, reg=0.01, learn.rate=60, moment=0.8, max.iter=200
PMF 0.909 1.197   00:00 factors=10, reg=0.01, learn.rate=60, moment=0.8, max.iter=200
SoRec 0.880 1.110 01:02 00:00 factors=5, reg=0.001, reg.c=1, learn.rate=0.001, max.iter=100
SoRec 0.886 1.144 01:39 00:00 factors=10, reg=0.001, reg.c=1, learn.rate=0.001, max.iter=100
SoReg 0.994 1.315 08:36 00:00 factors=5, reg=0.001, beta=0.1, learn.rate=0.001, max.learn.rate=0.001, max.iter=600
SoReg 0.932 1.232 08:58 00:00 factors=10, reg=0.001, beta=0.1, learn.rate=0.001, max.learn.rate=0.001, max.iter=600
RSTE 0.950 1.196 06:22:56   factors=5, reg=0.001, learn.rate=0.001, alpha=0.4, max.iter=100
RSTE 0.958 1.278 05:38:21 00:01 factors=10, reg=0.001, learn.rate=0.001, alpha=0.4, max.iter=100
SocialMF 0.822 1.067 01:12:41 00:00 factors=10, reg=0.001, reg.social=5, learn.rate=0.05, max.iter=200
TrustMF (Tr) 0.826 1.075 01:25 00:00 factors=5, reg=0.001, reg.social=1, learn.rate=0.05, max.iter=200
TrustMF (Te) 0.828 1.078 01:24 00:00 factors=5, reg=0.001, reg.social=1, learn.rate=0.05, max.iter=200
TrustMF (T) 0.818 1.069 03:25 00:00 factors=5, reg=0.001, reg.social=1, learn.rate=0.05, max.iter=200
TrustMF (Tr) 0.827 1.104 01:51 00:00 factors=10, reg=0.001, reg.social=1, learn.rate=0.05, max.iter=200
TrustMF (Te) 0.829 1.107 01:50 00:00 factors=10, reg=0.001, reg.social=1, learn.rate=0.05, max.iter=200
TrustMF (T) 0.819 1.095 04:15 00:00 factors=10, reg=0.001, reg.social=1, learn.rate=0.05, max.iter=200
SVD++ 0.818 1.057 20:23 00:00 factors=5/10, reg=0.35, learn.rate=0.01, max.iter=100
TrustSVD 0.804 1.047 17:10 00:01 factors=5, reg=0.6, reg.social=0.5, learn.rate=0.001, max.iter=100
TrustSVD 0.806 1.046 23:44 00:01 factors=10, reg=0.6, reg.social=0.5, learn.rate=0.001, max.learn.rate=0.001, max.iter=130

FilmTrust (35K)

Algorithm MAE RMSE Train Time (s) Test Time (s)
MMLite LibRec MMLite LibRec MMLite LibRec MMLite LibRec
GlobalAvg 0.715 0.715 0.919 0.919 00:00 00:00 00:00 00:00
UserAvg 0.635 0.636 0.822 0.823 00:00 00:00 00:00 00:00
ItemAvg 0.726 0.725 0.929 0.927 00:00 00:00 00:00 03:06
UserKNN 0.608 0.627 0.795 0.824 00:03 00:03 00:35 00:01
neighbors=50, similarity=pcc, shrinkage=30 (MMLite: reg_u=15, reg_i=10)
ItemKNN 0.603 0.619 0.792 0.818 00:01 01:06 00:13 00:00
neighbors=50, similarity=pcc, shrinkage=30 (MMLite: reg_u=15, reg_i=10)
SlopeOne 0.631 0.630 0.837 0.836 00:01 00:00 00:00 00:00
BiasedMF 0.616 0.615 0.810 0.808 00:02 00:00 00:00 00:00
factors=5, reg=0.1, learn.rate=0.01, max.iter=100
BiasedMF 0.616 0.611 0.807 0.802 00:03 00:00 00:00 00:00
factors=10, reg=0.1, learn.rate=0.01, max.iter=100
BiasedMF 0.652 0.637 0.878 0.862 00:02 00:00 00:00 00:00
factors=5, reg=0.01, learn.rate=0.01, max.iter=100
SocialMF 0.626 0.638 0.804 0.837 00:03 00:02 00:00 00:00
factors=5, reg=0.001, reg.social=1 learn.rate=0.001, max.iter=200
SocialMF 0.630 0.642 0.810 0.844 00:05 00:02 00:00 00:00
factors=10, reg=0.001, reg.social=1 learn.rate=0.001, max.iter=200
Algorithm MAE RMSE Train Test Configuration
PMF 0.714 0.949 00:00 00:00 factors=5, reg=0.1, learn.rate=30, momentum=0.8, max.iter=200
PMF 0.735 0.968 00:00 00:00 factors=10, reg=0.1, learn.rate=30, momentum=0.8, max.iter=200
NMF 0.643 0.859 00:04 00:00 factors=100, max.iter=10
BPMF 0.659 0.871 00:11 00:00 factors=5, max.iter=50
RSTE 0.628 0.810 00:05 00:00 factors=5, reg=0.001, learn.rate=0.001, alpha=1.0, max.iter=100
RSTE 0.640 0.835 00:05 00:00 factors=10, reg=0.001, learn.rate=0.001, alpha=1.0, max.iter=100
SoRec 0.628 0.810 00:01 00:00 factors=5, reg=0.001, reg.c=0.01, learn.rate=0.001, max.iter=100
SoRec 0.638 0.831 00:01 00:00 factors=10, reg=0.001, reg.c=0.01, learn.rate=0.001, max.iter=100
SoReg 0.674 0.878 00:00 00:00 factors=5, reg=0.001, beta=0.1, learn.rate=0.001, max.iter=80
SoReg 0.668 0.875 00:00 00:00 factors=10, reg=0.001, beta=0.1, learn.rate=0.001,max.iter=100
TrustMF (Tr) 0.629 0.812 00:01 00:00 factors=5, reg=0.001, reg.social=1 learn.rate=0.05, max.iter=200
TrustMF (Te) 0.629 0.813 00:01 00:00 factors=5, reg=0.001, reg.social=1 learn.rate=0.05, max.iter=200
TrustMF (T) 0.631 0.810 00:01 00:00 factors=5, reg=0.001, reg.social=1 learn.rate=0.05, max.iter=200
TrustMF (Tr) 0.633 0.824 00:01 00:00 factors=10, reg=0.001, reg.social=1 learn.rate=0.05, max.iter=200
TrustMF (Te) 0.630 0.821 00:01 00:00 factors=10, reg=0.001, reg.social=1 learn.rate=0.05, max.iter=200
TrustMF (T) 0.631 0.819 00:05 00:00 factors=10, reg=0.001, reg.social=1 learn.rate=0.05, max.iter=200
SVD++ 0.613 0.804 01:08 00:00 factors=5, reg=0.1, learn.rate=0.01, max.iter=100
SVD++ 0.611 0.802 01:26 00:00 factors=10, reg=0.1, learn.rate=0.01, max.iter=100
TrustSVD 0.609 0.789 00:20 00:00 factors=5, reg=1.2, reg.social=0.9, learn.rate=0.01, max.iter=100
TrustSVD 0.607 0.787 00:25 00:00 factors=10, reg=1.2, reg.social=0.9, learn.rate=0.01, max.iter=100

Ciao (280K)

Algorithm MAE RMSE Train Test Configuration
GlobalAvg 0.820 1.059 00:00 00:00  
UserAvg 0.781 1.031 00:00 00:00  
ItemAvg 0.760 1.026 00:00 00:09  
UserKNN 0.796 1.047 01:47 01:46 neighbors=50, similarity=pcc, shrinkage=30
PMF 0.920 1.206 00:10 00:00 factors=5, reg=0.1, learn.rate=50, momentum=0.8, max.iter=100
PMF 0.822 1.078 00:18 00:00 factors=10, reg=0.1, learn.rate=50, momentum=0.8, max.iter=100
PMF 0.777 1.040 00:42 00:00 factors=20, reg=0.1, learn.rate=50, momentum=0.8, max.iter=100
PMF 0.795 1.090 04:22 00:00 factors=50, reg=0.1, learn.rate=10, momentum=0.8, max.iter=200
RSTE 0.767 1.020 03:57:34 00:00 factors=5, reg=0.05, learn.rate=0.001, alpha=1.0, max.iter=100
RSTE 0.763 1.013 03:17:41 00:00 factors=10, reg=0.05, learn.rate=0.001, alpha=1.0, max.iter=100
SoRec 0.765 1.013 00:39 00:00 factors=5, reg=0.001, reg.c=0.01, learn.rate=0.05, max.iter=100
SoRec 0.761 1.010 01:12 00:00 factors=10, reg=0.001, reg.c=0.01, learn.rate=0.05, max.iter=100
SoReg 0.899 1.183 00:44 00:00 factors=5, reg=0.001, beta=0.1, learn.rate=0.0001,max.learn.rate=0.001, max.iter=300
SoReg 0.815 1.076 00:35 00:00 factors=10, reg=0.001, beta=0.1, learn.rate=0.0001,max.learn.rate=0.001, max.iter=200
SoReg 0.777 1.046 00:48 00:00 factors=20, reg=0.001, beta=0.1, learn.rate=0.0001,max.learn.rate=0.001, max.iter=200
SocialMF 0.749 0.981 11:33 00:00 factors=5, reg=0.001, reg.social=1, learn.rate=0.001, max.iter=200
SocialMF 0.749 0.976 16:38 00:00 factors=10, reg=0.001, reg.social=1, learn.rate=0.001, max.iter=200
TrustMF (T) 0.742 0.983 01:14 00:00 factors=5, reg=0.001, reg.social=1, learn.rate=0.1, max.iter=200
TrustMF (T) 0.753 1.014 02:27 00:00 factors=10, reg=0.001, reg.social=1, learn.rate=0.1, max.iter=300
SVD++ 0.752 1.013 10:23 00:00 factors=5, reg=0.1, learn.rate=0.01, max.iter=100
SVD++ 0.748 1.001 12:34 00:00 factors=10, reg=0.1, learn.rate=0.01, max.iter=100
TrustSVD 0.722 0.954 8:18 00:00 factors=5, reg=0.5, reg.social=1, learn.rate=0.001, max.learn.rate=0.005, max.iter=100
TrustSVD 0.723 0.956 12:10 00:00 factors=10, reg=0.5, reg.social=1, learn.rate=0.001, max.learn.rate=0.005, max.iter=100

Flixster (8.2M)

Algorithm MAE RMSE Train Test Configuration
GlobalAvg 0.871 1.092 00:00 00:00  
UserAvg 0.685 0.918 00:00 00:00  
ItemAvg 0.844 1.070 00:00 03:06  
BiasedMF 0.632 0.852 05:04 00:00 factors=5, reg=0.03, learn.rate=0.051, max.iter=50;
BiasedMF 0.630 0.853 05:07 00:00 factors=5, reg=0.015, learn.rate=0.051, max.iter=50;
SocialMF 0.689 0.904 01-17:25:16 00:00 factors=5, reg=0.001, reg.social=1 learn.rate=0.001, max.iter=200
SocialMF 0.690 0.907 02-22:05:38 00:00 factors=10, reg=0.001, reg.social=1 learn.rate=0.001, max.iter=200
TrustMF (Tr) 0.677 0.893 20:35 00:00 factors=5, reg=0.001, reg.social=1, learn.rate=0.03, max.iter=200
TrustMF (Te) 0.677 0.892 20:14 00:00 factors=5, reg=0.001, reg.social=1, learn.rate=0.03, max.iter=200
TrustMF (T) 0.670 0.881 57:50 00:00 factors=5, reg=0.001, reg.social=1, learn.rate=0.03, max.iter=200
TrustMF (T) 0.661 0.872 01:16:26 00:00 factors=10, reg=0.001, reg.social=1, learn.rate=0.03, max.iter=200
SVD++ 0.631 0.851 01-12:02:29 02:07 factors=5, reg=0.03, learn.rate=0.05, max.iter=100
SVD++ 0.622 0.842 01-19:21:30 02:11 factors=10, reg=0.03, learn.rate=0.05, max.iter=100

Flixster (410K)

Algorithm MAE RMSE Train Test Configuration
GlobalAvg 0.873 1.093 00:00 00:00  
UserAvg 0.729 0.979 00:00 00:00  
ItemAvg 0.858 1.088 00:00 00:08  
PMF 0.814 1.076 00:14 00:00 factors=5, reg=0.1, learn.rate=30, momentum=0.8, max.iter=200
PMF 0.769 1.009 00:20 00:00 factors=10, reg=0.1, learn.rate=30, momentum=0.8, max.iter=200
SoRec 0.750 0.974 01:25 00:00 factors=5, reg=0.001, reg.c=0.001, learn.rate=0.001, max.iter=100
SoRec 0.785 1.018 02:01 00:00 factors=10, reg=0.001, reg.c=0.001, learn.rate=0.001, max.iter=100
SoReg 0.820 1.087 02:18 00:00 factors=5, reg=0.001, beta=1.0, learn.rate=0.001, max.learn.rate=0.02, max.iter=150
SoReg 0.785 1.034 02:18 00:00 factors=10, reg=0.001, beta=1.0, learn.rate=0.001, max.learn.rate=0.02, max.iter=150
RSTE 0.751 0.975 46:20 00:00 factors=5, reg=0.001, learn.rate=0.001, alpha=1.0, max.iter=100

Item Recommendation

General Settings: item.ranking=on -topN -1 -ignore -1; -threshold 0

MovieLens (100K)

Algo Prec@5 Prec@10 Recall@5 Recall@10 AUC MAP NDCG MRR
MMLite LibRec MMLite LibRec MMLite LibRec MMLite LibRec MMLite LibRec MMLite LibRec MMLite LibRec MMLite LibRec
MostPop 0.212 0.211 0.192 0.190 0.071 0.070 0.115 0.116 0.865 0.857 0.136 0.135 0.479 0.477 0.416 0.417
ItemKNN 0.314 0.318 0.279 0.260 0.096 0.103 0.167 0.164 0.929 0.885 0.223 0.187 0.563 0.536 0.527 0.554
neighbors=80, similarity=cos, shrinkage=50, thresold=-1
UserKNN 0.397 0.338 0.334 0.280 0.138 0.116 0.218 0.182 0.929 0.884 0.268 0.208 0.610 0.554 0.655 0.569
neighbors=80, similarity=cos, shrinkage=50, thresold=-1
BPR 0.358 0.378 0.309 0.321 0.247 0.129 0.201 0.209 0.932 0.933 0.247 0.260 0.589 0.601 0.589 0.622
factors=10, reg=0.01, learn.rate=0.05, max.iter=30
WRMF 0.416 0.424 0.353 0.358 0.142 0.149 0.227 0.236 0.936 0.928 0.287 0.294 0.624 0.631 0.672 0.675
alpha=1.0, factors=20, reg=0.015, max.iter=10
Algorithm Prec@5 Prec@10 Rec@5 Rec@10 AUC MAP NDCG MRR Configuration
AR 0.332 0.268 0.110 0.166 0.892 0.202 0.551 0.590  

Epinions (665K)

Algorithm Prec@5 Prec@10 Rec@5 Rec@10 AUC MAP NDCG MRR Configuration
FISMrmse 0.00636 0.00522 0.0121 0.0195 0.891 0.011 0.148 0.0256 factors=10, max.iter=50, rho=3, alpha=0.5, lambda=0, beta=1e-4, gamma=0.005
RankALS 0.00850 0.00682 0.0103 0.0164 0.572 0.00903 0.125 0.0286 factors=10, max.iter=10, sw=on, binary.threshold=-1

Flixster (410K)

Algorithm Prec@5 Prec@10 Rec@5 Rec@10 AUC MAP NDCG MRR Configuration
FISMrmse 0.00995 0.00867 0.0328 0.0532 0.955 0.034 0.193 0.0460 factors=10, max.iter=50, rho=3, alpha=0.5, lambda=0, reg=0.001

FilmTrust

Algorithm Prec@5 Prec@10 Rec@5 Rec@10 AUC MAP NDCG MRR Configuration
MostPop 0.417 0.350 0.402 0.638 0.956 0.494 0.655 0.616  
AR 0.424 0.350 0.414 0.631 0.955 0.503 0.662 0.631  
LDA 0.421 0.351 0.409 0.635 0.930 0.435 0.542 0.619 factors=10, alpha=2, beta=0.5, max.iter=3000, burn.in=1000, sample.lag=100
BPR 0.4126 0.3477 0.404 0.636 0.9618 0.4831 0.6461 0.5970 factors=10, reg=0.1, learn.rate=0.01, max.iter=30
FISMauc 0.4172 0.3501 0.4005 0.6342 0.9579 0.4924 0.6543 0.6177 factors=10, rho=30, alpha=0.5, reg=0.01, learn.rate=0.0001, max.iter=100
WRMF 0.421 0.345 0.414 0.615 0.910 0.501 0.656 0.629 factors=10, max.iter=30, reg=0.001, reg.j=0.001, learn.rate=0.001, threshold=0
WRMF 0.429 0.350 0.426 0.630 0.925 0.518 0.671 0.647 factors=10, max.iter=30, reg=0.001, reg.j=0.001, learn.rate=0.001, threshold=-1
SLIM 0.434 0.358 0.436 0.647 0.909 0.530 0.681 0.671 num.neighbors=100, sim=cos-binary, max.iter=150, reg.l1=1, reg.l2=5, threshold=-1
RankALS 0.359 0.272 0.321 0.464 0.658 0.356 0.538 0.555 factors=10, max.iter=8, sw=on

Ciao (280K)

Algorithm Prec@5 Prec@10 Rec@5 Rec@10 AUC MAP NDCG MRR Configuration
MostPop 0.0267 0.0214 0.0221 0.0353 0.816 0.0209 0.176 0.074