Predicting Online Performance of News Recommender Systems Through Richer Evaluation Metrics

We investigate how metrics that can be measured offline can be used to predict the online performance of recommender systems, thus avoiding costly A-B testing. In addition to accuracy metrics, we combine diversity, coverage, and serendipity metrics to create a new performance model. Using the model, we quantify the trade-off between different metrics and propose to use it to tune the parameters of recommender algorithms without the need for online testing. Another application for the model is a self-adjusting algorithm blend that optimizes a recommender’s parameters over time. We evaluate our findings on data and experiments from news websites. Categories and Subject Descriptors H.3.3 [Information Storage and Retrieval]: Information Search and Retrieval—Information filtering Keywords recommender system;online evaluation;evaluation metrics

• Type: Journal
• Year: 2015
• Where: RECSYS
• Authors: Andrii Maksai, Florent Garcin, Boi Faltings
• Keywords: Control Systems, RECSYS 2015