Information and Recommender Systems

5Evaluating the Quality of Recommender Systems

Il y a de méchantes qualités qui font de grands talents (Some bad qualities form great talents)François de La Rochefoucauld, Maxims, 1665

In the previous chapters, we introduced different recommendation techniques and a certain number of systems. These techniques and systems evolve over time, attempting to move ever closer to the expectations and requirements of users. This process requires us to evaluate recommender systems in order to verify whether or not they are relevant and offer the required levels of performance for users in relation to context, objectives, response time, consideration of certain criteria, etc.

5.1. Data sets, sparsity and errors

In the context of recommender systems, consideration is given to specific groups within a population (online customers, Internet users, etc.) to propose or suggest suitable, personalized items. To do this, a set of data is required, whether synthetic or, better, “historic” records of user interactions with a system, that is a collection of user profiles with preferences, scores, transactions, etc. The use of a single data set to evaluate different recommender systems makes it easier to compare the performance of these different systems directly.

However, due consideration must be given to the density of the data set, which corresponds to the relationship between empty and full cells of the Users × Items matrix:

where R is the set of scores, I is the set of items and C is the ...