In addition, most approaches (81 %) neglected the user-modeling process and did not infer information automatically but let users provide keywords, text snippets, or a single paper as input. As a second limitation, we noted that many authors neglected to take into account factors other than accuracy, for example overall user satisfaction. Hence, finding the most promising approaches is a challenge. (C) We speculated that minor variations in datasets, algorithms, or user populations inevitably lead to strong variations in the performance of the approaches. Consequently, researchers use different implementations of the same recommendations approaches, which might lead to variations in the results. (B) Some authors provided little information about their algorithms, which makes it difficult to re-implement the approaches. They were based on strongly pruned datasets, few participants in user studies, or did not use appropriate baselines. We identified three potential reasons for the ambiguity of the results. Sometimes content-based filtering performed better than collaborative filtering and sometimes it performed worse. For instance, researchers reported different results on the performance of content-based and collaborative filtering. First, it remains unclear which recommendation concepts and approaches are the most promising. Our review revealed some shortcomings of the current research. In addition to simple terms, n-grams, topics, and citations were utilized to model users’ information needs. TF-IDF was the most frequently applied weighting scheme.

The content-based filtering approaches mainly utilized papers that the users had authored, tagged, browsed, or downloaded. Other recommendation concepts included stereotyping, item-centric recommendations, and hybrid recommendations. Collaborative filtering was applied by only 18 % of the reviewed approaches, and graph-based recommendations by 16 %. We found that more than half of the recommendation approaches applied content-based filtering (55 %). We reviewed these articles and present some descriptive statistics in this paper, as well as a discussion about the major advancements and shortcomings and an overview of the most common recommendation concepts and approaches. In the last 16 years, more than 200 research articles were published about research-paper recommender systems.

Software like docear 2017