Part of the MyMedia project
The MyMedia project is an international research cooperation under the European 7th Framework Program (FP7). The goal of the MyMedia project is to advance the state-of-the-art in multi-media recommender systems. The partners in this project acknowledge that the User Experience is a main criterion for the evaluation of such recommender systems.
Within the project, I am responsible for the user-centric evaluation of the developed recommender framework in several large-scale field trials. I provide the partners with experimental designs, user experience questionnaires, and requirements for click-stream logging. Based on the evaluation framework I developed (see figure below), I will analyze the data gathered in the field trials and integrate the gained knowledge to gain a better understanding of user-recommender interaction.
The evaluation framework links objective aspects of a recommender system to objective user behavior through a series of subjective constructs. Specifically, changes in the recommender system (objective system aspects) are perceived by the users (subjective system aspects), and influence their experience and interaction with the system. The interaction with the recommender system is furthermore influenced by personal and situational characteristics.
Knijnenburg, B.P., Willemsen, M.C., Gantner, Z., Soncu, H., Newell, C.: Explaining the User Experience of Recommender Systems. Paper accepted to the User Modeling and User-Adapted Interaction (UMUAI) journal, download final copy here.
UMUAI is ranked #3 among 26 HCI journals (Microsoft Academic Search) and #20 among 445 Computer Science journals (ISI/Thompson).
Knijnenburg, B.P., Willemsen, M.C., Kobsa, A.: A Pragmatic Procedure to Support the User-Centric Evaluation of Recommender Systems. Short paper accepted to the ACM Conference on Recommender Systems (RecSys) 2011. download camera-ready version here.
Acceptance rate: 27%.
Willemsen, M.C., Knijnenburg, B.P., Graus, M.P., Velter-Bremmers, L.C.M., Fu, K.: Using latent features diversification to reduce choice difficulty in recommendation lists Full paper at the RecSys 2011 workshop on Human Decision Making in Recommender Systems (Decisions@RecSys'11), download here.
Bollen, D.G.F.M., Knijnenburg, B.P., Willemsen, M.C., Graus, M.P.: Understanding Choice Overload in Recommender Systems. Full paper at the ACM Conference on Recommender Systems (RecSys) 2010, download here.
Acceptance rate: 19%. This paper is currently among the top 10 most downloaded papers of the RecSys conference series in the ACM digital library (among a total of 251 papers).
Knijnenburg, B.P., Willemsen, M.C., Hirtbach, S.: Getting Recommendations and Providing Feedback: The User-Experience of a Recommender System. Full paper at the 11th International Conference on Electronic Commerce and Web Technologies (EC-Web) 2010, download here.
Knijnenburg, B.P., Meesters, L.M.J., Marrow, P., Bouwhuis, D.G.: User-Centric Evaluation Framework for Multimedia Recommender Systems. Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering. Short workshop paper for the first international conference on user centric media (UCMedia) 2009, download here.
Willemsen, M.C., Knijnenburg, B.P., Bollen, D.G.F.M.: Recommending Less is More - Understanding Choice Overload Using a Movie Recommender System Accepted poster at the 31st annual meeting of the Society for Judgment and Decision Making (JDM), Nov. 19-22, 2010.
Reviewer for User Modeling and User-Adapted Interaction (UMUAI) journal.
Reviewer for CSCW 2011 and CHI 2012.
Keynote speaker of the RecSys2011 workshop on "User-Centric Evaluation of Recommender Systems and Their Interfaces 2" (UCERSTI::2).
Organizer of the RecSys2010 workshop on "User-Centric Evaluation of Recommender Systems and Their Interfaces" (UCERSTI). visit workshop website
Program committee member of the workshop on "Methods for User Studies of Interactive (TV) Technologies" (USIT) at the 8th European Conference on Interactive TV and Video (EuroITV) 2010. visit workshop website