MyMedia

User Experience in multi-media recommender systems

The MyMedia project is an international research cooperation under the European 7th Framework Program (FP7). The goal of the MyMedia project is to advance the state-of-the-art in multi-media recommender systems. The partners in this project acknowledge that the User Experience is a main criterion for the evaluation of such recommender systems.

My task

Within the project, I am responsible for the user-centric evaluation of the developed recommender framework in several large-scale field trials. I provide the partners with experimental designs, user experience questionnaires, and requirements for click-stream logging. Based on the evaluation framework I developed (see figure below), I will analyze the data gathered in the field trials and integrate the gained knowledge to gain a better understanding of user-recommender interaction.

The evaluation framework links objective aspects of a recommender system to objective user behavior through a series of subjective constructs. Specifically, changes in the recommender system (objective system aspects) are perceived by the users (subjective system aspects), and influence their experience and interaction with the system. The interaction with the recommender system is furthermore influenced by personal and situational characteristics.

Accomplishments

• Developed a framework for the evaluation of recommender systems
• Developed questionnaires for the constructs indicated in the evaluation framework
• Instructed the field trial partners on the data logging requirements for the evaluation
• Translated the field trial goals into testable hypotheses
• Instructed the partners on semi-structured interviews and grounded theory analysis
• Performed a pre-trial with one of the partners (the European Microsoft Innovation Center)

Publications

Knijnenburg, B.P., Meesters, L.M.J., Marrow, P., Bouwhuis, D.G.: User-Centric Evaluation Framework for Multimedia Recommender Systems. Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering. Short workshop paper for the first international conference on user centric media (UCMedia) 2009, download here.

Knijnenburg, B.P., Willemsen, M.C., Hirtbach, S.: Getting Recommendations and Providing Feedback: The User-Experience of a Recommender System. Accepted full paper at the 11th International Conference on Electronic Commerce and Web Technologies (EC-Web) 2010.

Bollen, D.G.F.M., Knijnenburg, B.P., Willemsen, M.C., Graus, M.P.: Understanding choice overload in recommender systems. Accepted full paper at the ACM Conference on Recommender Systems (RecSys) 2010.

Other activities

Organizer of the workshop on "User-Centric Evaluation of Recommender Systems and Their Interfaces" (UCERSTI) at the ACM Conference on Recommender Systems (RecSys) 2010. visit workshop website