MyMedia

User Experience in multi-media recommender systems

The MyMedia project is an international research cooperation under the European 7th Framework Program (FP7). The goal of the MyMedia project is to advance the state-of-the-art in multi-media recommender systems. The partners in this project acknowledge that the User Experience is a main criterion for the evaluation of such recommender systems.

My task

Within the project, I am responsible for the user-centric evaluation of the developed recommender framework in several large-scale field trials. I provide the partners with experimental designs, user experience questionnaires, and requirements for click-stream logging. Based on the evaluation framework I developed (see figure below), I will analyze the data gathered in the field trials and integrate the gained knowledge to gain a better understanding of user-recommender interaction.

The evaluation framework links objective aspects of a recommender system to objective user behavior through a series of subjective constructs. Specifically, changes in the recommender system (objective system aspects) are perceived by the users (subjective system aspects), and influence their experience and interaction with the system. The interaction with the recommender system is furthermore influenced by personal and situational characteristics.

Accomplishments

• Developed a framework for the evaluation of recommender systems
• Developed questionnaires for the constructs indicated in the evaluation framework
• Instructed the field trial partners on the data logging requirements for the evaluation
• Translated the field trial goals into testable hypotheses
• Instructed the partners on semi-structured interviews and grounded theory analysis
• Performed a pre-trial with one of the partners (the European Microsoft Innovation Center)

Publications

Knijnenburg, B.P., Meesters, L.M.J., Marrow, P., Bouwhuis, D.G.: User-Centric Evaluation Framework for Multimedia Recommender Systems. UCMedia2009 Workshop Experience Design and Evaluation of Social UCM Applications. Short paper, in press.

Knijnenburg, B.P., Willemsen, M.C., Hirtbach, S.: Getting Recommendations and Providing Feedback: The User-Experience of a Recommender System. Submitted as a full paper to the ACM Conference on User Modeling, Adaptation and Personalization (UMAP) 2010.