Adapting a recommender system for energy-saving behaviors to personal differences in decision-making
The Adaptive Advice project started as my graduation project for the Human-Technology Interaction program at TU/e.
The project combines research on recommender systems, human-like agents, interaction design and adaptive systems, and applies it to the domain of energy-saving.
Decision-making research has demonstrated in numerous experiments that humans differ in the way they make choices. Personal decision-making strategies tend to vary systematically: there are for example notable differences between experts and novices, and between people with different choice goals. Although recommender systems adapt their recommendations to user preferences, they typically do not adapt their interface to support the different decision-making strategies.
A notable example of a useful interface adaptation would be the Preference Elicitation (PE) method. Preference elicitation is the process of discovering what a user likes and dislikes. In our first experiment we compared an attribute-based and a case-based preference elicitation method, and found notable differences between experts and novices in terms of satisfaction and perceived usefulness of the system (see figure).
Surprisingly, a recent experiment with updated versions of these PE methods showed a reversal of this effect. I am currently working on an experiment that will investigate PE methods in much greater detail.
The finding that different PE methods work better for novices and experts suggests that the recommender system interface should be adapted to the user. Although it is possible to determine user characteristics using questionnaires, I opted for a less obtrusive approach in the second experiment of my graduation project.
Using the measured user characteristics and logging data from experiment 1, I looked for patterns in the click-stream data that could predict the user characteristics on the fly. I then used these "click rules" to construct an adaptive system that updates a user model of the predicted user characteristics, and morphs the interface when the predicted value passes certain thresholds.
The adaptive system was built in AJAX using a completely stateless and database-driven approach. The system saves its state and regenerates its interface after each click.
Users of the adaptive system were more satisfied and perceived the system as more useful than the static variant, but only when the system provided adequate explanations of its adaptive behavior.
Users do not expect a website to be adaptive, and may not accept such transfer of control to a system. In an attempt to increase the acceptance of the adaptive behavior, I tested a version of the system with a human-like agent providing the explanations. The idea behind this was that people expect and accept other humans (as well as human-like representations) to adaptively take control over a conversation.
This was surprisingly not the case. Whereas the adaptive system with explanations provided a better usability than a static system, these results did not hold when an agent provided the explanations. Acceptance rates for the agent-based explanations were also significantly lower. I suspect that the adaptive capabilities of the agent may have been overestimated by the users.
The system used in this project has gained quite a bit of attention from local newspapers, energy providers, urban planners and other companies interested in energy-saving. I am among the first people to frame the energy-saving problem as one of choice instead of motivation: people want to save energy, but they do not know where to start. Apparently, this unique view on this topic is widely supported.
Knijnenburg, B.P., Willemsen, M.C., Broeders, R.: Smart Sustainability through System Satisfaction: Tailored Preference Elicitation for Energy-saving Recommenders. Full paper accepted to the Americas Conference on Information Systems (AMCIS) 2014. download here.
Knijnenburg, B.P., Reijmer, N.J.M., Willemsen, M.C.: Each to His Own: How Different Users Call for Different Interaction Methods in Recommender Systems. ACM Conference on Recommender Systems (RecSys) 2011. DOI: 10.1145/2043932.2043960, download here.
Acceptance rate: 20%
Knijnenburg, B.P., Willemsen, M.C.: The Effect of Preference Elicitation Methods on the User Experience of a Recommender System. Proceedings of the 28th of the international conference extended abstracts on Human factors in computing systems (CHI) 2010. DOI: 10.1145/1753846.1754001, download here.
Knijnenburg, B.P., Willemsen, M.C.: Understanding the effect of adaptive preference elicitation methods on user satisfaction of a recommender system. ACM Conference on Recommender Systems (RecSys) 2009. DOI: 10.1145/1639714.1639793, download here.
Winner of the short paper award (among 47 papers).
Knijnenburg, B.P.: Adaptive Advice: Adapting a recommender system for energy-saving behaviors to personal differences in decision-making. Graduation thesis. download here.
Graded 9.5/10, nominated for the Dutch SIGCHI chapter Thesis Award (Gerrit van der Veer prijs), nominated for the Eindhoven University of Technology Academic Awards, winner of the Department of Innovation Sciences Best Thesis Award.