Bart Knijnenburg - Research

Overview

As many of our consumptions and interactions have moved to the Web, the number of decisions consumers must make (as well as the number of options per decision) has grown exponentially. If we want people to tap the full potential of this explosion of decision opportunities, we should not just give them more decisions; we should help them to make better decisions. I have pursued this goal in my research on adaptive consumer decision support, combining research in social computing and HCI. Within this broad area I have specialized in usable recommender systems and privacy decision-making.

Usable Recommender Systems

User-Centric Evaluation of Recommender Systems

Contrary to many other researchers in the field of recommender systems, my work has focused primarily on HCI aspects. My very first conference paper explained why behavioral measures of system effectiveness were misaligned with users’ perceptions of effectiveness (C1), sparking the development of my framework for investigating user-centric aspects of recommender systems (J1, see also W1). This framework presents an important methodological contribution that has become one of the de facto standards in the field, and I have written a chapter for the 2nd edition of the Recommender Systems Handbook (B2) and taught several tutorials on how to conduct studies and analyze data in line with this framework (see also P3). Many researchers have adopted the framework to improve the evaluation of their systems: The framework paper (J1) has over 1100 citations and has consistently been among the most downloaded papers in the UMUAI journal.

Preference Elicitation

My early work in recommender systems mainly dealt with the often-overlooked input and output sides of recommender systems. On the input side, my work with Martijn Willemsen on a self-developed recommender system that gives personalized energy-saving advice has shown that the effectiveness of the advice depends on the match between the preference elicitation interface of the system and the domain knowledge of the user (P1, C3, P2, C15). One of my former MS advisees at Clemson (Pratitee Sinha, MS 2018) has successfully implemented these findings in an internship at The Home Depot, where she investigated the effect of recommendations on “prosumers” and their power tool purchases. Later work on this topic included the internship work of my advisee Lijie Guo (PhD, 2023) and other work conducted with colleagues at IBM Ireland, studying more interactive forms of preference elicitation (C38, J23). Interactive means of preference elicitation were also a key part of my work as a Visiting Researcher (2024–2025) at Google Research, in which I studied LLM-based conversational recommender systems.

Choice Overload and Diversification

On the output side, my work with Martijn Willemsen, Dirk Bollen and Mark Graus has shown that recommender systems can easily cause choice overload (C2). Our work has found fruitful strategies to overcome these problems, i.e. by presenting small but diversified sets of recommendations (W2, J6), and by carefully explaining the basis for the selection of these recommendations (C5, P7, C41). Two of my PhD students (Daricia Wilkinson, PhD 2022 and Lijie Guo) have done internships exploring the concept of explanations for recommender systems at IBM Research in Ireland (C38, J22). Moreover, my lab has worked with Osnat Mokryn (University of Haifa) to investigate the use of emotional signatures of media items as a conduit for diversifying and visualizing recommendations (J56). Another study on the diversity of the output of various music recommender platforms was conducted by a local High School student and awarded with the Best Poster Award at an internationally renowned conference (P11).

Recommender Systems for Self-Actualization

My current work comprises a project on recommender systems for self-actualization (C20, W14). Previously funded by an NSF CRII grant (2017–2019) and currently funded by an NSF CAREER grant (2021–2026), this project aims to support users in developing, exploring, and understanding their preferences based on their long-term goals and ambitions. To this end, we focus on supporting—rather than replacing—users’ decision-making processes, on exploration rather than consumption, and on covering all—rather than some—of the users’ tastes. My team (past: Saadhika Sivakumar, MS 2017; Daricia Wilkinson; Lijie Guo; Aminata Mbodj; current: Mehtab Iqbal, PhD; Sushmita Khan, PhD; Laila Shafiee, PhD) has developed several innovative recommender system features, interfaces, and algorithms to support these principles (P15). An important aspect of this work is to investigate the societal effects of recommender systems (B9), as well as their effect on underrepresented users, such as older adults (W23, W24) and people with ADHD (with Emma Dixon; P13, J58).

Recommenders in the Real World

My most recent work focuses on the application of recommender systems in real-world settings. As part of our Department of Education SEED grant (with Nathan McNeese and the Clemson Education Department; 2020–2023) we designed, developed and tested a system that recommends personalized professional development courses to 267 in-service educators (94.4% of whom served concentrations of high-needs students) in the Upstate South Carolina area (B5). My PhD student Lijie Guo used this platform to test the effects of diversification and explanation on choice overload in a real-world setting. Furthermore, as part of our NSF CCRI grant (2023–2025), our multi-institutional team (University of Minnesota, University of Colorado Boulder, Northwestern University, and Drexel University) is building a system that sends users daily personalized newsletters with news articles from the Associated Press. This system, called POPROX is built to be a platform for researchers to test their recommendation algorithms, presentation mechanisms, and other innovations in a real-world setting (W28). This work has supported PhD student Mehtab Iqbal and has sparked additional collaborations with the partner institutions (W27).

Privacy Decision-Making

I have made considerable contributions to the field of socio-technical privacy studies, including invited chapters (B1, B3, B10) and an edited book titled Modern Socio-Technical Perspectives on Privacy, the chapters of which have accessed a cumulative 417k times to date (E1, B6)— the 9th most accessed Open Access Computer Science volume Springer published in 2021–2022.

Privacy, Adoption and Online Safety

I opened my PhD dissertation (supervised by Alfred Kobsa) with: “Privacy issues are an undying obstacle to the adoption of social and mobile technologies.” These words have never been truer today (C21): my research has shown that privacy issues are a key limiting factor in the adoption of smart home technologies (J12), mobile applications (C37) and groupware applications during the pandemic (J25) and have led many users to abandon social media platforms (B4, J10). I have also shown that the lack of privacy protections can lead to severe threats to online safety (P9, C19, C45, J42). Notable in this regard is the work by my PhD student Daricia Wilkinson, who, sponsored by a Meta PhD Fellowship, a Google Women Techmakers’ fellowship, and a FRIDA Open Internet grant, coordinated a series of large-scale survey and interview studies to analyze online safety threats and protections in the Caribbean region (C39, J46). I also participated in an NSF CIRC Planning grant (2024–2025) to build a research infrastructure for mitigating privacy harms, and a grant from the National Institute of Justice (with Amira Jadoon in the Department of Political Science, in collaboration with the University of Massachusetts, Lowell and Boston University; 2024–2026) to study the effect of online misinformation on violent extremism (J59, J62).

User-Tailored Privacy

Research in the field of online privacy acknowledges that end-users should be given information about and control over the way their data is used and shared. At the same time, this research acknowledges that privacy decisions are inherently difficult, and that an abundance of information and control only confuses end-users in their privacy decision-making. Therefore, while I have contributed to a plethora of research into privacy-enhancing technologies (P4, C6, C14, C24, J4, W15, W16, J9, J17, J18, C50), my main research contribution is the idea that future information systems will have to actively assist consumers in their privacy decision-making practices—a concept I have dubbed user-tailored privacy (O1, O2, O4, O6, B8). The idea behind user-tailored privacy is to predict users’ privacy preferences and behaviors based on their past behavior and known characteristics, and then adapting their privacy settings and/or the settings interface to these preferences. User-tailored privacy reduces the burden of control, but at the same time respects users’ inherent privacy preferences.

Biases in Privacy Decision-Making

An important goal of user-tailored privacy is to overcome the biases inherent in users’ privacy decision-making process. My studies have mapped out users’ privacy decision-making processes (J33, C34, C36) and demonstrated several of these biases (W3, W4, C10, C16, J2, J19, J45). Together with several of my students and collaborators, I have replicated and extended my earlier work on default effects in form auto-completion tools (W12, C12, J33) in the areas of Facebook photo tagging (Reza Ghaiumy Anaraky, PhD 2022; W18, J20) smarthome privacy settings (Yang He, PhD 2019; P6, P10, W17, C33), privacy-enhancing technologies (Reza Ghaiumy Anaraky; C43, C47) and group privacy decisions (Nava Tintarev, Maastricht University; J34, J40).

Individual and Cross-Cultural Differences

The idea of user-tailored privacy is driven by individual differences in privacy decision-making between users, as well as differences caused by various contextual factors. The topic of individual differences has been an important part of my collaboration with Pamela Wisniewski and Xinru Page, who have collaborated extensively on this topic with my students and myself (C4, C8, C9, C17, J12, J16, C28), and I have extended this work to cross-cultural differences with my student Reza Ghaiumy Anaraky, Hichang Cho and Yao Li (J8, J11, J24, J26, J29), as well as deeper investigations into privacy in different cultures/jurisdictions (C27, J30, C39, J46). Regarding contextual dimensions, I have used state-of-the-art statistical methods to show that users’ disclosure behaviors can be mapped onto several distinct latent dimensions (i.e. “types” of information), and that distinct subgroups of users with similar “disclosure profiles” can be identified in several domains (W5, W7, W9, J3, J7).

Predicting Privacy Preferences

One of these domains is the Internet-of-Things (IoT). Indeed, with the rise of IoT, the need for decision support interfaces will become even more apparent (B7)—our work has shown that users have little interest (or knowledge) in regulating their disclosure and data-sharing in a digitally connected physical environment (J12). I was previously funded by Samsung Research America (2016) and by NSF SaTC and NWO (the Dutch NSF) on a collaborative grant (2017–2019) with UC Irvine and the TU Eindhoven to improve the privacy decisions of the users of smarthome devices. To this end, we developed a data-driven approach for the development of IoT privacy-setting interfaces. We have applied this approach not only to smarthomes (J13) but also to wearables (J14, J15; in a collaboration with Ilaria Torre, University of Genova) and the public IoT context (C25).

I similarly investigated the idea of predicting users’ privacy settings in the field of recommender systems (W8, C11), mobile privacy settings (C40), and location sharing (W6, P5, C7, C13, P5, C18). Note, though, that the ultimate goal of user-tailored privacy is not just to adapt users’ privacy settings to match their disclosure profile, but to align the privacy-setting user interface to their goals and expectations (C22, C26, J28). This way, end-users maintain control over their privacy settings, but can do so in the most convenient way possible.

User-Tailored Privacy in the Real World

My research on user-tailored privacy has also attracted considerable attention from corporate sponsors: My PhD student Moses Namara (PhD, 2022) received two Meta PhD fellowships and conducted an internship with them to study the implementation of this idea within Meta’s platforms. In 2017, I organized an international summit on user-tailored privacy that was attended by representatives from Meta, Intel, and the Mozilla Foundation. And our research into the privacy of personalized advertisement on social networks (C32, J52) has been supported by a research gift from Meta (2020) and a student internship at Norton LifeLock (for Daricia Wilkinson). The latter project resulted in a new framework to measure users’ perceptions of online personalized advertising, which was published in Internet Research (J48). Another project on privacy in personalized advertisement was conducted in collaboration with Johanna Björklund (Umeå University, Sweden) and Sara Leckner (Malmö University), which resulted in an ACM SigCHI conference paper (C53).

The mentioned user-tailored privacy summit was sponsored by a Advanced Distributed Learning Initiative research contract (2016–2020), for which my team investigated the implementation of privacy-by-design and user-tailored privacy in the “Total Learning Architecture” (TLA), a next generation learning platform that uses pervasive user monitoring to provide highly adaptive learning recommendations to soldiers and other defense personnel (O5, O7). This project has allowed my students to translate their work and other relevant literature into concrete recommendations for engineers implementing the TLA.

Privacy Education - Middle School

My most recent work on in the realm of privacy covers privacy education, which I have approached along two research tracks. The first track was sponsored by an NSF SaTC-EDU EAGER grant (with Kelly Caine and Nicole Bannister; 2020–2022) and involved teaching middle schoolers the impact of AI-related technologies on their privacy and cybersecurity. Unlike most existing privacy and cybersecurity education efforts, which typically run as extracurricular activities, we have embedded our educational modules directly into the middle school Math and Computer Science curriculum, which allows for a more equitable exposure to our educational materials (O9). Our integration with both CS and Math is rooted in the belief that an understanding of the mathematical principles that underlie AI technologies will help students gain a more fundamental understanding of their cybersecurity consequences (W20). Moreover, the results of our extensive interviews with students at our partner middle school suggested that the introduction of socially and personally relevant applications of mathematical principles can improve the quality of math education. Three of my PhD students (Sushmita Khan, Mehtab Iqbal, and Oluwafemi Osho, PhD 2026) developed six “mini modules” on AI data tracking, personalized advertising (C44), algorithmic fairness (J55), filter bubbles (J55), misinformation and phishing, and going viral, that were all deployed at our partner middle school (O9). We are still in the process of analyzing the data from these modules; the results from three of the modules have been published at the ACM SigCHI conference (C44) and in the Computers & Education: Artificial Intelligence journal (J55). We have recently added Jinkyung (Katie) Park to this team and are exploring opportunities to scale our education efforts.

Privacy Education - Older Adults

The second education track was sponsored by a Meta PhD fellowship for Reza Ghaiumy Anaraky (2021–2022) and a Meta Research Award (with Kaileigh Byrne in the Department of Psychology; 2022) and involved developing privacy education interventions that are particularly targeted to rural older adults (W22). In this project, my PhD students (Heba Aly, PhD 2025, Reza Ghaiumy Anaraky and Sushmita Khan) and Kaileigh Byrne’s student (Yizhou (Louis) Liu, PhD 2026) first tested the suitability of various educational modalities, inspired by my prior work on privacy education videos (C42) and privacy comics (W13, P8). Focus group results show marked differences between younger and older adults—but also among older adults—in terms of their preferred modality (J37), and a follow-up controlled online experiment quantified the ability of the different modalities to produce knowledge gains (as measured in a pre-/post-test) and provide a satisfying user experience (as measured through multi-item measurement scales). This second study has been published in the journal Technology in Society (J51). Louis' follow-up study tested the effect of privacy education on users' subsequent privacy behaviors (J57).

Heba Aly subsequently focused her research on one specific privacy education modality, by designing, implementing, and testing a privacy education chatbot powered by generative AI. While such a tool would allow users to learn about privacy in their own way (asking self-relevant questions) and at their own pace, the education modality research outlined above suggested that especially older adults showed a lack of trust in chatbot technology (something we confirmed in P14). To help increase older adults’ trust in her privacy education chatbot, we introduced the concept of “trust transfer” (C48).

Other Research

Video Games and Societal Biases

I worked with my PhD student Marie Jarrell (PhD, 2020) and Erin Ash (Communication Department) to study the ability of video games to reduce societal biases regarding race and gender. Marie developed a video game in which participants would play as, or alongside, a character of a specific race or gender. Her work shows that playing as an underrepresented character can indeed reduce subsequent biases in an evaluation task (C35).

Improving the Academic Retention and Succes of Underrepresented Students

I took over an NSF IUSE grant from Eileen Kraemer (2021–2025; in collaboration with Morehouse College, Howard University and Cazembe Kennedy (Vanderbilt University)) to study best practices in improving the academic retention and success of underrepresented students in Computer Science. This project tracks several cohorts of students at each institution by analyzing students' answers to standardized exam questions, evaluating their support structures and their attitudes and perceptions towards Computer Science as a major and as a career, and conducting in-depth interviews with both students and their instructors. My PhD student Oluwafemi Osho coordinated a team of graduate and undergraduate researchers at the three institutions to conduct the integrative analysis of these various sources of data (P12, C42, C55, J60).

Giving Social Media Post Translation Control

My PhD student Ananya Gupta is studying the effect of giving social media post authors control over the translation of their posts. Most social networks do not give the authors of posts any insight into—let alone control over—how their posts are translated into other languages to help their multilingual audience understand the post. Ananya’s work acknowledges authors choose a certain language because they have an implicit subset of their audience in mind as the intended recipient of the post, and that broadening this audience can have unintended consequences. Moreover, authors worry about reputation damage due to context-related misunderstandings (W25). Our controlled experiment demonstrates that author control features increase their confidence and user experience (J53).

Responsible Human-Centered AI Research Practices

I am working with several students (Hansen Lee, PhD, Han Alzughbi, PhD, Heba Aly, Sushmita Khan, Mehtab Iqbal and Laila Shafiee) to study responsible human-centered AI research practices. One project (led by Han Alzughbi, with Katie Park) involves studying how the parameters of human-centered AI-based research studies influence participants’ perceptions of, and tendency to participate in, the described study. Another project (led by Sushmita Khan and Laila Shafiee, with Katie Park) studies researchers’ practices and motivations in protecting research participants. Third, our NSF CICI project (2022–2025; with Kelly Caine, in collaboration with Susan McGregor at Columbia University) investigates human-centered AI researchers’ sample size determination strategies and demographic data collection and reporting practices. A goal of this project is to build usable tools to assist researchers in this process (O10, C56, C57).

End-Users' Use of Off-The-Shelf LLMs

My PhD student Hansen Lee is studying how end-users use off-the-shelf LLMs for learning to code, writing support (with Erin Ash), and mental health support (with Katie Park). This work is preliminary and has not yet led to significant publications.

Other Collaborations

Finally, I am a frequent collaborator of Clemson faculty members in the HCC division and beyond, supporting their use of advanced statistical methods to analyze research data. I have publications in VR/virtual humans (mostly with Sabarish Babu (Texas A&M University) and Matias Volonte; W10, J5, C29, C30, C31, C54, J21, J54), human-AI teams (with Nathan McNeese and Christopher Flathmann; J31, J32, J35, J36, J39, J41, J47), as well as with Guo Freeman (J27, J38), David Neyens (Industrial Engineering; C46), Da Li (Civil Engineering, J49), and Jacob Sorber (C51, J50).