Measurement & Evaluation of HCC Systems
HCC 4400 / 6400
This is the syllabus website for Clemson University Spring 2026 course HCC 4400 / 6400: Measurement & Evaluation of HCC Systems.
Meeting information:
Credit hours: 3
Location: McAdams Hall 107 and remote
Day and time: Monday & Wednesday, 2:30 – 3:45 pm
Instructor information:
Prof. Bart Knijnenburg
Email: bartk@clemson.edu
Office hours: Wednesday, 3:45 – 4:45 pm, upon request
Important: The information below may change!
Changes will be announced in class and via Canvas/email.
Course description
This course will teach you how to scientifically evaluate computing systems using a quantitative, user-centric approach. By the end of this course you will be able to statistically evaluate data obtained from a user experiment, a survey, or system usage log files. The basic idea of this class is explained in this TEDx-talk.
What are we going to do?
Course content and structure: This class will be a lot of work, but the advanced methods will give you a competitive advantage over other HCC students at other institutions. This course roughly consists of 3 parts:
- Part 1 (week 1-4): The practice of experimental evaluation (stuff you may have learned in Research Methods)
- Part 2 (week 5-8): Basic statistical methods: Correlation, Regression, ANOVA, and T-test (stuff you have forgotten from undergraduate statistics)
- Part 3 (week 9-15): Non-linear and multilevel statistics (stats for “messy” HCC variables)
Course modality and COVID/flu precautions
Those enrolled in remote sections of this course may join using Zoom. The Zoom link may also be used by anyone who feels under the weather, is in quarantine, or is in any other way concerned about their health and safety. Since some people in class have unvaccinated children, I strongly prefer that you are vaccinated and boosted if you join the class in person. Furthermore, if you have a cough or a runny nose, I strongly prefer that you wear a KN95 mask. The adoption of basic public health measures is entirely consistent with the exemplary role we embody as academics.
Course materials
Readings: This course uses the following resources:
- Knijnenburg B. P. and Willemsen, M. C. “Evaluating Recommender Systems with User Experiments”: author copy available for free here.
- Chapters 4 and 5 of MacKenzie, I. S. “Human-Computer Interaction: An Empirical Research Perspective”: available for free via our library proxy.
- Almost all chapters of Field, A. et al. “Discovering Statistics Using R”, 1st ed.: for sale or rent on Amazon.
Students are strongly recommended to buy the Field book, since it is an invaluable reference guide for future research projects.
Software: For the most part, we will use R and RStudio. R is like a programming language, and RStudio is an IDE for R (like how Eclipse is an IDE for Java). R and RStudio are both free. For sample size calculations we will use G*Power (also free).
Course structure
Office hours: Office hours will be directly after class on Wednesday (3:45-4:45 pm). If you want to meet outside of office hours, please send me an email with a few times you are available and the topic you want to talk about.
Slides: Presentation slides are linked in the course schedule below (topics listed in orange are clickable and link to the slides).
Assignments: There will be 4 assignments for this class. There will be some “insight” questions and some data analysis questions. Insight questions usually require a short (1-2 sentence) answer. Data analysis questions should be done in R (unless suggested otherwise), and the requisite dataset will be provided. The answers to data analysis questions should contain the executed R commands, a summary of the output (only the parts that answer the question), and an explanation/description of the results in your own words.
Assignments are each worth 10% of your grade. You are allowed to discuss the assignments or consult the Internet or an AI system, but you have to write your own write-up (i.e. you can discuss, but not copy). If you collaborate with others, please add a collaboration statement to your assignment (a simple statement saying “I collaborated with [name(s)]” is sufficient). If you found something online, add a citation. If you used AI, acknowledge this by mentioning the system and the prompt that you used to get to the answer. Unacknowledged collaboration, googling, or AI use will be considered plagiarism. Please note that based on my experience, collaborating with other students and following the book / slides is much more efficient and effective than using Google or AI tools!
Midterms and final exam: The midterms and the final are take-home and are each worth 15% of your grade. They will be very similar to the assignments, but you have a shorter amount of time for them, and you are not allowed to collaborate, use AI, or look stuff up online. For the midterms/final, collaboration, googling, or AI use will be considered cheating. I strongly recommend relying on the cheat sheets for completing the midterms and final.
Graduate/undergraduate differentiation: The assignments and midterms as listed in the syllabus are for the graduate course (HCC 6400). Undergraduate students are exempt from certain assignment/exam questions. Exempt questions will be announced throughout the semester.
Prerequisites: This course has no prerequisites, but you will get the most out of it if you have a basic understanding of human-centered computing and quantitative research methods.
Grading
- Assignments: 40% (10% each)
- Midterms and final: 60% (15% each)
In unusual circumstances these percentages could change, but I do not expect that to happen. Your final grade will be calculated by multiplying the percentages with the points you achieve on each assignment and midterm.
Graduate course grades (HCC 6400): In my default graduate grading scheme, 85+ is an A, 80+ is an A-, 75+ is a B+, 70+ is a B, 65+ is a B-, 60+ is a C+, 55+ is a C, 50+ is a C-, 45+ is a D, and less than 45 is an F. I sometimes apply a curve to lower some of these thresholds (this has historically happened mostly for the threshold between B and C).
Undergraduate course grades (HCC 4400): In my default undergraduate grading scheme, 80+ is an A, 70+ is a B, 60+ is a C, 40+ is a D, and less than 40 is an F. Undergraduate grades may be curved as well; this is done separately from the graduate grades.
Cheat sheets
For the homeworks, students should learn to apply the methods using the book and the lecture slides. For the midterms and the final, I will provide a cheat sheet for each method. Cheat sheets will become available after each homework.
I have created the following cheat sheets for your convenience:
- No cheat sheets available yet!
Course schedule
For your convenience, you can add the course schedule to your calendar (ICAL or HTML).
| Week | Dates | Topic and contents | Work |
| 1.2 | Wednesday Jan 7 | Read BEFORE class: Handbook chapter, Chapter 4 of MacKenzie | |
| 2.1 | Monday Jan 12 | Read before class: Chapter 5 of MacKenzie | |
| 2.2 | Wednesday Jan 14 | Read before class: Chapters 1-4 of Field | |
| 3.1 | Monday Jan 19 | MLKjr day (no class) | |
| 3.2 | Wednesday Jan 21 | Dealing with data - Part 2 | |
| 4.1 | Monday Jan 26 | Read before class: Chapter 5 of Field | |
| 4.2 | Wednesday Jan 28 | Assumptions, sample size - Part 2 | |
| 5.1 | Monday Feb 2 | Read before class: Chapter 6 of Field | |
| 5.2 | Wednesday Feb 4 | Read before class: Chapter 7 of Field | |
| 6.1 | Monday Feb 9 | Regression - Part 2 | Homework 1 available |
| 6.2 | Wednesday Feb 11 | Read before class: Chapter 9 of Field | |
| 7.1 | Monday Feb 16 | Read before class: Chapter 10 of Field | |
| 7.2 | Wednesday Feb 18 | ANOVA - part 2 | Due before class: Homework 1 |
| 8.1 | Monday Feb 23 | Review session 1 | Midterm 1 released after class |
| 8.2 | Wednesday Feb 25 | Due before class: Midterm 1 Read before class: Chapter 12 of Field Homework 2 available | |
| 9.1 | Monday Mar 2 | Midterm 1 feedback | |
| 9.2 | Wednesday Mar 4 | Read before class: Chapter 8 of Field (except 8.9) | |
| 10.1 | Monday Mar 9 | Logistic regression - Part 2 | Due before class: Homework 2 |
| 10.2 | Wednesday Mar 11 | Read before class: Chapter 18 of Field (except 18.7-18.12) Homework 3 available | |
| Mar 16+18 | No class - Spring Break | ||
| 11.1 | Monday Mar 23 | Review session 2 | Midterm 2 released after class |
| 11.2 | Wednesday Mar 25 | Due before class: Midterm 2 Read before class: Chapter 13 of Field | |
| 12.1 | Monday Mar 30 | Midterm 2 feedback | Read before class: Chapter 14 of Field |
| 12.2 | Wednesday Apr 1 | Due before class: Homework 3 Read before class: Chapter 19 of Field | |
| 13.1 | Monday Apr 6 | Review session 3 | Midterm 3 released after class |
| 13.2 | Wednesday Apr 8 | Multilevel linear models - Part 2 | Due before class: Midterm 3 Homework 4 available |
| 14.1 | Monday Apr 13 | Midterm 3 feedback Generalized multilevel linear models - Part 1 | |
| 14.2 | Wednesday Apr 15 | Generalized multilevel linear models - Part 2 | Due before class: Homework 4 |
| 15.1 | Monday Apr 20 | Review session 4 | |
| 15.2 | Wednesday Apr 22 | Preview of Advanced Measurement & Evaluation | |
| exam | Final exam released Tuesday April 28 at 5:30pm | Final exam due Thursday April 30 at 5:30pm |
Attending class, etc.
Things discussed in class are part of the course materials, and although the slides and recordings will be put on this website, I cannot guarantee that no additional material are discussed in class. Classes will include “follow along” examples, so please have R and RStudio installed on your computer.
You will get an email notification in the event that class is cancelled. If the instructor is more than 15 minutes late, you can assume a last-minute cancellation. Hopefully this will not happen!
University Policies
You can find information about campus wide policies, including student affairs information and accessibility services, by clicking on Clemson‘s University Policies Page. There, you can find information about academic integrity, access and equity (including student accessibility and Title IX info), student financial services, emergency planning, and more. My expectation is that you will review these policies carefully and be responsible for them this semester.
Academic integrity
As members of the Clemson University community, we have inherited Thomas Green Clemson‘s vision of this institution as a “high seminary of learning.” Fundamental to this vision is a mutual commitment to truthfulness, honor, and responsibility, without which we cannot earn trust and respect of others. Futhermore, we recognize that academic dishonesty detracts from the value of a Clemson degree. Therefore, we shall not tolerate lying, cheating, or stealing in any form. Using materials generated using artificial intelligence (AI) that are turned in without attribution is considered plagiarism.
Practically speaking: Do not cheat (e.g.: do not collaborate or use Google/AI on the midterms and/or final). Plagiarism will not be tolerated, and be dealt with through official university channels, see: http://www.clemson.edu/academics/integrity/plagiarism.html.
Cheat sheets are created for your convenience to help you with the midterms and the exam. Please do not use cheat sheets (e.g., from last year) when making the homework. Using last years‘ cheat sheets on the homeworks is considered cheating, and will be dealth with accordingly.
All infractions of academic dishonesty by undergraduates must be reported to Undergraduate Studies for resolution through that office. In cases of plagiarism instructors may use the Plagiarism Resolution Form. See the following resources:
Student Accessibility Services
Clemson University values the diversity of our student body as a strength and a critical component of our dynamic community. Students with disabilities or temporary injuries/conditions may require accommodations due to barriers in the structure of facilities, course design, technology used for curricular purposes, or other campus resources. Students who experience a barrier to full access to this class should let the instructor know and make an appointment to meet with a staff member in Student Accessibility Services as soon as possible. You can make an appointment by calling 864-656-6848, by emailing CUSAS@clemson.edu, or by visiting Suite 239 in the Academic Success Center building. Appointments are strongly encouraged — drop-ins will be seen, if at all possible, but there could be a significant wait due to scheduled appointments. Students who have accommodations are strongly encouraged to request, obtain, and send these to their instructors via SAS as early in the semester as possible so that accommodations can be made in a timely manner. It is the student‘s responsibility to follow this process each semester.
Title IX (Sexual Harassment) statement
The Clemson University Title IX statement: Clemson University is committed to creating and continuously fostering a caring community based on the core values of integrity, honesty and respect. Sexual discrimination, which includes sexual harassment, sexual violence, stalking and domestic and/or relationship violence, is unacceptable and has no place in Clemson’s community. Consistent with its Title IX obligation, the University prohibits discrimination, including sexual and gender-based harassment and violence, in all its programs and activities, including academics, employment, athletics, and other extracurricular activities. This Title IX policy is available online. Katherine Weathers is the Clemson University Title IX Coordinator and VP of Inclusive Excellence. She can be reached at (864) 656-3413 or via email at kweath3@clemson.edu. Remember, email is not a fully secured method of communication and should not be used to discuss Title IX issues.