Collective Consent: Who Needs to Consent to the Donation of Data Representing Multiple People?Data donation is a growing form of personal data collection that foregrounds consent and conscious participation of the data donor. There remains little guidance on who must consent to data donation, particularly when the data represents multiple people. We provide empirical perspectives on this question through in-situ observation and interviews (N=18) with online daters who chose to donate messaging interactions with potential sexual partners for sexual violence research. Findings elucidate two diverging perspectives. Participants advocating for “unilateral consent” argued that consent of their messaging partner is not necessary, in part, because the anticipated benefit of data donation superceded consent. Participants advocating for “collective consent” wanted both messaging partners to consent to its donation, citing concerns for privacy of, and personal relationships with, the other person. Findings suggest that collective consent interfaces should be incorporated in data donation platforms, even if not strictly required by legal regulation, to increase donation of multi-person data.2025EWEmma Walquist et al.Toward More Ethical and Transparent Systems and EnvironmentsCSCW
Oak Story: Improving Learner Outcomes with LLM-Mediated Interactive NarrativesNarrative-based education engages children in learning, but traditional approaches offer limited adaptability to individual preferences. Although large language models (LLMs) offer promising opportunities for interactive narratives, balancing their unpredictability with structured learning objectives remains challenging. To answer this challenge, we designed and built Oak Story, an educational mobile application for 4th–6th graders centered on local oak woodland ecosystems. Oak Story employs a learning-goal-directed LLM architecture that adapts the narrative, as well as multimodal real-world activities, to each individual student while ensuring that learning goals are met. In a between-participants study (N = 47), we find that Oak Story produces statistically significant increases in learning gains, engagement, and perceived agency compared to a control with static sequencing within and between scenes. These findings demonstrate an effective architectural approach for LLM-based educational systems that successfully balances learner agency with pedagogical structure.2025ACAlan Y. Cheng et al.Human-LLM CollaborationOnline Learning & MOOC PlatformsCollaborative Learning & Peer TeachingUIST
HEPHA: A Mixed-Initiative Image Labeling Tool for Specialized DomainsImage labeling is an important task for training computer vision models. In specialized domains, such as healthcare, it is expensive and challenging to recruit specialists for image labeling. We propose HEPHA, a mixed-initiative image labeling tool that elicits human expertise via inductive logic learning to infer and refine labeling rules. Each rule comprises visual predicates that describe the image. HEPHA enables users to iteratively refine the rules by either direct manipulation through a visual programming interface or by labeling more images. To facilitate rule refinement, HEPHA recommends which rule to edit and which predicate to update. For users unfamiliar with visual programming, HEPHA suggests diverse and informative images to users for further labeling. We conducted a within-subjects user study with 16 participants and compared HEPHA with a variant of HEPHA and a deep learning-based approach. We found that HEPHA outperforms the two baselines in both specialized-domain and general-domain image labeling tasks. Our code is available at https://github.com/Neural-Symbolic-Image-Labeling/NSILWeb.2025SZShiyuan Zhou et al.Explainable AI (XAI)Interactive Data VisualizationMedical & Scientific Data VisualizationIUI
ReactGenie: A Development Framework for Complex Multimodal Interactions Using Large Language ModelsBy combining voice and touch interactions, multimodal interfaces can surpass the efficiency of either modality alone. Traditional multimodal frameworks require laborious developer work to support rich multimodal commands where the user’s multimodal command involves possibly exponential combinations of actions/function invocations. This paper presents ReactGenie, a programming framework that better separates multimodal input from the computational model to enable developers to create efficient and capable multimodal interfaces with ease. ReactGenie translates multimodal user commands into NLPL (Natural Language Programming Language), a programming language we created, using a neural semantic parser based on large-language models. The ReactGenie runtime interprets the parsed NLPL and composes primitives in the computational model to implement complex user commands. As a result, ReactGenie allows easy implementation and unprecedented richness in commands for end-users of multimodal apps. Our evaluation showed that 12 developers can learn and build a non-trivial ReactGenie application in under 2.5 hours on average. In addition, compared with a traditional GUI, end-users can complete tasks faster and with less task load using ReactGenie apps.2024JYJackie (Junrui) Yang et al.Stanford UniversityVoice User Interface (VUI) DesignGenerative AI (Text, Image, Music, Video)Human-LLM CollaborationCHI
"It’s Not What We Were Trying to Get At, but I Think Maybe It Should Be": Learning How to Do Trauma-Informed Design With a Data Donation Platform for Online Dating Sexual ViolenceA majority of people experience trauma, spurring calls to incorporate trauma-informed approaches (TIA) from public health and social work into technology design. While technologies touted as trauma-informed are starting to propagate the literature, there persists a gap in knowledge around how design teams apply TIA and qualify their technology as adhering to trauma-informed principles. We address this through a 12-month development project with trauma and sexual violence experts to produce Ube, a data donation platform for collecting online dating sexual consent data to improve sexual risk detection AI. Through analysis of design documentation we retrospectively articulate a trauma-informed design process that evolved through the course of Ube’s development, comprising three elements for integrating trauma-informed principles: design goals that adapt the definition of TIA to the application domain, design activities that map to trauma-informed principles, and consequent design choices. We conclude with methodological recommendations to improve trauma-informed design processes.2024WZWenqi Zheng et al.Oakland UniversityEmpowerment of Marginalized GroupsTechnology Ethics & Critical HCIParticipatory DesignCHI
Modeling the Trade-off of Privacy Preservation and Activity Recognition on Low-Resolution ImagesA computer vision system using low-resolution image sensors can provide intelligent services (e.g., activity recognition) but preserve unnecessary visual privacy information from the hardware level. However, preserving visual privacy and enabling accurate machine recognition have adversarial needs on image resolution. Modeling the trade-off of privacy preservation and machine recognition performance can guide future privacy-preserving computer vision systems using low-resolution image sensors. In this paper, using the at-home activity of daily livings (ADLs) as the scenario, we first obtained the most important visual privacy features through a user survey. Then we quantified and analyzed the effects of image resolution on human and machine recognition performance in activity recognition and privacy awareness tasks. We also investigated how modern image super-resolution techniques influence these effects. Based on the results, we proposed a method for modeling the trade-off of privacy preservation and activity recognition on low-resolution images.2023YWYuntao Wang et al.Tsinghua UniversityHuman Pose & Activity RecognitionPrivacy Perception & Decision-MakingCHI
Squeez'In: Private Authentication on Smartphones based on Squeezing GesturesIn this paper, we proposed \emph{Squeez'In}, a technique on smartphones that enabled private authentication by holding and squeezing the phone with a unique pattern. We first explored the design space of practical squeezing gestures for authentication by analyzing the participants' self-designed gestures and squeezing behavior. Results showed that varying-length gestures with two levels of touch pressure and duration were the most natural and unambiguous. We then implemented \emph{Squeez'In} on an off-the-shelf capacitive sensing smartphone, and employed an SVM-GBDT model for recognizing gestures and user-specific behavioral patterns, achieving 99.3\% accuracy and 0.93 F1-score when tested on 21 users. A following 14-day study validated the memorability and long-term stability of \proj. During usability evaluation, compared with gesture and pin code, \emph{Squeez'In} achieved significantly faster authentication speed and higher user preference in terms of privacy and security.2023XYXin Yi et al.Tsinghua UniversityForce Feedback & Pseudo-Haptic WeightPasswords & AuthenticationCHI
In-vehicle Performance and Distraction for Midair and Touch Directional GesturesWe compare the performance and level of distraction of expressive directional gesture input in the context of in-vehicle system commands. Center console touchscreen swipes and midair swipe-like movements are tested in 8-directions, with 8-button touchscreen tapping as a baseline. Participants use these input methods for intermittent target selections while performing the Lane Change Task in a virtual driving simulator. Input performance is measured with time and accuracy, cognitive load with deviation of lane position and speed, and distraction from frequency of off-screen glances. Results show midair gestures were less distracting and faster, but with lower accuracy. Touchscreen swipes and touchscreen tapping are comparable across measures. Our work provides empirical evidence for vehicle interface designers and manufacturers considering midair or touch directional gestures for centre console input.2023AHArman Hafizi et al.Computer ScienceIn-Vehicle Haptic, Audio & Multimodal FeedbackHand Gesture RecognitionCHI
Collection of Metaphors for Human-Robot InteractionThe word “robot” frequently conjures unrealistic expectations of utilitarian perfection: tireless, efficient, and flawless agents. However, real-world robots are far from perfect - they fail and make mistakes. Thus, roboticists should consider altering their current assumptions and cultivating new perspectives that account for a more complete range of robot roles, behaviors, and interactions. To encourage this, we explore the use of metaphors for generating novel ideas and reframing existing problems, eliciting new perspectives of human-robot interaction. Our work makes two contributions. We (1) surface current assumptions that accompany the term “robots,” and (2) present a collection of alternative perspectives of interaction with robots through metaphors. By identifying assumptions, we provide a comprehensible list of aspects to reconsider regarding robots’ physicality, roles, and behaviors. Through metaphors, we propose new ways of examining how we can use, relate to, and co-exist with the robots that will share our future.2021PAPatrícia Alves-Oliveira et al.Social Robot InteractionHuman-Robot Collaboration (HRC)Technology Ethics & Critical HCIDIS
IM Receptivity and Presentation-type Preferences among Users of a Mobile App with Automated Receptivity-status AdjustmentResearchers have long attempted to estimate instant-messaging (IM) users’ attentiveness, responsiveness, and interruptibility. Yet, IM users’ self-presentation of their receptivity, and their perceptions of automated adjustment/revelation of their receptivity status (e.g., Facebook Messenger’s green dot that deems a user to be “active”), remain under-explored. We therefore told our 43 participants that our IM app, IMStatus, was capable of automatically estimating and adjusting their receptivity status to responsive, attentive, or interruptible based on their smartphone activity. These statuses were also presented to their IM contacts in three different styles. Over a two-week period, the participants rarely chose the status interruptible, and when they did, it was usually to indicate low availability. Textual presentation was usually chosen to express statuses precisely, especially at high and low extremes of receptivity; while graphical and numeric presentations were preferred when self-perceived receptivity levels were more ambiguous. Conflicts between recipients’ and senders’ perspectives are also discussed.2021TWTing-Wei Wu et al.Computer ScienceContext-Aware ComputingNotification & Interruption ManagementCHI