Sprout: Using a Visual Metaphor to Support Customizable and Collaborative Health TrackingSelf-tracking tools can support health awareness and behavior change, though sustaining engagement remains difficult. Prior work has explored qualitative visualization, customization, and collaborative features to promote engagement; but little is known about how these strategies interact when combined. We present Sprout, a mobile application that integrates qualitative, customizable, and collaborative health tracking using a garden metaphor. Sprout allows users to choose what they track, customize how data is visually encoded, and participate in anonymous communities where collective progress unlocks shared features. In a 2-week field study with N=22 participants, users reported that qualitative displays worked best as a complement to quantitative tools, customization mostly happened during app setup, social features were the most engaging though collaboration produced both motivation and frustration, and anonymity protected privacy but limited social connection. Our findings show how multiple design strategies coexist in one system, sometimes competing and sometimes aligning in supporting users' tracking needs.2026PTPape Sow Traore et al.Dartmouth CollegeHealth Self-TrackingBehavior Change & Reflection TechnologyPrivacy & Data Ownership in Self-TrackingCHI
Family In The Loop: Enabling Family Involvement in Dementia Care at Long-term Facilities with Person-centered AI ToolsFamily involvement is crucial to person-centered dementia care, yet communication breakdowns, fragmented documentation, and resource constraints often hinder meaningful collaboration in long-term care facilities. This paper introduces Family In The Loop (FITL), an AI-enabled platform designed through a year of fieldwork and deployed across multiple facilities to strengthen person-centered care coordination between families and staff. FITL ingests passive data, (video streams and electronic health records), along with active user feedback, integrating these into four interconnected realtime views: Essentials View provides concise AI-generated care recommendations; Reels View offers personalized video highlights to reassure families; Explanations View answers user queries via a natural-language interface with links to supporting evidence; and Archives View ensures accountability by aggregating, indexing, and presenting the underlying care data in a structured historical record. Our contributions are: (1) empirical insights into how communication breakdowns, documentation gaps, and trust barriers undermine person-centered care and family involvement; (2) field-based design principles, developed in response to these challenges and instantiated in FITL, that enhance transparency and emotional reassurance without adding staff burden; (3) deployment findings detailing practical strategies for integrating AI tools into real-world care routines; and (4) a conceptual vision for scalable, data-driven, and role-sensitive person-centered care. These contributions advance CSCW by showing how integrated AI can support person-centered care in complex, resource-limited environments.2025DMDylan Edward Moore et al.Palliative CareCSCW
An Investigation of Interaction and Information Needs for Protocol Reverse Engineering AutomationProtocol reverse engineering (ProtocolREing) consists of taking streams of network data and inferring the communication protocol. ProtocolREing is critical task in malware and system security analysis. Several ProtocolREing automation tools have been developed, however, in practice, they are not used because they offer limited interaction. Instead, reverse engineers (ProtocolREs) perform this task manually or use less complex visualization tools. To give ProtocolREs the power of more complex automation, we must first understand ProtocolREs processes and information and interaction needs to design better interfaces. We interviewed 16 ProtocolREs, presenting a paper prototype ProtocolREing automation interface, and ask them to discuss their approach to ProtocolREing while using the tool and suggest missing information and interactions. We designed our prototype based on existing ProtocolREing tool features and prior reverse engineering research's usability guidelines. We found ProtocolREs follow a flexible, hypothesis-driven process and identified multiple information and interaction needs when validating the automation's inferences. We provide suggestions for future interaction design.2025SKSamantha Katcher et al.Tufts University, Department of Computer Science; MITRE CorporationExplainable AI (XAI)AI-Assisted Decision-Making & AutomationAlgorithmic Transparency & AuditabilityCHI
Enhancing the Educational Potential of Online Movement Videos: System Development and Empirical Studies with TikTok Dance ChallengesWe hypothesize that online movement videos have untapped potential for teaching physical skills, and we developed a platform that automatically generates practice plans from raw TikTok dance videos. The practice plans teach one segment at a time using fading guidance and part-learning principles and are presented using a web-based interface featuring concurrent visual aids. Two user studies (n=54, n=38) were conducted. The first showed significant improvements in learning outcomes compared to standard tutorials, underscoring the importance of well-structured practice plans and offering nuanced insights into the design and effectiveness of visual aids. The second study found that segmentation and emoji-based dual-coding only benefit learning when integrated into a well-designed lesson structure. We provide a set of practical recommendations for enhancing online movement learning, focusing on the need for substantive part-learning activities and careful use of visual aids to prevent cognitive overload.2025JBJules Brooks Blanchet et al.Dartmouth CollegeDance & Body Movement ComputingCHI
Therapy for Therapists: Design Opportunities to Support the Psychological Well-being of Mental Health WorkersOn-demand mental health services—including counseling, crisis hotlines, and peer support programs—are vital to the healthcare system, providing acute and ongoing support through telephone, online, and text-based communication. Although such services have proven effective at reducing hopelessness, psychological pain, and suicidality, they put mental health professionals at high risk of burnout, secondary traumatic stress, and compassion fatigue. Our interviews with mental health workers across professions from four mental health organizations revealed that while mental health workers have a strong motivation to help individuals struggling to meet their mental health needs, they face various challenges, including heavy caseloads, having to navigate dealing with crisis clients and managing the impact of abuse and harassment. Although organizations spend a significant time training workers prior to their involvement with clients, the training lacks components of self-compassion and self-care. To overcome their challenges, participants identify the need to be self-reliant and engage in care practices ranging from socializing with coworkers to yoga and meditation. Although technology is an integral part of their work routine, participants, irrespective of their age, had misapprehensions regarding technology use in the mental health care space and for managing one’s own psychological well-being. We recommend design guidelines for HCI researchers, including developing contextualized just-in-time adaptive interventions to promote self-compassion and educating workers regarding the use of various technologies to manage their psychological well-being.2024ACAishwarya Chandrasekaran et al.Session 1c: Care for the CaregiversCSCW
EarSE: Bringing Robust Speech Enhancement to COTS HeadphonesDuan 等人开发 EarSE 系统，基于深度学习为普通商业耳机提供稳健语音增强，使消费级耳机也能实现专业级降噪效果，语音信噪比提升 25dB。2024DDDi Duan et al.Voice User Interface (VUI) DesignIntelligent Voice Assistants (Alexa, Siri, etc.)UbiComp
PyroSense: 3D Posture Reconstruction Using Pyroelectric Infrared SensingZeng 等人提出 PyroSense 系统，利用热释电红外传感技术实现无摄像头的3D姿势重建，在保护隐私的同时提供高精度人体姿态感知。2024HZHuaili Zeng et al.Human Pose & Activity RecognitionBiosensors & Physiological MonitoringSmart Home Interaction DesignUbiComp
Laser-Powered Vibrotactile RenderingSu 等人提出激光触觉渲染技术，利用激光在皮肤表面产生热振感，实现无接触触觉反馈，为 VR/AR 提供新型交互方案。2024YSYuning Su et al.Mid-Air Haptics (Ultrasonic)UbiComp
ECSkin: Tessellating Electrochromic Films for Customizable On-skin DisplaysKu等人提出ECSkin，利用镶嵌电致变色薄膜制作可定制皮肤显示器，实现个性化可穿戴显示功能。2024PKPin-Sung Ku et al.Shape-Changing Interfaces & Soft Robotic MaterialsOn-Skin Display & On-Skin InputUbiComp
Symptom Detection with Text Message Log Distributions for Holistic Depression and Anxiety ScreeningReisch 等人利用短信日志分布特征进行症状检测，实现抑郁和焦虑障碍的自动化筛查。2024MRMiranda Reisch et al.Mental Health Apps & Online Support CommunitiesUbiComp
Capturing the College Experience: A Four-year Mobile Sensing Study of Mental Health, Resilience and Behavior of College Students during the PandemicNepal 等人通过四年移动传感研究，追踪疫情间大学生的心理健康、适应力与行为变化，为高校心理健康干预提供数据支持。2024SNSubigya Nepal et al.Mental Health Apps & Online Support CommunitiesSleep & Stress MonitoringUbiComp
StructCurves: Interlocking Block-Based Line StructuresWe present a new class of curved block-based line structures whose component chains are flexible when separated, and provably rigid when assembled together into an interlocking double chain. The joints are inspired by traditional zippers, where a binding fabric or mesh connects individual teeth. Unlike traditional zippers, the joint design produces a rigid interlock with programmable curvature. This allows fairly strong curved structures to be built out of easily stored flexible chains. In this paper, we introduce a pipeline for generating these curved structures using a novel block design template based on revolute joints. Mesh embedded in these structures maintains block spacing and assembly order. We evaluate the rigidity of the curved structures through mechanical performance testing and demonstrate several applications.2024ZSZezhou Sun et al.Shape-Changing Interfaces & Soft Robotic MaterialsShape-Changing Materials & 4D PrintingUIST
MoodCapture: Depression Detection using In-the-Wild Smartphone ImagesMoodCapture presents a novel approach that assesses depression based on images automatically captured from the front-facing camera of smartphones as people go about their daily lives. We collect over 125,000 photos in the wild from N=177 participants diagnosed with major depressive disorder for 90 days. Images are captured naturalistically while participants respond to the PHQ-8 depression survey question: "I have felt down, depressed, or hopeless''. Our analysis explores important image attributes, such as angle, dominant colors, location, objects, and lighting. We show that a random forest trained with face landmarks can classify samples as depressed or non-depressed and predict raw PHQ-8 scores effectively. Our post-hoc analysis provides several insights through an ablation study, feature importance analysis, and bias assessment. Importantly, we evaluate user concerns about using MoodCapture to detect depression based on sharing photos, providing critical insights into privacy concerns that inform the future design of in-the-wild image-based mental health assessment tools.2024SNSubigya Nepal et al.Dartmouth CollegeMental Health Apps & Online Support CommunitiesPrivacy by Design & User ControlBiosensors & Physiological MonitoringCHI
Teaching artificial intelligence in extracurricular contexts through narrative-based learnersourcingCollaborative technology provides powerful opportunities to engage young people in active learning experiences that are inclusive, immersive, and personally meaningful. In particular, interactive narratives have proven to be effective scaffolds for learning, and learnersourcing has emerged as a promising student-driven approach to enable personalized education and quality control at-scale. We introduce the first synthesis of these ideas in the context of teaching artificial intelligence (AI), which is now seen as a critical component of 21st-century education. Specifically, we explore the design of a narrative-based learnersourcing platform where engagement is centered around a learner-made choose-your-own-adventure story. In grounding our approach, we draw from pedagogical literature, digital storytelling, and recent work on learnersourcing. We report on our iterative, learner-centered design process as well as our study findings that demonstrate the platform’s positive effects on knowledge gains, interest in AI concepts, and the overall user experience of narrative-based learnersourcing technology.2024DMDylan Edward Moore et al.Dartmouth CollegeSTEM Education & Science CommunicationInteractive Narrative & Immersive StorytellingCHI
Sketching AI Concepts with Capabilities and Examples: AI Innovation in the Intensive Care UnitAdvances in artificial intelligence (AI) have enabled unprecedented capabilities, yet innovation teams struggle when envisioning AI concepts. Data science teams think of innovations users do not want, while domain experts think of innovations that cannot be built. A lack of effective ideation seems to be a breakdown point. How might multidisciplinary teams identify buildable and desirable use cases? This paper presents a first hand account of ideating AI concepts to improve critical care medicine. As a team of data scientists, clinicians, and HCI researchers, we conducted a series of design workshops to explore more effective approaches to AI concept ideation and problem formulation. We detail our process, the challenges we encountered, and practices and artifacts that proved effective. We discuss the research implications for improved collaboration and stakeholder engagement, and discuss the role HCI might play in reducing the high failure rate experienced in AI innovation.2024NYNur Yildirim et al.Carnegie Mellon UniversityGenerative AI (Text, Image, Music, Video)Human-LLM CollaborationMental Health Apps & Online Support CommunitiesCHI
The Power of Speech in the Wild: Discriminative Power of Daily Voice Diaries in Understanding Auditory Verbal Hallucinations Using Deep Learning"Mobile phone sensing is increasingly being used in clinical research studies to assess a variety of mental health conditions (e.g., depression, psychosis). However, in-the-wild speech analysis -- beyond conversation detecting -- is a missing component of these mobile sensing platforms and studies. We augment an existing mobile sensing platform with a daily voice diary to assess and predict the severity of auditory verbal hallucinations (i.e., hearing sounds or voices in the absence of any speaker), a condition that affects people with and without psychiatric or neurological diagnoses. We collect 4809 audio diaries from N=384 subjects over a one-month-long study period. We investigate the performance of various deep-learning architectures using different combinations of sensor behavioral streams (e.g., voice, sleep, mobility, phone usage, etc.) and show the discriminative power of solely using audio recordings of speech as well as automatically generated transcripts of the recordings; specifically, our deep learning model achieves a weighted f-1 score of 0.78 solely from daily voice diaries. Our results surprisingly indicate that a simple periodic voice diary combined with deep learning is sufficient enough of a signal to assess complex psychiatric symptoms (e.g., auditory verbal hallucinations) collected from people in the wild as they go about their daily lives." https://doi.org/10.1145/36108902023WWWeichen Wang et al.Brain-Computer Interface (BCI) & NeurofeedbackMental Health Apps & Online Support CommunitiesUbiComp
“It’s Not an Issue of Malice, but of Ignorance”: Towards Inclusive Video Conferencing for Presenters Who are d/Deaf or Hard of Hearing"As video conferencing (VC) has become necessary for many professional, educational, and social tasks, people who are d/Deaf and hard of hearing (DHH) face distinct accessibility barriers. We conducted studies to understand the challenges faced by DHH people during VCs and found that they struggled to easily present or communicate effectively due to accessibility limitations of VC platforms. These limitations include the lack of tools for DHH speakers to discreetly communicate their accommodation needs to the group. Based on these findings, we prototyped a suite of tools, called Erato that enables DHH speakers to be aware of their performance while speaking and remind participants of proper etiquette. We evaluated Erato by running a mock classroom case study over VC for three sessions. All participants felt more confident in their speaking ability and paid closer attention to making the classroom more inclusive while using our tool. We share implications of these results for the design of VC interfaces and human-the-the-loop assistive systems that can support users who are DHH to communicate effectively and advocate for their accessibility needs." https://doi.org/10.1145/36109012023JDJosh Urban Davis et al.Deaf & Hard-of-Hearing Support (Captions, Sign Language, Vibration)Universal & Inclusive DesignUbiComp
Narrative-Based Visual Feedback to Encourage Sustained Physical Activity: A Field Trial of the WhoIsZuki Mobile Health Platform"Stories are a core way human beings make meaning and sense of the world and our lived experiences, including our behaviors, desires, and goals. Narrative structures, both visual and textual, help us understand and act on information, while also evoking strong emotions. Focusing on the health context, this research examines the effectiveness of narrative-based feedback in motivating physical activity behaviors and underlying attitudes over longitudinal periods. After collecting two weeks of baseline physical activity levels, N=39 participants installed our smartphone application, WhoIsZuki. The WhoIsZuki app supports goal setting and semi-automated activity tracking, and it provides an ambient display that visually encodes these tracked activities as well as progress toward goals. Half of participants received a version of the interface that supplied behavioral feedback in the form of a multi-chapter episodic narrative, while the other half received a control condition version that provided an aesthetically-similar visualization but without any characterization, episodic structure, dramatic effect, or other narrative elements. After interacting with these versions for four months, our analysis showed that participants receiving the multi-chapter narrative feedback performed more physical activity, achieved more goals, experienced more positive psychological shifts, and overall engaged more meaningfully with the digital intervention. https://dl.acm.org/doi/10.1145/3580786"2023EMElizabeth L Murnane et al.Mental Health Apps & Online Support CommunitiesFitness Tracking & Physical Activity MonitoringInteractive Narrative & Immersive StorytellingUbiComp
GLOBEM: Cross-Dataset Generalization of Longitudinal Human Behavior ModelingThere is a growing body of research revealing that longitudinal passive sensing data from smartphones and wearable devices can capture daily behavior signals for human behavior modeling, such as depression detection. Most prior studies build and evaluate machine learning models using data collected from a single population. However, to ensure that a behavior model can work for a larger group of users, its generalizability needs to be verified on multiple datasets from different populations. We present the first work evaluating cross-dataset generalizability of longitudinal behavior models, using depression detection as an application. We collect multiple longitudinal passive mobile sensing datasets with over 500 users from two institutes over a two-year span, leading to four institute-year datasets. Using the datasets, we closely re-implement and evaluated nine prior depression detection algorithms. Our experiment reveals the lack of model generalizability of these methods. We also implement eight recently popular domain generalization algorithms from the machine learning community. Our results indicate that these methods also do not generalize well on our datasets, with barely any advantage over the naive baseline of guessing the majority. We then present two new algorithms with better generalizability. Our new algorithm, Reorder, significantly and consistently outperforms existing methods on most cross-dataset generalization setups. However, the overall advantage is incremental and still has great room for improvement. Our analysis reveals that the individual differences (both within and between populations) may play the most important role in the cross-dataset generalization challenge. Finally, we provide an open-source benchmark platform GLOBEM- short for Generalization of Longitudinal BEhavior Modeling - to consolidate all 19 algorithms. GLOBEM can support researchers in using, developing, and evaluating different longitudinal behavior modeling methods. We call for researchers' attention to model generalizability evaluation for future longitudinal human behavior modeling studies. https://dl.acm.org/doi/10.1145/35694852023XXXuhai Xu et al.Human Pose & Activity RecognitionMental Health Apps & Online Support CommunitiesBiosensors & Physiological MonitoringUbiComp
From Gap to Synergy: Enhancing Contextual Understanding through Human-Machine Collaboration in Personalized SystemsThis paper presents LangAware, a collaborative approach for constructing personalized context for context-aware applications. The need for personalization arises due to significant variations in context between individuals based on scenarios, devices, and preferences. However, there is often a notable gap between humans and machines in the understanding of how contexts are constructed, as observed in trigger-action programming studies such as IFTTT. LangAware enables end-users to participate in establishing contextual rules in-situ using natural language. The system leverages large language models (LLMs) to semantically connect low-level sensor detectors to high-level contexts and provide understandable natural language feedback for effective user involvement. We conducted a user study with 16 participants in real-life settings, which revealed an average success rate of 87.50% for defining contextual rules in a variety of 12 campus scenarios, typically accomplished within just two modifications. Furthermore, users reported a better understanding of the machine's capabilities by interacting with LangAware.2023WCWeihao Chen et al.Human-LLM CollaborationAI-Assisted Decision-Making & AutomationContext-Aware ComputingUIST