Hear You in Silence: Designing for Active Listening in Human Interaction with Conversational Agents Using Context-Aware PacingIn human conversation, empathic dialogue requires nuanced temporal cues indicating whether the conversational partner is paying attention. This type of "active listening" is overlooked in the design of Conversational Agents (CAs), which use the same pacing for one conversation. To model the temporal cues in human conversation, we need CAs that dynamically adjust response pacing according to user input. We qualitatively analyzed ten cases of active listening to distill five context-aware pacing strategies: Reflective Silence, Facilitative Silence, Empathic Silence, Holding Space, and Immediate Response. In a between-subjects study (N=50) with two conversational scenarios (relationship and career-support), the context-aware agent scored higher than static-pacing control on perceived human-likeness, smoothness, and interactivity, supporting deeper self-disclosure and higher engagement. In the career-support scenario, the CA yielded higher perceived listening quality and affective trust. This work shows how insights from human conversation like context-aware pacing can empower the design of more empathic human-AI communication.2026ZJZhihan Jiang et al.The University of Hong KongConversational ChatbotsAffective Human-Computer DialogueAgent Personality & AnthropomorphismCHI
Augmenting Clinical Decision-Making with an Interactive and Interpretable AI Copilot: A Real-World User Study with Clinicians in Nephrology and ObstetricsClinician skepticism toward opaque AI hinders adoption in high-stakes healthcare. We present AICare, an interactive and interpretable AI copilot for collaborative clinical decision-making. By analyzing longitudinal electronic health records, AICare grounds dynamic risk predictions in scrutable visualizations and LLM-driven diagnostic recommendations. Through a within-subjects counterbalanced study with 16 clinicians across nephrology and obstetrics, we comprehensively evaluated AICare using objective measures (task completion time and error rate), subjective assessments (NASA-TLX, SUS, and confidence ratings), and semi-structured interviews. Our findings indicate AICare's reduced cognitive workload. Beyond performance metrics, qualitative analysis reveals that trust is actively constructed through verification, with interaction strategies diverging by expertise: junior clinicians used the system as cognitive scaffolding to structure their analysis, while experts engaged in adversarial verification to challenge the AI's logic. This work offers design implications for creating AI systems that function as transparent partners, accommodating diverse reasoning styles to augment rather than replace clinical judgment.2026YZYinghao Zhu et al.Peking UniversityExplainable AI (XAI)AI-Assisted Decision-Making & AutomationEV Charging & Eco-Driving InterfacesCHI
MuseForge: Enhancing Creative Learning in Digital Museum Education with Generative AICreative learning has enriched on-site museum education by fostering engagement, exploration, and active participation. However, the structured integration of creative learning processes into digital museum education remains relatively underexplored. Although Generative Artificial Intelligence (GenAI) presents considerable potential to support creative learning, its comprehensive application across all stages in non-formal learning environments, such as museums, remains limited. To investigate this potential, we conducted a formative study employing a previously developed prototype with seven learners and four senior experts to identify learners’ specific needs and challenges. Informed by the findings, we designed and developed MuseForge, a platform integrating GenAI with the five stages of the iterative creative development path to support a personalized and dynamic creative learning experience. A between-subjects study with 32 participants demonstrated that learners using MuseForge achieved significantly higher learning motivation, engagement, and learning gain in creative self-efficacy, highlighting its effectiveness in supporting creative learning in digital museum environments.2026WLWeiyue Lin et al.The University of Hong KongGenerative AI (Text, Image, Music, Video)Museum & Cultural Heritage DigitizationSTEM Education & Science CommunicationCHI
Mapping the Wizards' Path: A Systematic Review of Wizard-of-Oz in HCIThe Wizard-of-Oz (WoZ) method has long been a core prototyping technique in Human-Computer Interaction (HCI), in which users interact with systems that seem autonomous but are actually controlled by hidden human operators. Advances in interactive technologies have expanded the landscape of future system behaviors, broadening both where and how WoZ is used. However, as more envisioned behaviors become technically feasible, the distinction between engineering a system and simulating an interaction becomes blurred, making it essential to clarify when and why to employ wizarding. This paper presents the first systematic review of WoZ in HCI, drawing on 194 papers from SIGCHI venues to identify ten application domains, five wizard control types, eight motivations, and five categories of concerns. Building on these findings, we propose a reciprocal evolution framework that interprets how technology and wizarding shape each other, and derive guidelines for the rigorous application of WoZ. We further illustrate the framework through emerging prototyping practices with Large Language Models (LLMs).2026RYRuoxuan Yang et al.The University of Hong KongParticipatory DesignUser Research Methods (Interviews, Surveys, Observation)Prototyping & User TestingCHI
MindfulAgents: Personalizing Mindfulness Meditation via an Expert-Aligned Multi-Agent SystemMindfulness meditation is a widely accessible and evidence-based method for supporting mental health. Despite the proliferation of mindfulness meditation apps, sustaining user engagement remains a persistent challenge. Personalizing the meditation experience is a promising strategy to improve engagement, but it often requires costly and unscalable manual effort. We present MindfulAgents, a multi-agent system powered by large language models that: (1) generates guided meditation scripts based on an expert-established mindfulness framework, (2) encourages users' reflection on emotional states and mindfulness skills, and (3) enables real-time personalization of the mindfulness meditation experience for each user. In a formative lab study (N=13), MindfulAgents significantly improved in-session engagement (p = 0.011) and self-awareness (p = 0.014), as well as reduced momentary stress (p = 0.020). Furthermore, a four-week deployment study (N=62) demonstrated a notable increase (p = 0.002) in long-term engagement and level of mindfulness (p = 0.023). Participants reported that MindfulAgents offered more relevant meditation sessions personalized to individual needs in various contexts, supporting sustained practice. Our findings highlight the potential of LLM-driven personalization for enhancing user engagement in digital mindfulness meditation interventions.2026MWMengyuan Wu et al.Columbia UniversityMental Health Apps & Online Support CommunitiesAffective Human-Computer DialogueHuman-LLM CollaborationCHI
CoGrader: Transforming Instructors' Assessment of Project Reports through Collaborative LLM IntegrationGrading project reports are increasingly significant in today’s educational landscape, where they serve as key assessments of students' comprehensive problem-solving abilities. However, it remains challenging due to the multifaceted evaluation criteria involved, such as creativity and peer-comparative achievement. Meanwhile, instructors often struggle to maintain fairness throughout the time-consuming grading process. Recent advances in AI, particularly large language models, have demonstrated potential for automating simpler grading tasks, such as assessing quizzes or basic writing quality. However, these tools often fall short when it comes to complex metrics, like design innovation and the practical application of knowledge, that require an instructor’s educational insights into the class situation. To address this challenge, we conducted a formative study with six instructors and developed CoGrader, which introduces a novel grading workflow combining human-LLM collaborative metrics design, benchmarking, and AI-assisted feedback. CoGrader was found effective in improving grading efficiency and consistency while providing reliable peer-comparative feedback to students. We also discuss design insights and ethical considerations for the development of human-AI collaborative grading systems.2025ZCZixin Chen et al.Human-LLM CollaborationIntelligent Tutoring Systems & Learning AnalyticsSTEM Education & Science CommunicationUIST
NoteIt: A System Converting Instructional Videos to Interactable Notes Through Multimodal Video UnderstandingUsers often take notes for instructional videos to access key knowledge later without revisiting long videos. Automated note generation tools enable users to obtain informative notes efficiently. However, notes generated by existing research or off-the-shelf tools fail to preserve the information conveyed in the original videos comprehensively, nor can they satisfy users’ expectations for diverse presentation formats and interactive features when using notes digitally. In this work, we present NoteIt, a system, which automatically converts instructional videos to interactable notes using a novel pipeline that faithfully extracts hierarchical structure and multimodal key information from videos. With NoteIt’s interface, users can interact with the system to further customize the content and presentation formats of the notes according to their preferences. We conducted both a technical evaluation and a comparison user study (N=36). The solid performance in objective metrics and the positive user feedback demonstrated the effectiveness of the pipeline and the overall usability of NoteIt.2025RZRunning Zhao et al.Voice User Interface (VUI) DesignData StorytellingOnline Learning & MOOC PlatformsUIST
JournalAIde: Empowering Older Adults in Digital Journal WritingDigital journaling offers a means for older adults to express themselves, document their lives, and engage in self-reflection, contributing to the maintenance of cognitive function and social connectivity. Although previous works have investigated the motivations and benefits of digital journaling for older adults, little technical support has been designed to offer assistance. We conducted a formative study with older adults and uncovered their encountered challenges and preferences for technical support. Informed by the findings, we designed a Large Language Model (LLM) empowered tool, JournalAIde, which provides vicarious experience, idea organization, sample text generation, and visual editing cues to enhance older adults’ confidence, writing ability, and sustained attention during digital journaling. Through a between-subjects study and a field deployment, we demonstrated the JournalAIde’s significant effectiveness compared to a baseline system in empowering older adults in digital journaling. We further investigated older adults' experiences and perceptions of LLM writing assistance.2025SZShixu Zhou et al.The Hong Kong University of Science and Technology (Guangzhou); Hong Kong University of Science and TechnologyHuman-LLM CollaborationAging-Friendly Technology DesignAI-Assisted Creative WritingCHI
"Ronaldo's a poser!": How the Use of Generative AI Shapes Debates in Online ForumsOnline debates can enhance critical thinking but may escalate into hostile attacks. As humans are increasingly reliant on Generative AI (GenAI) in writing tasks, we need to understand how people utilize GenAI in online debates. To examine the patterns of writing behavior while making arguments with GenAI, we created an online forum for soccer fans to engage in turn-based and free debates in a post format with the assistance of ChatGPT, arguing on the topic of "Messi vs Ronaldo". After 13 sessions of two-part study and semi-structured interviews with 39 participants, we conducted content and thematic analyses to integrate insights from interview transcripts, ChatGPT records, and forum posts. We found that participants prompted ChatGPT for aggressive responses, created posts with similar content and logical fallacies, and sacrificed the use of ChatGPT for better human-human communication. This work uncovers how polarized forum members work with GenAI to engage in debates online.2025YZYuhan Zeng et al.City University of Hong Kong, Department of Computer ScienceGenerative AI (Text, Image, Music, Video)Social Platform Design & User BehaviorMisinformation & Fact-CheckingCHI
RFBoost: Understanding and Boosting Deep WiFi Sensing via Physical Data AugmentationHou 等人提出 RFBoost 框架，通过创新的物理层数据增强技术提升深度 WiFi 感知模型的泛化能力，在人体行为识别任务中准确率提升 18%。2024WHWeiying Hou et al.Context-Aware ComputingComputational Methods in HCIUbiComp
CrowdBot: An Open-Environment Robot Management System for On-Campus ServicesWang 等人设计 CrowdBot 开放环境机器人管理系统，实现校园场景下机器人的自主导航与任务调度，为校园服务机器人的高效管理提供解决方案。2024YWYufei Wang et al.Domestic RobotsSocial Robot InteractionUbiComp
Rethinking Human-AI Collaboration in Complex Medical Decision Making: A Case Study in Sepsis DiagnosisToday's AI systems for medical decision support often succeed on benchmark datasets in research papers but fail in real-world deployment. This work focuses on the decision making of sepsis, an acute life-threatening systematic infection that requires an early diagnosis with high uncertainty from the clinician. Our aim is to explore the design requirements for AI systems that can support clinical experts in making better decisions for the early diagnosis of sepsis. The study begins with a formative study investigating why clinical experts abandon an existing AI-powered Sepsis predictive module in their electrical health record (EHR) system. We argue that a human-centered AI system needs to support human experts in the intermediate stages of a medical decision-making process (e.g., generating hypotheses or gathering data), instead of focusing only on the final decision. Therefore, we build SepsisLab based on a state-of-the-art AI algorithm and extend it to predict the future projection of sepsis development, visualize the prediction uncertainty, and propose actionable suggestions (i.e., which additional laboratory tests can be collected) to reduce such uncertainty. Through heuristic evaluation with six clinicians using our prototype system, we demonstrate that \system enables a promising human-AI collaboration paradigm for the future of AI-assisted sepsis diagnosis and other high-stakes medical decision making.2024SZShao Zhang et al.Northeastern UniverisityExplainable AI (XAI)AI-Assisted Decision-Making & AutomationMedical & Scientific Data VisualizationCHI
Persuasion or Insulting? Unpacking Discursive Strategies of Gender Debate in Everyday Feminism in ChinaSpeaking out for women's daily needs on social media has become a crucial form of everyday feminism in China. Gender debate naturally intertwines with such feminist advocacy, where users in opposite stances discuss gender-related issues through intense discourse. The complexities of gender debate necessitate a systematic understanding of discursive strategies for achieving effective gender communication that balances civility and constructiveness. To address this problem, we adopted a mixed-methods study to navigate discursive strategies in gender debate, focusing on 38,636 posts and 187,539 comments from two representative cases in China. Through open coding, we identified a comprehensive taxonomy of linguistic strategies in gender debate, capturing five overarching themes including derogation, gender distinction, intensification, mitigation, and cognizance guidance. Further, we applied regression analysis to unveil these strategies' correlations with user participation and response, illustrating the tension between debating tactics and public engagement. We discuss design implications to facilitate feminist advocacy on social media. Content Warning: This paper contains discussions on gender debate that may include swear words and sensitive topics, such as sex, potentially causing discomfort.2024YDYue DENG et al.The Hong Kong University of Science and TechnologyGender & Race Issues in HCIEmpowerment of Marginalized GroupsCHI
mmStress: Distilling Human Stress from Daily Activities via Contact-less Millimeter-wave Sensing"Long-term exposure to stress hurts human's mental and even physical health,and stress monitoring is of increasing significance in the prevention, diagnosis, and management of mental illness and chronic disease. However, current stress monitoring methods are either burdensome or intrusive, which hinders their widespread usage in practice. In this paper, we propose mmStress, a contact-less and non-intrusive solution, which adopts a millimeter-wave radar to sense a subject's activities of daily living, from which it distills human stress. mmStress is built upon the psychologically-validated relationship between human stress and "displacement activities", i.e., subjects under stress unconsciously perform fidgeting behaviors like scratching, wandering around, tapping foot, etc. Despite the conceptual simplicity, to realize mmStress, the key challenge lies in how to identify and quantify the latent displacement activities autonomously, as they are usually transitory and submerged in normal daily activities, and also exhibit high variation across different subjects. To address these challenges, we custom-design a neural network that learns human activities from both macro and micro timescales and exploits the continuity of human activities to extract features of abnormal displacement activities accurately. Moreover, we also address the unbalance stress distribution issue by incorporating a post-hoc logit adjustment procedure during model training. We prototype, deploy and evaluate mmStress in ten volunteers' apartments for over four weeks, and the results show that mmStress achieves a promising accuracy of ~80% in classifying low, medium and high stress. In particular, mmStress manifests advantages, particularly under free human movement scenarios, which advances the state-of-the-art that focuses on stress monitoring in quasi-static scenarios." https://doi.org/10.1145/36109262023KLKun Liang et al.Human Pose & Activity RecognitionSleep & Stress MonitoringBiosensors & Physiological MonitoringUbiComp
Radio2Text: Streaming Speech Recognition Using mmWave Radio Signals"Millimeter wave (mmWave) based speech recognition provides more possibility for audio-related applications, such as conference speech transcription and eavesdropping. However, considering the practicality in real scenarios, latency and recognizable vocabulary size are two critical factors that cannot be overlooked. In this paper, we propose Radio2Text, the first mmWave-based system for streaming automatic speech recognition (ASR) with a vocabulary size exceeding 13,000 words. Radio2Text is based on a tailored streaming Transformer that is capable of effectively learning representations of speech-related features, paving the way for streaming ASR with a large vocabulary. To alleviate the deficiency of streaming networks unable to access entire future inputs, we propose the Guidance Initialization that facilitates the transfer of feature knowledge related to the global context from the non-streaming Transformer to the tailored streaming Transformer through weight inheritance. Further, we propose a cross-modal structure based on knowledge distillation (KD), named cross-modal KD, to mitigate the negative effect of low quality mmWave signals on recognition performance. In the cross-modal KD, the audio streaming Transformer provides feature and response guidance that inherit fruitful and accurate speech information to supervise the training of the tailored radio streaming Transformer. The experimental results show that our Radio2Text can achieve a character error rate of 5.7% and a word error rate of 9.4% for the recognition of a vocabulary consisting of over 13,000 words." https://doi.org/10.1145/36108732023RZRunning Zhao et al.Voice User Interface (VUI) DesignIntelligent Voice Assistants (Alexa, Siri, etc.)UbiComp
CrowdQ: Predicting the Queue State of Hospital Emergency Department Using Crowdsensing Mobility Data-Driven Models"Hospital Emergency Departments (EDs) are essential for providing emergency medical services, yet often overwhelmed due to increasing healthcare demand. Current methods for monitoring ED queue states, such as manual monitoring, video surveillance, and front-desk registration are inefficient, invasive, and delayed to provide real-time updates. To address these challenges, this paper proposes a novel framework, CrowdQ, which harnesses spatiotemporal crowdsensing data for real-time ED demand sensing, queue state modeling, and prediction. By utilizing vehicle trajectory and urban geographic environment data, CrowdQ can accurately estimate emergency visits from noisy traffic flows. Furthermore, it employs queueing theory to model the complex emergency service process with medical service data, effectively considering spatiotemporal dependencies and event context impact on ED queue states. Experiments conducted on large-scale crowdsensing urban traffic datasets and hospital information system datasets from Xiamen City demonstrate the framework's effectiveness. It achieves an F1 score of 0.93 in ED demand identification, effectively models the ED queue state of key hospitals, and reduces the error in queue state prediction by 18.5%-71.3% compared to baseline methods. CrowdQ, therefore, offers valuable alternatives for public emergency treatment information disclosure and maximized medical resource allocation." https://doi.org/10.1145/36108752023TSTieqi Shou et al.Content Moderation & Platform GovernancePublic Transit & Trip PlanningUbiComp
UQRCom: Underwater Wireless Communication Based on QR Code"While communication in the air has been a norm with the pervasiveness of WiFi and LTE infrastructure, underwater communication still faces a lot of challenges. Even nowadays, the main communication method for divers in underwater environment is hand gesture. There are multiple issues associated with gesture-based communication including limited amount of information and ambiguity. On the other hand, traditional RF-based wireless communication technologies which have achieved great success in the air can hardly work in underwater environment due to the extremely severe attenuation. In this paper, we propose UQRCom, an underwater wireless communication system designed for divers. We design a UQR code which stems from QR code and address the unique challenges in underwater environment such as color cast, contrast reduction and light interfere. With both real-world experiments and simulation, we show that the proposed system can achieve robust real-time communication in underwater environment. For UQR codes with a size of 19.8 cm x 19.8 cm, the communication distance can be 11.2 m and the achieved data rate (6.9 kbps ~ 13.6 kbps) is high enough for voice communication between divers. https://dl.acm.org/doi/10.1145/3571588"2023XLTieqi Shou et al.Ubiquitous ComputingUbiComp
A Data-Driven Context-Aware Health Inference System for Children during School Closures"Many countries have implemented school closures due to the outbreak of the COVID-19 pandemic, which has inevitably affected children's physical and mental health. It is vital for parents to pay special attention to their children's health status during school closures. However, it is difficult for parents to recognize the changes in their children's health, especially without visible symptoms, such as psychosocial functioning in mental health. Moreover, healthcare resources and understanding of the health and societal impact of COVID-19 are quite limited during the pandemic. Against this background, we collected real-world datasets from 1,172 children in Hong Kong during four time periods under different pandemic and school closure conditions from September 2019 to January 2022. Based on these data, we first perform exploratory data analysis to explore the impact of school closures on six health indicators, including physical activity intensity, physical functioning, self-rated health, psychosocial functioning, resilience, and connectedness. We further study the correlation between children's contextual characteristics (i.e., demographics, socioeconomic status, electronic device usage patterns, financial satisfaction, academic performance, sleep pattern, exercise habits, and dietary patterns) and the six health indicators. Subsequently, a health inference system is designed and developed to infer children's health status based on their contextual features to derive the risk factors of the six health indicators. The evaluation and case studies on real-world datasets show that this health inference system can help parents and authorities better understand key factors correlated with children's health status during school closures. https://doi.org/10.1145/3580800"2023ZJZhihan Jiang et al.Cognitive Impairment & Neurodiversity (Autism, ADHD, Dyslexia)Mental Health Apps & Online Support CommunitiesUbiComp
RingVKB: A Ring-Shaped Virtual Keyboard Using Low-Cost IMUWearable devices have been important components for ubiquitous computing. However, text input remains challenging on wearables due to the lack of a physical keyboard. In this paper, we propose a novel ring-shaped virtual keyboard system named RingVKB for convenient text input using low-cost IMUs available on any wearables. At the core of RingVKB are two novel designs: 1) A circular keyboard layout with 12 equal sectors, which assembles all common keys on classical keyboards while allowing users to type with only one finger effectively, and 2) an error control algorithm that calculates the relative displacement of keystrokes from the noisy IMU sensor data. The two components, coupled together, enable high-accuracy and efficient text input for ubiquitous scenarios. We implement RingVKB using a small device consisting of a microcontroller and a MEMS sensor, which can be attached to the user's index finger. Experimental results show that RingVKB can effectively improve the relative displacement estimation accuracy, and achieves an overall keystroke recognition accuracy of 93% for 25 key positions. A user study also shows that RingVKB is easy to learn and use. Using only low-cost IMU sensors, RingVKB provides a virtual keyboard solution that can be widely adopted on wearables.2023ZLZhenjiang Li et al.Haptic WearablesFoot & Wrist InteractionUbiquitous ComputingMobileHCI
A Personalized Visual Aid for Selections of Appearance Building Products with Long-term EffectsIt is challenging for customers to select appearance building products (e.g., skincare products, weight loss programs) that suit them personally as such products usually demonstrate efficacy only after long-term usage. Although e-retailers generally provide product descriptions or other customers' reviews, users often find it hard to relate to their own situations. In this work, we proposed a pipeline to display envisioned users' appearance after long-term use of appearance building products to deliver their efficacy on each individual visually. We selected skincare as a case and developed SkincareMirror which predicts skincare effects on users' facial images by analyzing product function labels, efficacy ratings, and skin models' images. The results of a between-subjects study (N=48) show that (1) SkincareMirror outperforms the baseline shopping site in terms of perceived usability, usefulness, user satisfaction and helps users select products faster; (2) SkincareMirror is especially effective to males and users with limited product domain knowledge.2022CSChuhan Shi et al.Hong Kong University of Science and TechnologyRecommender System UXInteractive Data VisualizationCHI