SmartWalkCoach: An AI Companion for End-to-End Walking Guidance, Motivation, and ReflectionWe present SmartWalkCoach, a mobile AI companion that supports the full walking journey: from pre-walk planning to in-walk guidance through to post-walk reflection. Addressing a gap between map navigation and motivational coaching, SmartWalkCoach orchestrates three lightweight agents: (1) GeographyAgent for conversational route curation from nearby points of interest and user preferences while delegating pathfinding to map APIs; (2) AccompanyAgent for context-aware, just-in-time prompts that blend informational cues with relational encouragement; and (3) SummaryAgent for concise reflection and next-step planning. This end-to-end, tool-using design aims to lower cognitive load in planning and sustain engagement and motivation during walking through delivering dynamic, cadence-aware interventions. We conducted an in-the-wild, two-period AB/BA crossover study (N=12), where each participant completed two comparable walks with counterbalanced conditions: Information-only versus Information+Motivation. Linear mixed models show that adding motivational, companion-like dialogue significantly improved outcomes: participants reported higher positive feelings and better user experience, with no evidence of carryover. Thematic analysis surfaced two design imperatives for mobile companions: supportive, relational expression and context-aware timing (e.g., avoiding high-load moments, intervening at fatigue/milestones). Our contributions are: (i) an end-to-end, tool-using agent architecture for everyday walking that reduces cognitive load during planning and accompaniment; (ii) a controlled field evaluation linking context-aware motivation to affect and UX gains; and (iii) actionable design guidance on expression, timing, and frequency for mHealth companions. We outline limitations and paths toward multimodal, voice-first companions, with adaptive personalization mechanisms.2026XZXianzhe Zhang et al.Xi'an Jiaotong-Liverpool UniversityFitness Tracking & Physical Activity MonitoringBehavior Change & Reflection TechnologyContext-Aware ComputingIUI
N-ary Gaussian Model Modeling Pointing Uncertainty Across Task Scenarios Using an Automated Multi-Gaussian Modeling PipelineThis paper presents an N-ary Gaussian Model for predicting endpoint distributions in pointing tasks across task scenarios. Built on the foundational principles of the Ternary Gaussian model series, our model framework allows researchers to define parameter constraints and automatically refine model combinations, eliminating the need for predefined equations based on data analysis. We utilize the Bayesian Information Criterion (BIC) for model selection, ensuring simplicity while maintaining predictive accuracy. We conducted a comparative analysis against published baselines across 7 diverse datasets, covering 1D, 2D, and 3D tasks, different input modalities, different display devices, and time-constrained scenarios, demonstrating the robustness and generalization of the N-ary Gaussian Model. The N-ary Gaussion model offers an automated solution for modeling pointing uncertainty, and also incorporates cross output device, input modality, and temporal constraint factors into spatial pointing uncertainty modeling for the first time.2026HZHao Zhang et al.Chinese Academy of SciencesTouchscreen Usability & Performance Modeling (Fitts' Law)Touch Target Selection & PointingComputational Methods in HCICHI
Augmenting Clinical Decision-Making with an Interactive and Interpretable AI Copilot: A Real-World User Study with Clinicians in Nephrology and ObstetricsClinician skepticism toward opaque AI hinders adoption in high-stakes healthcare. We present AICare, an interactive and interpretable AI copilot for collaborative clinical decision-making. By analyzing longitudinal electronic health records, AICare grounds dynamic risk predictions in scrutable visualizations and LLM-driven diagnostic recommendations. Through a within-subjects counterbalanced study with 16 clinicians across nephrology and obstetrics, we comprehensively evaluated AICare using objective measures (task completion time and error rate), subjective assessments (NASA-TLX, SUS, and confidence ratings), and semi-structured interviews. Our findings indicate AICare's reduced cognitive workload. Beyond performance metrics, qualitative analysis reveals that trust is actively constructed through verification, with interaction strategies diverging by expertise: junior clinicians used the system as cognitive scaffolding to structure their analysis, while experts engaged in adversarial verification to challenge the AI's logic. This work offers design implications for creating AI systems that function as transparent partners, accommodating diverse reasoning styles to augment rather than replace clinical judgment.2026YZYinghao Zhu et al.Peking UniversityExplainable AI (XAI)AI-Assisted Decision-Making & AutomationEV Charging & Eco-Driving InterfacesCHI
Intrinsic vs. Extrinsic Programming Challenges in Educational Games: How they shape Children’s Computational Thinking, Learning Drive, and Game EngagementEducational programming games (EPGs) build computational thinking (CT), a vital 21st-century skill. A core design challenge is how pedagogical and gameplay challenges are integrated to balance educational objectives with player engagement. This study formalizes two contrasting challenge design patterns that reflect distinct integration strategies: extrinsic programming challenges (C1), a pedagogy-oriented design where programming is the core challenge enforced through external constraints; and intrinsic programming challenges (C2), a gameplay-oriented design where programming serves as a tool for overcoming gameplay challenges raised by in-game puzzles. To examine these challenge design patterns, we developed two isomorphic EPGs WannaBone1 (C1) and WannaBone2 (C2), each featuring 20 levels introducing sequences, loops, conditionals, and global variables. A controlled classroom study with 306 primary school students reveals that both designs improve CT, whereas C2 yields significantly higher intrinsic learning motivation and high-order immersion of flow. These findings indicate that a gameplay-oriented rather than pedagogy-oriented design perspective better unites education and entertainment, guiding future EPGs design.2026BCBaijun Chen et al.Beijing University of Posts and TelecommunicationsProgramming Education & Computational ThinkingSerious & Functional GamesCHI
Towards Understanding the Design of Mixed Reality Systems to Enrich the Beverage ExperienceDrinking is an inherently multisensory activity, yet the potential of immersive technology to dynamically shape flavor experiences remains underexplored in Human-Food Interaction (HFI) research. We introduce “XTea”, an adaptive beverage cup-based system that integrates large language models to translate natural language input into modifications of a parameterized immersive environment experienced through a headset when drinking bubble tea. Through a study with 12 bubble tea enthusiasts, we derived themes that demonstrate how “XTea” can enrich sensory engagement, support personalized and agentic experiences, and foster social qualities of drinking, pointing toward new explorations for multisensory HFI design. We also present four design strategies for multisensory beverage experiences. Ultimately, we aim to contribute to the advancement of HFI research on how multisensory interaction design can enrich flavor perception and engagement.2026YZYuchen Zheng et al.Monash UniversityMultisensory Fusion ExperienceHuman-Nature Relationships (More-than-Human Design)Generative AI (Text, Image, Music, Video)CHI
Towards Immersive Mixed Reality Street Play: Understanding Co-located Bodily Play with See-through Head-Mounted Displays in Public SpacesWe are witnessing an upcoming paradigm shift as Mixed Reality (MR) See-through Head-Mounted Displays (HMDs) become ubiquitous, with use shifting from controlled, private settings to spontaneous, public ones. While location-based pervasive mobile games like Pokémon GO have achieved success, the embodied interaction of MRHMDs is moving us from phone-based screen-touching gameplay to MRHMD-enabled co-located bodily play. Major tech companies are continuously releasing visionary videos where urban streets transform into vast MR playgrounds—imagine Harry Potter-style wizard duels on city streets. However, few researchers have conducted real-world, in-the-wild studies of such Immersive Mixed Reality Street Play (IMRSP) in public spaces in anticipation of a near future with prevalent MRHMDs. Through empirical studies on a series of research-through-design game probes called Multiplayer Omnipresent Fighting Arena (MOFA), we gain an initial understanding of this under-explored area by identifying the social implications, challenges, and opportunities of this new paradigm.2025BHBotao Amber Hu et al.Perspectives on VRCSCW
From Sports Videos to Immersive Training: Augmenting Human Motion to Enrich Basketball Training ExperienceVideo plays a crucial role in sports training, enabling participants to analyze their movements and identify opponents' weaknesses. Despite the easy access to sports videos, the rich motion data within them remains underutilized due to the lack of clear performance indicators and discrepancies from real-game conditions. To address this, we employed advanced computer vision algorithms to reconstruct human motions in an immersive environment, where users can freely observe and interact with the movements. Basketball shooting was chosen as a representative scenario to validate this framework, given its fast pace and extensive physical contact. Collaborating with experts, we iteratively designed motion-related visualizations to improve the understanding of complex movements. A one-on-one matchup simulating real games was also provided, allowing users to compete directly with the reconstructed motions. Our user studies demonstrate that this method enhances participants' movement comprehension and engagement, while insights derived from interviews inform future immersive training designs.2025YWYihong Wu et al.Full-Body Interaction & Embodied InputHuman Pose & Activity RecognitionUIST
CPVis: Evidence-based Multimodal Learning Analytics for Evaluation in Collaborative ProgrammingAs programming education becomes more widespread, many college students from non-computer science backgrounds begin learning programming. Collaborative programming emerges as an effective method for instructors to support novice students in developing coding and teamwork abilities. However, due to limited class time and attention, instructors face challenges in monitoring and evaluating the progress and performance of groups or individuals. To address this issue, we collect multimodal data from real-world settings and develop CPVis, an interactive visual analytics system designed to assess student collaboration dynamically. Specifically, CPVis enables instructors to evaluate both group and individual performance efficiently. CPVis employs a novel flower-based visual encoding to represent performance and provides time-based views to capture the evolution of collaborative behaviors. A within-subject experiment (N=22), comparing CPVis with two baseline systems, reveals that users gain more insights, find the visualization more intuitive, and report increased confidence in their assessments of collaboration.2025GZGefei Zhang et al.Zhejiang University of TechnologyInteractive Data VisualizationCollaborative Learning & Peer TeachingCHI
RouteFlow: Trajectory-Aware Animated TransitionsAnimating objects’ movements is widely used to facilitate tracking changes and observing both the global trend and local hotspots where objects converge or diverge. Existing methods, however, often obscure critical local hotspots by only considering the start and end positions of objects' trajectories. To address this gap, we propose RouteFlow, a trajectory-aware animated transition method that effectively balances the global trend and local hotspots while minimizing occlusion. RouteFlow is inspired by a real-world bus route analogy: objects are regarded as passengers traveling together, with local hotspots representing bus stops where these passengers get on and off. Based on this analogy, animation paths are generated like bus routes, with the object layout generated similarly to seat allocation according to their destinations. Compared with state-of-the-art methods, RouteFlow better facilitates identifying the global trend and locating local hotspots while performing comparably in tracking objects' movements.2025DLDuan Li et al.Tsinghua UniversityInteractive Data VisualizationVisualization Perception & CognitionCHI
VisCourt: In-Situ Guidance for Interactive Tactic Training in Mixed RealityIn team sports like basketball, understanding and executing tactics---coordinated plans of movements among players---are crucial yet complex, requiring extensive practice. These tactics require players to develop a keen sense of spatial and situational awareness. Traditional coaching methods, which mainly rely on basketball tactic boards and video instruction, often fail to bridge the gap between theoretical learning and the real-world application of tactics, due to shifts in view perspectives and a lack of direct experience with tactical scenarios. To address this challenge, we introduce VisCourt, a Mixed Reality (MR) tactic training system, in collaboration with a professional basketball team. To set up the MR training environment, we employed semi-automatic methods to simulate realistic 3D tactical scenarios and iteratively designed visual in-situ guidance. This approach enables full-body engagement in interactive training sessions on an actual basketball court and provides immediate feedback, significantly enhancing the learning experience. A user study with athletes and enthusiasts shows the effectiveness and satisfaction with VisCourt in basketball training and offers insights for the design of future SportsXR training systems.2024LCLiqi Cheng et al.Full-Body Interaction & Embodied InputMixed Reality WorkspacesImmersion & Presence ResearchUIST
Stick-To-XR: Understanding Stick-Based User Interface Design for Extended RealityThis work explores the design of stick-shaped Tangible User Interfaces (TUI) for Extended Reality (XR). While sticks are widely used in everyday objects, their applications as a TUI in XR have not been systematically studied. We conducted a participatory design session with twelve experts in XR and Human-Computer Interaction to investigate the affordances of stick-based objects and how to utilize them in XR. As a result, we presented a taxonomy of stick-based objects' affordances in terms of their functions and holding gestures. Following that, we proposed four types of stick-based XR controller forms and discussed their advantages and limitations. In the end, we juxtaposed twenty-six existing XR controllers against our proposed forms and identified Landed (Cane) Stick, Thin Stick's flexible usages, and Modular Design as the major opportunities that remain unexamined yet for stick-based XR TUI design.2024YZYaying Zhang et al.Hand Gesture RecognitionMixed Reality WorkspacesDIS
Communication, Collaboration, and Coordination in a Co-located Shared Augmented Reality Game: Perspectives From Deaf and Hard of Hearing PeopleCo-located collaborative shared augmented reality (CS-AR) environments have gained considerable research attention, mainly focusing on design, implementation, accuracy, and usability. Yet, a gap persists in our understanding regarding the accessibility and inclusivity of such environments for diverse user groups, such as deaf and Hard of Hearing (DHH) people. To investigate this domain, we used Urban Legends, a multiplayer game in a co-located CS-AR setting. We conducted a user study followed by one-on-one interviews with 17 DHH participants. Our findings revealed the usage of multimodal communication (verbal and non-verbal) before and during the game, impacting the amount of collaboration among participants and how their coordination with AR components, their surroundings, and other participants improved throughout the rounds. We utilize our data to propose design enhancements, including onscreen visuals and speech-to-text transcription, centered on participant perspectives and our analysis.2024SLSanzida Mojib Luna et al.Rochester Institute of TechnologySocial & Collaborative VRDeaf & Hard-of-Hearing Support (Captions, Sign Language, Vibration)Accessible GamingCHI
Make Interaction Situated: Designing User Acceptable Interaction for Situated Visualization in Public EnvironmentsSituated visualization blends data into the real world to fulfill individuals’ contextual information needs. However, interacting with situated visualization in public environments faces challenges posed by users’ acceptance and contextual constraints. To explore appropriate interaction design, we first conduct a formative study to identify users’ needs for data and interaction. Informed by the findings, we summarize appropriate interaction modalities with eye-based, hand-based and spatially-aware object interaction for situated visualization in public environments. Then, through an iterative design process with six users, we explore and implement interactive techniques for activating and analyzing with situated visualization. To assess the effectiveness and acceptance of these interactions, we integrate them into an AR prototype and conduct a within-subjects study in public scenarios using conventional hand-only interactions as the baseline. The results show that participants preferred our prototype over the baseline, attributing their preference to the interactions being more acceptable, flexible, and practical in public.2024QZQian Zhu et al.The Hong Kong University of Science and Technology, The Hong Kong University of Science and TechnologyAR Navigation & Context AwarenessContext-Aware ComputingField StudiesCHI
"Please Be Nice": Robot Responses to User Bullying - Measuring Performance Across Aggression LevelsAs robots become integral to public services, addressing harmful user behaviors like bullying is crucial. Existing research often overlooks the gradual nature of human bullying. This study fills this gap by exploring how robots can counter bullying through optimized responses. Using a simulated human-robot interaction study, we manipulated robot response behaviors and styles across escalating bullying severity. Results show that empathetic verbal responses promptly reduce users' bullying tendencies by eliciting remorse and redirecting attention to social awareness. However, users' underlying dispositions may override these reflexive reactions, emphasizing the need for a holistic understanding. In conclusion, a comprehensive approach is essential, involving immediate reaction optimization, emotional state assessment, and ongoing behavioral adjustment through empathetic dialogue. By implementing such strategies, we can transform human-robot relationships from potential bullying situations to harmonious interactions. This study provides an empirical foundation for response protocols that discourage bullying and enhance mutual understanding.2024YLYiming Luo et al.Xian UniversitySocial Robot InteractionCHI
Data Cubes in Hand: A Design Space of Tangible Cubes for Visualizing 3D Spatio-Temporal Data in Mixed RealityTangible interfaces in mixed reality (MR) environments allow for intuitive data interactions. Tangible cubes, with their rich interaction affordances, high maneuverability, and stable structure, are particularly well-suited for exploring multi-dimensional data types. However, the design potential of these cubes is underexplored. This study introduces a design space for tangible cubes in MR, focusing on interaction space, visualization space, sizes, and multiplicity. Using spatio-temporal data, we explored the interaction affordances of these cubes in a workshop (N=24). We identified unique interactions like rotating, tapping, and stacking, which are linked to augmented reality (AR) visualization commands. Integrating user-identified interactions, we created a design space for tangible-cube interactions and visualization. A prototype visualizing global health spending with small cubes was developed and evaluated, supporting both individual and combined cube manipulation. This research enhances our grasp of tangible interaction in MR, offering insights for future design and application in diverse data contexts.2024SHShuqi He et al.Xi'an Jiaotong - Liverpool UniversityMixed Reality WorkspacesInteractive Data VisualizationTime-Series & Network Graph VisualizationCHI
Exploration of Foot-based Text Entry Techniques for Virtual Reality EnvironmentsFoot-based input can serve as a supplementary or alternative approach to text entry in virtual reality (VR). This work explores the feasibility and design of foot-based techniques that are hands-free. We first conducted a preliminary study to assess foot-based text entry in standing and seated positions with tap and swipe input approaches. The findings showed that foot-based text input was feasible, with the possibility for performance and usability improvements. We then developed three foot-based techniques, including two tap-based techniques (FeetSymTap and FeetAsymTap) and one swipe-based technique (FeetGestureTap), and evaluated their performance via another user study. The results show that the two tap-based techniques supported entry rates of 11.12 WPM and 10.80 WPM, while the swipe-based technique led to 9.16 WPM. Our findings provide a solid foundation for the future design and implementation of foot-based text entry in VR and have the potential to be extended to MR and AR.2024TWTingjie Wan et al.Xi'an Jiaotong-Liverpool UniversityFoot & Wrist InteractionSocial & Collaborative VRImmersion & Presence ResearchCHI
AR-Enhanced Workouts: Exploring Visual Cues for At-Home Workout Videos in AR EnvironmentIn recent years, with growing health consciousness, at-home workout has become increasingly popular for its convenience and safety. Most people choose to follow video guidance during exercising. However, our preliminary study revealed that fitness-minded people face challenges when watching exercise videos on handheld devices or fixed monitors, such as limited movement comprehension due to static camera angles and insufficient feedback. To address these issues, we reviewed popular workout videos, identified user requirements, and came up with an augmented reality (AR) solution. Following a user-centered iterative design process, we proposed a design space of AR visual cues for workouts and implemented an AR-based application. Specifically, we captured users’ exercise performance with pose-tracking technology and provided feedback via AR visual cues. Two user experiments showed that incorporating AR visual cues could improve movement comprehension and enable users to adjust their movements based on real-time feedback. Finally, we presented several suggestions to inspire future design and apply AR visual cues to sports training.2023YWYihong Wu et al.Fitness Tracking & Physical Activity MonitoringContext-Aware ComputingUIST
Predicting Gaze-based Target Selection in Augmented Reality Headsets based on Eye and Head Endpoint DistributionsTarget selection is a fundamental task in interactive Augmented Reality (AR) systems. Predicting the intended target of selection in such systems can provide users with a smooth, low-friction interaction experience. Our work aims to predict gaze-based target selection in AR headsets with eye and head endpoint distributions, which describe the probability distribution of eye and head 3D orientation when a user triggers a selection input. We first conducted a user study to collect users’ eye and head behavior in a gaze-based pointing selection task with two confirmation mechanisms (air tap and blinking). Based on the study results, we then built two models: a unimodal model using only eye endpoints and a multimodal model using both eye and head endpoints. Results from a second user study showed that the pointing accuracy is improved by approximately 32% after integrating our models into gaze-based selection techniques.2023YWYushi Wei et al.Xi'an Jiaotong-Liverpool UniversityEye Tracking & Gaze InteractionAR Navigation & Context AwarenessCHI
Tasks of a Different Color: How Crowdsourcing Practices Differ per Complex Task Type and Why This MattersCrowdsourcing in China is a thriving industry. Among its most interesting structures, we find crowdfarms, in which crowdworkers self-organize as small organizations to tackle macrotasks. Little, however, is known as to which practices these crowdfarms use to tackle the macrotasks, and this goes hand in hand with the current practice of the HCI research community to treat all forms of complex crowdsourcing work as practically the same. However, macrotasks differ substantially regarding structure and decomposability. Treating them under one umbrella term - macrotasking - can lead to an imprecise understanding of the workforce involved. We address this gap by examining the work practices of 31 Chinese crowdfarms on the four main macrotask types, namely: modular, interlaced, wicked, and container macrotasks. Our results confirm essential differences in how these nascent crowd organizations address different macrotasks and shed light on what platforms can do to improve the uptake of such work.2023YWYihong Wang et al.Xi'an Jiaotong-Liverpool UniversityMental Health Apps & Online Support CommunitiesCrowdsourcing Task Design & Quality ControlDeveloping Countries & HCI for Development (HCI4D)CHI
DreamVR: Curating an Interactive Exhibition in Social VR Through an Autobiographical Design StudyVirtual exhibitions have long been regarded as an extension of information delivery for physical exhibitions. However, what virtual exhibitions can offer audiences as a novel experience independently from physical exhibitions has been largely unexplored. In this study, we aim to understand the promises and challenges of experiencing and curating exhibitions in VR by interviewing nine expert curators. Drawing from expert insights, we summarized a set of design guidelines to inform what we can learn and adapt from physical exhibitions when curating in VR. Then, using an autobiographical design approach, we curated an interactive exhibition in VRChat to explore novel interaction techniques. We also hosted an open tour guide in the user study to validate our design guidelines with thirty participants. Results show that our approach of curating an exhibition in VRChat provided the participants with engaging and novel experiences interacting with the exhibits and other audiences.2023JCJiaxun Cao et al.Duke Kunshan University, Duke Kunshan UniversitySocial & Collaborative VRInteractive Narrative & Immersive StorytellingCHI