Walking Through or Detour? Investigating Walking Paths with World‑Anchored Mixed Reality ObjectsIn mixed reality environments, virtual objects can obscure real-world obstacles, creating a risk of collision when users walk through them. Although users may choose to detour around virtual objects, this behavior also carries risks, such as colliding with obstacles or pedestrians along the detour route. To reduce collision risks, it is essential to understand the factors that determine whether users walk through or detour, as well as the walking paths associated with each behavior. In this research, we investigated users’ walking behavior toward both a static virtual obstacle and a virtual obstacle that disappeared as the user approached. Our findings suggest that individual characteristics and the width of the virtual obstacle influence the decision to walk through or detour. Furthermore, while most users initially chose paths that detoured around the virtual obstacle, once the obstacle began to disappear, they switched their walking paths toward the space where it had been.2026MCMyungguen Choi et al.University of TsukubaImmersion & Presence ResearchAR Navigation & Context AwarenessPedestrian & Cyclist SafetyCHI
User Perceptions of Responsible Gambling Messages as Nudges for Gambling SafetyNudges are subtle interventions designed to influence user behavior without restricting choice. Responsible gambling messages (RGMs) exemplify such nudges by encouraging safer decision-making in gambling environments. Prior research has examined how pop-up messages influence gambling behavior in experimental settings and has explored the design of effective slogan messages. However, little is known about how different types of RGMs shape users’ real-world gambling behavior and safety. To address this gap, we apply a nudging perspective to examine how RGMs support gambling safety throughout gamblers’ decision-making journey. We conducted semi-structured interviews with 22 gamblers and found that participants were generally aware of RGMs, yet some misunderstood their intended purpose. Participants perceived the safety impact of RGMs as reflected in both attitudinal and behavioral dimensions. We further discuss users’ message reception practices and the effectiveness of RGMs as nudges, and conclude with design implications for promoting gambling safety.2026MGMaggie Yongqi Guan et al.University of MacauBehavior Change & Reflection TechnologyParticipatory DesignUser Research Methods (Interviews, Surveys, Observation)CHI
Eyes on the Finger: Investigating a Ring-Shaped Camera for Seamless Accessible Tactile ExplorationTactile exploration is essential for blind and low vision (BLV) individuals to understand objects and spaces. Yet little is known about how camera-based devices can support hand-centric exploration: tactilely examining exhibits while inquiring about and processing information. We investigate a finger-worn ring camera that captures images from the palm side while allowing tactile exploration, comparing it with hand-centered smartphones. We conducted a Wizard-of-Oz study with 11 BLV participants in a science museum. Results showed that the ring camera supported effective bimanual strategies: exploring with both hands, lifting the camera-worn hand while keeping the other as an anchor during inquiry, and resuming bimanual touch for information processing. In contrast, smartphones led to effortful, fragmented exploration. Building on these findings, we developed an interactive system and evaluated its reliability and practicality with 6 BLV participants. We contribute insights and design implications for wearable camera systems that augment tactile exploration in real-world settings.2026ATAyaka Tsutsui et al.University of TsukubaVisual Impairment Technologies (Screen Readers, Tactile Graphics, Braille)Haptic WearablesSmartwatches & Fitness BandsCHI
Bevel or Not: Identifying the Potential of Bevels for Touch Input Accuracy on Ring DeviceSmart rings can serve as a wearable platform for off-device control of nearby devices. However, their thin bands lead to a small touch surface on the ring, limiting touch input expressivity and accuracy. To address this, we investigate the effectiveness of ring shapes that use beveled surfaces in addition to the flat outer surface as input surfaces. The distinct angles of the three surfaces limit touches to the intended surfaces, improving accuracy. A first study showed that with band widths of 6 mm or less, flat rings achieved low accuracy (63.8%) in distinguishing between left and right edge touches, whereas beveled and rounded rings achieved high accuracy (90.0%, 93.3%). In a second study of nine touch gestures on 6-mm rings, beveled rings outperformed the flat rings, achieving 92.9% (sighted) and 91.8% (eyes-free) with FPR ≤ 1%. Through these investigations, we identify beveled surfaces as a promising method toward expressive, precise touch input.2026YKYuki Kubo et al.NTT, Inc.Haptic WearablesOn-Skin Display & On-Skin InputContext-Aware ComputingCHI
BRIDGE: Borderless Reconfiguration for Inclusive and Diverse Gameplay Experience via Embodiment TransformationTraining resources for parasports are limited, reducing opportunities for athletes and coaches to engage with sport-specific movements and tactical coordination. To address this gap, we developed BRIDGE, a system that integrates a reconstruction pipeline, which detects and tracks players from broadcast video to generate 3D play sequences, with an embodiment-aware visualization framework that decomposes head, trunk, and wheelchair base orientations to represent attention, intent, and mobility. We evaluated BRIDGE in two controlled studies with 20 participants (10 national wheelchair basketball team players and 10 amateur players). The results showed that BRIDGE significantly enhanced the perceived naturalness of player postures and made tactical intentions easier to understand. In addition, it supported functional classification by realistically conveying players’ capabilities, which in turn improved participants’ sense of self-efficacy. This work advances inclusive sports learning and accessible coaching practices, contributing to more equitable access to tactical resources in parasports.2026HSHayato Saiki et al.University of TsukubaSerious & Functional GamesGame AccessibilityHuman Pose & Activity RecognitionCHI
ViSTAR: Virtual Skill Training with Augmented Reality with 3D Avatars and LLM coaching agentWe present ViSTAR, a Virtual Skill Training system in AR that supports self-guided basketball skill practice, with feedback on balance, posture, and timing. From a formative study with basketball players and coaches, the system addresses three challenges: understanding skills, identifying errors, and correcting mistakes. ViSTAR follows the Behavioral Skills Training (BST) framework—instruction, modeling, rehearsal, and feedback. It provides feedback through visual overlays, rhythm and timing cues, and an AI-powered coaching agent using 3D motion reconstruction. We generate verbal feedback by analyzing spatio-temporal joint data and mapping features to natural-language coaching cues via a Large Language Model (LLM). A key novelty is this feedback generation: motion features become concise coaching insights. In two studies (N=16), participants generally preferred our AI-generated feedback to coach feedback and reported that ViSTAR helped them notice posture and balance issues and refine movements beyond self-observation.2026CLChunggi Lee et al.Harvard UniversityFull-Body Interaction & Embodied InputEye Tracking & Gaze InteractionBrain-Computer Interface (BCI) & NeurofeedbackCHI
Hitchhiking Hands: Enabling “Virtually Direct” VR Manipulation by Switching Multiple Hand Avatars with GazeDexterous freehand manipulation in virtual reality offers rich interaction but is limited by physical reach. Existing indirect remote manipulation techniques often sacrifice this dexterity. We address this by defining "virtually direct" manipulation, a conceptual framework for techniques that break from a purely direct or indirect model by decoupling the virtual hand from the physical one. Within this framework, we present Hitchhiking Hands (HH), a novel implementation designed to preserve the rich dexterity of direct manipulation at a distance. HH allows users to instantly switch control between multiple pre-defined virtual hands using gaze. We evaluated HH in two user studies. A first study showed our approach, which consistently maintains direct-touch properties, surpasses an established baseline in 6DoF manipulation performance and embodiment. A second qualitative study revealed HH excels in structured spaces but is ill-suited for unstructured global tasks, highlighting a trade-off between flexibility and learnability.2026RBReigo Ban et al.The University of TokyoIdentity & Avatars in XRFull-Body Interaction & Embodied InputEye Tracking & Gaze InteractionCHI
MapStory: Prototyping Editable Map Animations with LLM AgentsWe introduce MapStory, an LLM‑powered animation prototyping tool that generates editable map animation sequences directly from natural language text by leveraging a dual-agent LLM architecture. Given a user-written script, MapStory automatically produces a scene breakdown, which decomposes the text into key map animation primitives such as camera movements, visual highlights, and animated elements. Our system includes a researcher agent that accurately queries geospatial information by leveraging an LLM with web search, enabling automatic extraction of relevant regions, paths, and coordinates while allowing users to edit and query for changes or additional information to refine the results. Additionally, users can fine-tune parameters of these primitive blocks through an interactive timeline editor. We detail the system’s design and architecture, informed by formative interviews with professional animators and by an analysis of 200 existing map animation videos. Our evaluation, which includes expert interviews (N=5), and a usability study (N=12), demonstrates that MapStory enables users to create map animations with ease, facilitates faster iteration, encourages creative exploration, and lowers barriers to creating map-centric stories.2025AGAditya Gunturu et al.Geospatial & Map VisualizationComputational Methods in HCIUIST
Vestibular Stimulation Enhances Hand RedirectionWe demonstrate how the vestibular system (i.e., the sense of balance) influences the perception of hand position in VR. By exploiting this via galvanic vestibular stimulation (GVS), we can enhance the degree to which we can redirect the user’s hands in VR without them noticing, i.e., raising the detection threshold of hand redirection. Our novel cross-modal illusion relies on the principle that a GVS-induced subtle body sway aligns with the user’s expected body balance during hand redirection. This alignment reduces the sensory conflict between the expected and actual body balance, allowing for a larger hand redirection than would normally be noticed. In our user study, we validated that our approach raises the detection threshold of VR hand redirection by approximately 55% for outward and 45% for inward movements. With this increase, our approach broadens the applicability of hand redirection (e.g., compressing a VR space into an even smaller physical area).2025KKKensuke Katori et al.Force Feedback & Pseudo-Haptic WeightShape-Changing Interfaces & Soft Robotic MaterialsFull-Body Interaction & Embodied InputUIST
Video2MR: Automatically Generating Mixed Reality 3D Instructions by Augmenting Extracted Motion from 2D VideosThis paper introduces Video2MR, a mixed reality system that automatically generates 3D sports and exercise instructions from 2D videos. Mixed reality instructions have great potential for physical training, but existing works require substantial time and cost to create these 3D experiences. Video2MR overcomes this limitation by transforming arbitrary instructional videos available online into MR 3D avatars with AI-enabled motion capture (DeepMotion). Then, it automatically enhances the avatar motion through the following augmentation techniques: 1) contrasting and highlighting differences between the user and avatar postures, 2) visualizing key trajectories and movements of specific body parts, 3) manipulation of time and speed using body motion, and 4) spatially repositioning avatars for different perspectives. Developed on Hololens 2 and Azure Kinect, we showcase various use cases, including yoga, dancing, soccer, tennis, and other physical exercises. The study results confirm that Video2MR provides more engaging and playful learning experiences, compared to existing 2D video instructions.2025KIKeiichi Ihara et al.Full-Body Interaction & Embodied InputMixed Reality WorkspacesBiosensors & Physiological MonitoringIUI
Understanding Usability of VR Pointing Methods with a Handheld-style HMD for Onsite ExhibitionsHandheld-style head-mounted displays (HMDs) are becoming increasingly popular as a convenient option for onsite exhibitions. However, they lack established practices for basic interactions, particularly pointing methods. Through our formative study involving practitioners, we discovered that controllers and hand gestures are the primary pointing methods being utilized. Building upon these findings, we conducted a usability study to explore seven different pointing methods, incorporating insights from the formative study and current virtual reality (VR) practices. The results showed that while controllers remain a viable option, hand gestures are not recommended. Notably, dwell time-based methods, which are not fast and are not commonly recognized by practitioners, demonstrate high usability and user confidence, particularly for inexperienced VR users. We recommend the use of dwell-based methods for onsite exhibition contexts. This research provides insights for the adoption of handheld-style HMDs, laying the groundwork for improving user interaction in exhibition environments, thereby potentially enhancing visitor experiences.2025YAYuki Abe et al.Hokkaido University, Human-Computer Interaction LabEye Tracking & Gaze InteractionSocial & Collaborative VRImmersion & Presence ResearchCHI
Exploring the Design of LLM-based Agent in Enhancing Self-disclosure Among the Older AdultsSocial difficulties have become an increasingly serious issue among older adults. For older adults, regular self-disclosure is essential for maintaining mental health and building close relationships. Leveraging conversational agents to encourage self-disclosure in older adults has shown increasing potential. Understanding how LLM-based agents can influence and stimulate self-disclosure across different topics is crucial for designing future agents tailored to older users. This study introduces Disclosure-Agent, an LLM-based conversational agent, and examines its impact on self-disclosure in older adults through a user study involving 20 participants, 8 topics, and two interactive interfaces equipped with Disclosure-Agent. The findings provide valuable insights into how LLM-based agents can promote self-disclosure in older adults and offer design recommendations for future elderly-oriented conversational agents.2025YGYijie Guo et al.Tsinghua University, Academy of Arts and Design; Tsinghua University, The Future LaboratoryAgent Personality & AnthropomorphismHuman-LLM CollaborationCHI
FlexEar-Tips: Shape-Adjustable Ear Tips Using Pressure ControlWe introduce FlexEar-Tips, a dynamic ear tip system designed for the next-generation hearables. The ear tips are controlled by an air pump and solenoid valves, enabling size adjustments for comfort and functionality. FlexEar-Tips includes an air pressure sensor to monitor ear tip size, allowing it to adapt to environmental conditions and user needs. In the evaluation, we conducted a preliminary investigation of the size control accuracy and the minimum amount of variability of haptic perception in the user's ear. We then evaluated the user's ability to identify patterns in the haptic notification system, the impact on the music listening experience, the relationship between the size of the ear tips and the sound localization ability, and the impact on the reduction of humidity in the ear using a model. We proposed new interaction modalities for adaptive hearables and discussed health monitoring, immersive auditory experiences, haptics notifications, biofeedback, and sensing.2025TATakashi Amesaka et al.Keio University, Lifestyle Computing LabHaptic WearablesShape-Changing Interfaces & Soft Robotic MaterialsCHI
PrISM-Observer: Intervention Agent to Help Users Perform Everyday Procedures Sensed using a SmartwatchWe routinely perform procedures (such as cooking) that include a set of atomic steps. Often, inadvertent omission or misordering of a single step can lead to serious consequences, especially for those experiencing cognitive challenges such as dementia. This paper introduces PrISM-Observer, a smartwatch-based, context-aware, real-time intervention system designed to support daily tasks by preventing errors. Unlike traditional systems that require users to seek out information, the agent observes user actions and intervenes proactively. This capability is enabled by the agent's ability to continuously update its belief in the user's behavior in real-time through multimodal sensing and forecast optimal intervention moments and methods. We first validated the steps-tracking performance of our framework through evaluations across three datasets with different complexities. Then, we implemented a real-time agent system using a smartwatch and conducted a user study in a cooking task scenario. The system generated helpful interventions, and we gained positive feedback from the participants. The general applicability of PrISM-Observer to daily tasks promises broad applications, for instance, including support for users requiring more involved interventions, such as people with dementia or post-surgical patients.2024RARiku Arakawa et al.Fitness Tracking & Physical Activity MonitoringElderly Care & Dementia SupportContext-Aware ComputingUIST
EarHover: Mid-Air Gesture Recognition for Hearables Using Sound Leakage SignalsWe introduce EarHover, an innovative system that enables mid-air gesture input for hearables. Mid-air gesture input, which eliminates the need to touch the device and thus helps to keep hands and the device clean, has been known to have high demand based on previous surveys. However, existing mid-air gesture input methods for hearables have been limited to adding cameras or infrared sensors. By focusing on the sound leakage phenomenon unique to hearables, we have realized mid-air gesture recognition using a speaker and an external microphone that are highly compatible with hearables. The signal leaked to the outside of the device due to sound leakage can be measured by an external microphone, which detects the differences in reflection characteristics caused by the hand's speed and shape during mid-air gestures. Among 27 types of gestures, we determined the seven most suitable gestures for EarHover in terms of signal discrimination and user acceptability. We then evaluated the gesture detection and classification performance of two prototype devices (in-ear type/open-ear type) for real-world application scenarios.2024SSShunta Suzuki et al.In-Vehicle Haptic, Audio & Multimodal FeedbackHand Gesture RecognitionUIST
Coaching Copilot: Blended Form of an LLM-Powered Chatbot and a Human Coach to Effectively Support Self-Reflection for Leadership GrowthChatbots' role in fostering self-reflection is now widely recognized, especially in inducing users' behavior change. While the benefits of 24/7 availability, scalability, and consistent responses have been demonstrated in contexts such as healthcare and tutoring to help one form a new habit, their utilization in coaching necessitating deeper introspective dialogue to induce leadership growth remains unexplored. This paper explores the potential of such a chatbot powered by recent Large Language Models (LLMs) in collaboration with professional coaches in the field of executive coaching. Through a design workshop with them and two weeks of user study involving ten coach-client pairs, we explored the feasibility and nuances of integrating chatbots to complement human coaches. Our findings highlight the benefits of chatbots' ubiquity and reasoning capabilities enabled by LLMs while identifying their limitations and design necessities for effective collaboration between human coaches and chatbots. By doing so, this work contributes to the foundation for augmenting one's self-reflective process with prevalent conversational agents through the human-in-the-loop approach.2024RARiku Arakawa et al.Conversational ChatbotsHuman-LLM CollaborationCUI
HIFU Embossment of Acrylic SheetsTactile interfaces such as embossment facilitate information transfer through touch in Human-Computer Interaction (HCI). Traditional embossing methods, while enabling the creation of intricate patterns, face limitations due to mold reliance and material thickness restrictions, hindering bespoke embossment creation. In this study, we propose High-Intensity Focused Ultrasound (HIFU) as an alternative technique to produce tailored embossed designs on acrylic without the need for traditional molds. We uncover specific HIFU parameters, such as amplitude, irradiation time, and distance that directly impact essential qualities of embossment including embossment height, transparency, and line generation. Additionally, the capability of embossing without the use of molds expands the applications for quick prototyping and customization of embossed designs within HCI. Furthermore, we introduce a user interface developed to streamline the design and application of customizable tactile graphics using HIFU, aimed at non-expert users. Preliminary user studies reveal positive feedback on the interface’s intuitiveness and the quality of the HIFU embossment. Our study indicates that HIFU embossment presents a viable approach for creating embossed features in interactive systems, with the potential to offer methods for personal customization in the design of tactile materials.2024ATAyaka Tsutsui et al.University of TsukubaMid-Air Haptics (Ultrasonic)Shape-Changing Interfaces & Soft Robotic MaterialsCHI
HoloBots: Augmenting Holographic Telepresence with Mobile Robots for Tangible Remote Collaboration in Mixed RealityThis paper introduces HoloBots, a mixed reality remote collaboration system that augments holographic telepresence with synchronized mobile robots. Beyond existing mixed reality telepresence, HoloBots lets remote users not only be visually and spatially present, but also \textit{physically} engage with local users and their environment. HoloBots allows the users to touch, grasp, manipulate, and interact with the remote physical environment as if they were co-located in the same shared space. We achieve this by synchronizing holographic user motion (Hololens 2 and Azure Kinect) with tabletop mobile robots (Sony Toio). Beyond the existing physical telepresence, HoloBots contributes to an exploration of broader design space, such as object actuation, virtual hand physicalization, world-in-miniature exploration, shared tangible interfaces, embodied guidance, and haptic communication. We evaluate our system with twelve participants by comparing it with hologram-only and robot-only conditions. Both quantitative and qualitative results confirm that our system significantly enhances the level of co-presence and shared experience, compared to the other conditions.2023KIKeiichi Ihara et al.Teleoperated DrivingMixed Reality WorkspacesTeleoperation & TelepresenceUIST
User Authentication Method for Hearables Using Sound Leakage SignalsWe propose a novel biometric authentication method that leverages sound leakage signals from hearables that are captured by an external microphone. A sweep signal is played from hearables, and sound leakage is recorded using an external microphone. This sound leakage signal represents the acoustic characteristics of the ear canal, auricle, or hand. Then, our system analyzes the echoes and authenticates the user. The proposed method is highly adaptable to hearables because it leverages widely available sensors, such as speakers and external microphones. In addition, the proposed method has the potential to be used in combination with existing methods. In this study, we investigate the characteristics of sound leakage signals using an experimental model and measure the authentication performance of our method using acoustic data from 16 people. The results show that the balanced accuracy (BAC) scores were in the range of 87.0%-96.7% in several scenarios.2023TATakashi Amesaka et al.Passwords & AuthenticationUbiComp
CatAlyst: Domain-Extensible Intervention for Preventing Task Procrastination Using Large Generative ModelsCatAlyst uses generative models to help workers’ progress by influencing their task engagement instead of directly contributing to their task outputs. It prompts distracted workers to resume their tasks by generating a continuation of their work and presenting it as an intervention that is more context-aware than conventional (predetermined) feedback. The prompt can function by drawing their interest and lowering the hurdle for resumption even when the generated continuation is insufficient to substitute their work, while recent human-AI collaboration research aiming at work substitution depends on a stable high accuracy. This frees CatAlyst from domain-specific model-tuning and makes it applicable to various tasks. Our studies involving writing and slide-editing tasks demonstrated CatAlyst’s effectiveness in helping workers swiftly resume tasks with a lowered cognitive load. The results suggest a new form of human-AI collaboration where large generative models publicly available but imperfect for each individual domain can contribute to workers’ digital well-being.2023RARiku Arakawa et al.Carnegie Mellon UniversityHuman-LLM CollaborationNotification & Interruption ManagementWorkplace Wellbeing & Work StressCHI