From Prompt to Presence: Co-Creating Personalised Emotional Sanctuaries in VR with Generative AIThe emergence of generative artificial intelligence (GenAI), combined with immersive virtual reality (VR), enables the rapid creation of personalised virtual content from simple text prompts, holding potential for emotional support. However, most current VR systems rely on pre-authored content and limit user agency in designing emotionally meaningful experiences. We introduce OasisMind, an AI-assisted VR system that empowers users to co-create 360° environments, corresponding ambient soundscapes, and context-aware digital companions through natural language prompts. In a user study (N=24), we observed how participants constructed virtual worlds for emotionally meaningful use cases and compared their creations to validated, pre-defined VR scenes recommended by previous research. Our results indicate a subjective preference for self-created environments, while no significant differences were observed in perceived satisfaction or presence between conditions. These findings suggest that user agency contributes to the emotional resonance of virtual experiences and inform the design of future personalised companion systems.2026RWRuoyu Wen et al.University of CanterburyGenerative AI (Text, Image, Music, Video)Immersion & Presence ResearchSocial & Collaborative VRIUI
Understanding How Mobile Interactions Shape Grasp and Contact Patterns Beyond the TouchscreenThe way users hold a smartphone depends on the interaction task, yet little is known about the fingers' engagement with the device's surfaces beyond the touchscreen. Such an understanding not only opens up opportunities for novel on- and off-screen interactions, but also the device’s possible physical affordances. We present a study (N=23) that examines the hands' physical engagement with the smartphone beyond the touchscreen across nine mobile interactions. Grasps were annotated from photographs, and contact regions were captured using residual heat traces from grasping the device. Our findings show that fingers and palms adopt a variety of support roles and postures when engaging with the smartphone's back and side edges. The hand-contact maps reveal distinct patterns, differing in contact frequency and placement. This work contributes an empirical characterisation of hands' back and edge engagement, highlighting design opportunities for future smartphone usage extending beyond the touchscreen.2026CSCarolin Stellmacher et al.University of BremenOne-Handed Operation & Mobile GesturesTouch Target Selection & PointingCHI
How are Vibrotactile Experiences Visually Represented? A Taxonomy of Illustration CharacteristicsVibrotactile experiences (VTX) consist of a multitude of design parameters and experiential dimensions that can be challenging to communicate visually. To understand how this is commonly done in scientific communication, we systematically reviewed VTX illustrations in academic publications. Using inductive and deductive methods, we built a taxonomy detailing characteristics of VTX illustrations that focuses on what is illustrated and how it is depicted. Using the taxonomy, we coded a total of 768 figures spanning 409 publications. These results indicate that (1) half of the illustrations communicate on the timing of vibrotactile feedback with regards to users’ actions, (2) illustrations depict stimuli rather than experiences and infrequently communicate multimodal aspects of the experiences, and (3) contextual information of vibrotactile displays and experiential aspects are often distributed across several complementary figures. We conclude by discussing the benefits and limitations of this taxonomy to support the design process.2026BFBruno Fruchard et al.Univ. Lille, Inria, CNRS, Centrale LilleVibrotactile Feedback & Skin StimulationCHI
Determining Perception Thresholds for Real and Virtual Inclinations While Cycling in Virtual RealityIn virtual reality (VR) experiences, mismatches between reality and virtuality are usually undesirable, as they can disrupt immersion and induce cybersickness. However, when carefully controlled, they may expand the design space of VR. This research investigates perceptual detection thresholds for mismatches between real and virtual inclinations during cycling in VR. Using a custom simulation, N=30 participants cycled through a virtual city while physical and visual inclinations were independently manipulated. Real inclinations were implemented with a tilting indoor bike, providing vestibular and proprioceptive feedback, while virtual inclinations within the simulated environment were presented visually. A multiple staircase procedure derived estimates for perceptual thresholds that approximate which mismatches in visual and physical inclination were still perceived as congruent. These thresholds reveal a window of perceived congruence before mismatches become noticeable to users. These findings advance understanding of sensory integration in VR cycling and inform applications in immersive training, exergames, and rehabilitation systems.2026JKJonas Keppel et al.University of Duisburg-EssenImmersion & Presence ResearchFitness Tracking & Physical Activity MonitoringMotion Sickness & Passenger ExperienceCHI
AI of Oz: Enhancing Wizard of Oz Studies in HCI with AI Assistance for Human ModerationThe Wizard of Oz (WoZ) method is a common and popular approach for simulating interactive systems in Human-Computer Interaction. Running such studies is demanding for researchers because the human wizard must manage human–agent interactions in real time while keeping participants safe and the interaction natural. Many WoZ systems struggle to reproduce complex agent behaviours without minimal delays or heavy workload for the moderator. We introduce AI of Oz, a framework that uses large language models to support researchers by monitoring ongoing interactions, detecting sensitive moments, and suggesting contextually appropriate responses. In a study with 20 HCI-related researchers, the system improved participants’ ability to manage interactions and maintain control compared to a version without AI support. We outline implications for WoZ research and note current limitations and future directions.2026RWRuoyu Wen et al.University of CanterburyHuman-LLM CollaborationUser Research Methods (Interviews, Surveys, Observation)Prototyping & User TestingCHI
Building Care That Fits Its People: Insights from Social-Media–Enabled Community-Based Rehabilitation in ThailandDigital health interventions in the Global South often rely on transmission models, assuming that delivering correct medical information yields better care. We challenge this view through an analysis of a multi-platform social media intervention for Community-Based stroke Rehabilitation(CBR) in rural Thailand. Following a collaborative development process with clinicians and a deployment across roughly 2,000 villages, we interviewed 28 caregivers, patients, and health volunteers. We found that communities appropriated the technology in unexpected ways, such as using videos as social objects to manage family hierarchies, integrating rehabilitation into Buddhist merit-making, and prioritising offline peer networks over online discussion. Our findings suggest that effective Human-Computer-Interaction (HCI) for digital health in Low- and Middle-Income Countries (LMICs) should look beyond engagement metrics to support the appropriation of digital tools, enabling communities to integrate clinical protocols into their existing cultural and relational fabrics.2026ANAcarima Nanthanasit et al.University of KentDeveloping Countries & HCI for Development (HCI4D)Mental Health Apps & Online Support CommunitiesTelemedicine & Remote Patient MonitoringCHI
User-reconfigured Haptics: Combining User-Reconfiguration and Visual Manipulations to Enhance Dynamic Passive Haptic Experiences for VRVirtual Reality (VR) depends on haptic feedback to create immersive experiences. Traditional passive proxies align physical props with their virtual counterparts but remain limited in scalability and expressiveness, or require bulky actuators to support reconfiguration. We introduce User-reconfigured Haptics, an approach that utilizes implicit user actions to reconfigure haptic interfaces to extend the gamut of VR haptic experiences. Modular 3D-printed cells are assembled into dynamic interfaces that express diverse haptic properties such as softness and weight. By masking physical reconfigurations with visual (re)mapping, user actions unnoticeably change haptic properties, resulting in user-driven, dynamic haptic experiences. User studies show that our design can provide distinguishable haptic experiences and is perceived as realistic and enjoyable in a VR task. We further showcase four applications: a fishing rod that changes weight and flexibility, a dynamic desktop of pressable buttons, a glove with adjustable squeezing, and a crossbow with variable pulling resistance.2026XWXinrong Wang et al.Saarland Informatics Campus (DFKI)Haptic WearablesImmersion & Presence ResearchShape-Changing Interfaces & Soft Robotic MaterialsCHI
Exploring Opportunities for Flexible Wearables to Support Physical TrainingAdvances in digital fabrication with flexible and conductive materials are enabling new opportunities for customisable wearables that enhance physical training across various sports through on-body posture and movement monitoring. However, before developing new wearable devices, it is crucial to understand the unique challenges and opportunities of real-world use and the diverse needs of stakeholders such as athletes, trainers, and sports enthusiasts. In this paper, we report results of two co-design workshops conducted with 11 participants representing a range of sports backgrounds and interests to identify challenges and opportunities for flexible wearables. The participants' designs and prototypes highlight the importance of supporting personalised wearable design, addressing varying contexts, and accommodating different skill levels. We propose design considerations for wearable devices that prioritise flexibility, both in their materiality to support a wide range of movement and in their adaptability to accommodate changing conditions and user progression across skill levels.2025KSKatarzyna Stawarz et al.Haptic WearablesFitness Tracking & Physical Activity MonitoringDIS
TactStyle: Generating Tactile Textures with Generative AI for Digital FabricationRecent work in Generative AI enables the stylization of 3D models based on image prompts. However, these methods do not incorporate tactile information, leading to designs that lack the expected tactile properties. We present TactStyle, a system that allows creators to stylize 3D models with images while incorporating the expected tactile properties. TactStyle accomplishes this using a modified image-generation model fine-tuned to generate heightfields for given surface textures. By optimizing 3D model surfaces to embody a generated texture, TactStyle creates models that match the desired style and replicate the tactile experience. We utilize a large-scale dataset of textures to train our texture generation model. In a psychophysical experiment, we evaluate the tactile qualities of a set of 3D-printed original textures and TactStyle's generated textures. Our results show that TactStyle successfully generates a wide range of tactile features from a single image input, enabling a novel approach to haptic design.2025FFFaraz Faruqi et al.MIT CSAILForce Feedback & Pseudo-Haptic WeightShape-Changing Interfaces & Soft Robotic MaterialsGenerative AI (Text, Image, Music, Video)CHI
Can AI Prompt Humans? Multimodal Agents Prompt Players’ Game Actions and Show Consequences to Raise Sustainability AwarenessUnsustainable behaviors are challenging to prevent due to their long-term, often unclear consequences. Serious games offer a promising solution by creating artificial environments where players can immediately experience the outcomes of their actions. To explore this potential, we developed EcoEcho, a GenAI-powered game leveraging multimodal agents to raise sustainability awareness. These agents engage players in natural conversations, prompting them to take in-game actions that lead to visible environmental impacts. We evaluated EcoEcho using a mixed-methods approach with 23 participants. Results show a significant increase in intended sustainable behaviors post-game, although attitudes towards sustainability had only marginal effects, suggesting that in-game actions likely can motivate intended real world behaviors despite similar opinions on sustainability. This finding highlights multimodal agents and action-consequence mechanics to effectively raising sustainability awareness and the potential of motivating real-world behavioral change.2025QZQinshi Zhang et al.University of California, San Diego, University of California, San DiegoGenerative AI (Text, Image, Music, Video)Serious & Functional GamesSustainable HCICHI
Understanding and Improving the Performance of Action PointingAction pointing involves choosing and executing an action at a specific place in the workspace (e.g., choosing a tool and clicking to start drawing, or selecting an object and copying with a shortcut). The elements of action pointing (choosing an action, specifying a position, and triggering the action) can be carried out in many ways - and our analysis of current techniques identified limitations on performance, particularly for repeated sequences of interactions. To empirically analyse interaction alternatives for action pointing, we developed and evaluated two techniques: ModeKeys removes modifier keys from keyboard shortcuts used to choose actions; AimKeys goes further by using the shortcut (not the mouse) to trigger the action. Three studies over three tasks showed that these reconfigurations were highly effective - in all studies, either AimKeys or ModeKeys were faster, easier, and preferred overall. Our studies show that small variations in the configuration of action pointing can have a large impact, offering opportunities to improve performance with direct-manipulation systems.2025CBCameron Beattie et al.University of SaskatchewanFull-Body Interaction & Embodied InputKnowledge Worker Tools & WorkflowsCHI
CollabJam: Studying Collaborative Haptic Experience Design for On-Body Vibrotactile PatternsDesigning vibrotactile experiences collaboratively requires communicating using multiple senses. This is challenging in remote scenarios as designers need to effectively express and communicate their intention while iteratively building and refining experiences, ideally in real-time. We formulate design considerations for collaborative haptic design tools, and propose CollabJam, a collaborative prototyping suite enabling remote synchronous design of vibrotactile experiences for on-body applications. We first outline CollabJam’s features and present a technical evaluation. Second, we use CollabJam to understand communication and design patterns used during haptic experience design. We performed an in-depth design evaluation spanning four sessions in which four pairs of participants designed and reviewed vibrotactile experiences remotely. A qualitative content analysis revealed how multi-sensory communication is essential to convey ideas, how stimulating the tactile sense can interfere with personal boundaries, and how freely placing actuators on the skin can provide both benefits and challenges.2025DWDennis Wittchen et al.Dresden University of Applied Sciences, Faculty of Informatics / Mathematics; Max Planck Institute for Informatics, Saarland Informatics Campus, Sensorimotor InteractionVibrotactile Feedback & Skin StimulationHaptic WearablesCreative Collaboration & Feedback SystemsCHI
Motion-Coupled Asymmetric Vibration for Pseudo Force Rendering in Virtual RealityIn Virtual Reality (VR), rendering realistic forces is crucial for immersion, but traditional vibrotactile feedback fails to convey force sensations effectively. Studies of asymmetric vibrations that elicit pseudo forces show promise but are inherently tied to unwanted vibrations, reducing realism. Leveraging sensory attenuation to reduce the perceived intensity of self-generated vibrations during user movement, we present a novel algorithm that couples asymmetric vibrations with user motion, which mimics self-generated sensations. Our psychophysics study with 12 participants shows that motion-coupled asymmetric vibration attenuates the experience of vibration (equivalent to a \textasciitilde 30\% reduction in vibration-amplitude) while preserving the experience of force, compared to continuous asymmetric vibrations (state-of-the-art). We demonstrate the effectiveness of our approach in VR through three scenarios: shooting arrows, lifting weights, and simulating haptic magnets. Results revealed that participants preferred forces elicited by motion-coupled asymmetric vibration for tasks like shooting arrows and lifting weights. This research highlights the potential of motion-coupled asymmetric vibrations, offers new insights into sensory attenuation, and advances force rendering in VR.2025NSNihar Sabnis et al.Max Planck Institute for Informatics, Saarland Informatics Campus, Sensorimotor InteractionForce Feedback & Pseudo-Haptic WeightCHI
Spatial Haptics: A Sensory Substitution Method for Distal Object Detection Using Tactile CuesWe present a sensory substitution-based method for representing locations of remote objects in 3D space via haptics. By imitating auditory localization processes, we enable vibrotactile localization abilities similar to those of some spiders, elephants, and other species. We evaluated this concept in virtual reality by modulating the vibration amplitude of two controllers depending on relative locations to a target. We developed two implementations applying this method using either ear or hand locations. A proof-of-concept study assessed localization performance and user experience, achieving under 30° differentiation between horizontal targets with no prior training. This unique approach enables localization by using only two actuators, requires low computational power, and could potentially assist users in gaining spatial awareness in challenging environments. We compare the implementations and discuss the use of hands as ears in motion, a novel technique not previously explored in the sensory substitution literature.2025IWIddo Yehoshua Wald et al.University of Bremen, Digital Media LabVibrotactile Feedback & Skin StimulationFull-Body Interaction & Embodied InputCHI
Effects of Device Environment and Information Layout on Spatial Memory and Performance in VR Selection TasksVirtual Reality systems are increasingly proposed as a platform for everyday interactive software. Many applications are dependent on actions such as navigation and selection, but it is not clear how well immersive environments support these basic activities. Previous studies have suggested advantages for spatial learning in VR, so we carried out a study that investigated two aspects of immersion on spatial memory and selection: the degree to which the user is immersed in the data, and whether the system uses immersive input and output. The study showed that more-immersive conditions had substantially worse selection performance, and did not improve spatial learning. However, most participants believed that the immersive conditions were better for learning object locations, and most people preferred the immersive layout and the HMD. Our study suggests that designers should be cautious about assuming that everyday software applications will benefit from being deployed in an immersive VR environment.2024KKKim Kargut et al.University of SaskatchewanEye Tracking & Gaze InteractionImmersion & Presence ResearchCHI
GestureExplorer: Immersive Visualisation and Exploration of Gesture DataThis paper presents the design and evaluation of GestureExplorer, an Immersive Analytics tool that supports the interactive exploration, classification and sensemaking with large sets of 3D temporal gesture data. GestureExplorer features 3D skeletal and trajectory visualisations of gestures combined with abstract visualisations of clustered sets of gestures. By leveraging the large immersive space afforded by a Virtual Reality interface our tool allows free navigation and control of viewing perspective for users to gain a better understanding of gestures. We explored a selection of classification methods to provide an overview of the dataset that was linked to a detailed view of the data that showed different visualisation modalities. We evaluated GestureExplorer with two user studies and collected feedback from participants with diverse visualisation and analytics backgrounds. Our results demonstrated the promising capability of GestureExplorer for providing a useful and engaging experience in exploring and analysing gesture data.2023ALAng Li et al.Monash UniversityHand Gesture RecognitionInteractive Data VisualizationCHI
`Specially For You' -- Examining the Barnum Effect's Influence on the Perceived Quality of System RecommendationsThe ‘Barnum effect’ is a psychological phenomenon under which people assign higher quality ratings to personality descriptions developed ‘specially for you’ than the same descriptions described as ‘generally true of people.’ This effect suggests that recommender interfaces could elevate the perceived quality of recommendations simply by indicating that they are explicitly personalised. We therefore conducted a crowd-sourced experiment (n=492) that examined the perceived quality of personalised versus non-personalised movie recommendations for good and bad movies – importantly, the actual recommendations were identical, and were merely presented as being either personalised or not. Contrary to the Barnum effect, results showed numerically lower mean quality scores for personalised recommendations, but with no significant difference. Our findings suggest that Barnum-like effects of personalisation have at most a small influence on perceived quality, and that designers should not rely on this effect to improve user experience (despite online design guidance suggesting the opposite).2023PSPang Suwanaposee et al.University of CanterburyRecommender System UXVisualization Perception & CognitionCHI
XR-LIVE: Enhancing Asynchronous Shared-Space Demonstrations with Spatial-temporal Assistive Toolsets for Effective Learning in Immersive Virtual LaboratoriesAn immersive virtual laboratory (VL) could offer flexibility of time and space, as well as safety, for remote students to conduct laboratory activities through online experiential learning. Recording an instructor’s demonstration inside a VL is an approach that allows students to learn directly from a demonstration. However, students have to learn from a recording while controlling the playback, which requires additional spatial and temporal attention. This additional attention load could lead to mistakes in following laboratory procedures. We have identified four design requirements to reduce attention load in VLs; namely, organized learning steps, improved student sense of co-presence, reduction of task-instructor split-attention, and learning independent of interpersonal distance. Based on these requirements, we have designed and implemented spatial-temporal assistive toolsets for laboratories in virtual environment, namely XR-LIVE, that reduces mental load and enhance effective learning in an asynchronous shared-space demonstration, implemented based on the setup of a standard civil engineering laboratory. We also analyzed students’ behavior in the VL demonstration to design guidelines applicable to generic VLs.2022STSantawat Thanyadit et al.XR in Place and Space; XR in Place and SpaceCSCW
VRhook: A Data Collection Tool for VR Motion Sickness ResearchDespite the increasing popularity of VR games, one factor hindering the industry's rapid growth is motion sickness experienced by the users. Symptoms such as fatigue and nausea severely hamper the user experience. Machine Learning methods could be used to automatically detect motion sickness in VR experiences, but generating the extensive labeled dataset needed is a challenging task. It needs either very time consuming manual labeling by human experts or modification of proprietary VR application source codes for label capturing. To overcome these challenges, we developed a novel data collection tool, VRhook, which can collect data from any VR game without needing access to its source code. This is achieved by dynamic hooking, where we can inject custom code into a game's run-time memory to record each video frame and its associated transformation matrices. Using this, we can automatically extract various useful labels such as rotation, speed, and acceleration. In addition, VRhook can blend a customized screen overlay on top of game contents to collect self-reported comfort scores. In this paper, we describe the technical development of VRhook, demonstrate its utility with an example, and describe directions for future research.2022EWElliott Wen et al.Motion Sickness & Passenger ExperienceImmersion & Presence ResearchUIST
Probability Weighting in Interactive Decisions: Evidence for Overuse of Bad Assistance, Underuse of Good AssistanceThe effective use of assistive interfaces (i.e. those that offer suggestions or reform the user's input to match inferred intentions) depends on users making good decisions about whether and when to engage or ignore assistive features. However, prior work from economics and psychology shows systematic decision-making biases in which people overreact to low probability events and underreact to high probability events -- modelled using a probability weighting function. We examine the theoretical implications of this probability weighting for interaction, including its suggestion that users will overuse inaccurate interface assistance and underuse accurate assistance. We then conduct a new analysis of data from a previously published study, quantifying the degree of bias users exhibited, and demonstrating conformance with these predictions. We discuss implications for design, including strategies that could be used to mitigate the deleterious effects of the observed biases.2022ACAndy Cockburn et al.University of CanterburyExplainable AI (XAI)AI-Assisted Decision-Making & AutomationAI Ethics, Fairness & AccountabilityCHI