From Prompt to Presence: Co-Creating Personalised Emotional Sanctuaries in VR with Generative AIThe emergence of generative artificial intelligence (GenAI), combined with immersive virtual reality (VR), enables the rapid creation of personalised virtual content from simple text prompts, holding potential for emotional support. However, most current VR systems rely on pre-authored content and limit user agency in designing emotionally meaningful experiences. We introduce OasisMind, an AI-assisted VR system that empowers users to co-create 360° environments, corresponding ambient soundscapes, and context-aware digital companions through natural language prompts. In a user study (N=24), we observed how participants constructed virtual worlds for emotionally meaningful use cases and compared their creations to validated, pre-defined VR scenes recommended by previous research. Our results indicate a subjective preference for self-created environments, while no significant differences were observed in perceived satisfaction or presence between conditions. These findings suggest that user agency contributes to the emotional resonance of virtual experiences and inform the design of future personalised companion systems.2026RWRuoyu Wen et al.University of CanterburyGenerative AI (Text, Image, Music, Video)Immersion & Presence ResearchSocial & Collaborative VRIUI
Less is More! Visual Suppression for Bottom-up and Top-down Attention in Dynamic EnvironmentsDynamic virtual environments pose growing challenges for users who must manage attention across competing visual elements, where distractors can divert focus from relevant objects in these scenarios. Because human attention functions as a filter, it is shaped by competing influences from bottom-up salience and top-down relevance. We explore the salience and relevance of objects and introduce suppression-based visual filtering mechanisms, implemented through Dim and Blur visual filters at Weak and Strong intensity levels. A controlled abstract virtual environment with colorful moving objects was used to evaluate these against Baseline (no filtering) across nine varied salience-relevance situations, involving 38 participants in visual search and sustained monitoring tasks. Results showed that visual suppression enhanced participants' attention over Baseline, with Dim outperforming Blur, Strong exceeding Weak, and Dim-Strong achieving superior performance overall. These findings imply the principle of attention redistribution and offer insights for domains involving objects with varying salience and relevance.2026CZChenkai Zhang et al.Adelaide UniversityImmersion & Presence ResearchEye Tracking & Gaze InteractionAffective Feedback & Emotion Regulation InterfacesCHI
Investigating Bystander Privacy in Chinese Smart Home AppsBystander privacy in smart homes has been widely studied in Western contexts, yet it remains underexplored in non-Western countries such as China. In this study, we analyze 49 Chinese smart home apps using a mixed-methods approach, including privacy policy review, UX/UI evaluation, and assessment of Apple App Store privacy labels. While most apps nominally comply with national regulations, we identify significant gaps between written policies and actual implementation. Our traceability analysis highlights inconsistencies in data controls and a lack of transparency in data-sharing practices. Crucially, bystander privacy---particularly for visitors and non-user individuals---is largely absent from both policy documents and interface design. Additionally, discrepancies between privacy labels and actual data practices threaten user trust and undermine informed consent. We provide design recommendations to strengthen bystander protections, improve privacy-oriented UI transparency, and enhance the credibility of privacy labels, supporting the development of inclusive smart home ecosystems in non-Western contexts.2026SHShijing He et al.King's College LondonSmart Home Privacy & SecurityPrivacy by Design & User ControlPrivacy Perception & Decision-MakingCHI
Challenges in Synchronous & Remote Collaboration Around VisualizationWe characterize 16 challenges faced by those investigating and developing remote and synchronous collaborative experiences around visualization. Our work reflects the perspectives and prior research efforts of an international group of 29 experts from across human-computer interaction and visualization sub-communities. The challenges are anchored around five collaborative activities that exhibit a centrality of visualization and multimodal communication. These activities include exploratory data analysis, creative ideation, visualization-rich presentations, joint decision making grounded in data, and real-time data monitoring. The challenges also reflect the changing dynamics of these activities in the face of recent advances in extended reality (XR) and artificial intelligence (AI). As an organizing scheme for future research at the intersection of visualization and computer-supported cooperative work, we align the challenges with a sequence of four sets of research and development activities: technological choices, social factors, AI assistance, and evaluation.2026MBMatthew Brehmer et al.University of WaterlooInteractive Data VisualizationRemote Work Tools & ExperienceMulti-User Large Display CollaborationCHI
AI of Oz: Enhancing Wizard of Oz Studies in HCI with AI Assistance for Human ModerationThe Wizard of Oz (WoZ) method is a common and popular approach for simulating interactive systems in Human-Computer Interaction. Running such studies is demanding for researchers because the human wizard must manage human–agent interactions in real time while keeping participants safe and the interaction natural. Many WoZ systems struggle to reproduce complex agent behaviours without minimal delays or heavy workload for the moderator. We introduce AI of Oz, a framework that uses large language models to support researchers by monitoring ongoing interactions, detecting sensitive moments, and suggesting contextually appropriate responses. In a study with 20 HCI-related researchers, the system improved participants’ ability to manage interactions and maintain control compared to a version without AI support. We outline implications for WoZ research and note current limitations and future directions.2026RWRuoyu Wen et al.University of CanterburyHuman-LLM CollaborationUser Research Methods (Interviews, Surveys, Observation)Prototyping & User TestingCHI
Cognitive Bridge: AI-Generated Boundary Objects for Cross-Functional CollaborationCross-functional teams struggle when static collaboration tools fail to keep pace with dynamic conversations. Through a formative study with seven professionals, we identified a critical gap: designers and developers speak different vocabularies, causing semantic misalignments. We present Cognitive Bridge, an AI system that monitors multimodal cues (facial expressions, speech, workspace activity) to detect emerging misunderstandings, then generates adaptive boundary objects, visual diagrams, wireframes, and flowcharts that translate between professional perspectives in real-time. Our controlled study with 16 designer-developer dyads found that Cognitive Bridge reduced communication conflicts by 47% and increased implementable solutions by 34% compared to baseline tools. However, analysis revealed a solution-exploration tradeoff: while AI accelerated alignment, it risked premature convergence that constrained creative exploration. We contribute: (1) a novel system for AI-generated boundary objects, and (2) design implications for balancing cognitive scaffolding with creative agency preservation.2026TGTamil Selvan Gunasekaran et al.The University of AucklandHuman-LLM CollaborationCrowdsourcing Task Design & Quality ControlDistributed Team CollaborationCHI
The People's Gaze: Co-Designing and Refining Gaze Gestures with Users and ExpertsAs eye-tracking becomes increasingly common in modern mobile devices, the potential for hands-free, gaze-based interaction grows, but current gesture sets are largely expert-designed and often misaligned with how users naturally move their eyes. To address this gap, we introduce a two-phase methodology for developing intuitive gaze gestures. First, four co-design workshops with 20 non-expert participants generated 102 initial concepts. Next, four gaze interaction experts reviewed and refined these into a set of 32 gestures. We found that non-experts, after a brief introduction, intuitively anchor gestures in familiar metaphors and develop a compositional grammar; i.e., activation (dwell) + action (gaze gesture or blink), to ensure intentionality and mitigate the classic Midas Touch problem. Experts prioritized gestures that are ergonomically sound, aligned with natural saccades, and reliably distinguishable. The resulting user-grounded, expert-validated gesture set, along with actionable design principles, provides a foundation for developing intuitive, hands-free interfaces for gaze-enabled devices.2026YLYaxiong Lei et al.University of St AndrewsEye Tracking & Gaze InteractionHand Gesture RecognitionPrototyping & User TestingCHI
“My Happiness Makes You Smile”: Beginning to Understand Telepathic Superpower Design Via Brain-Muscle Interfaces Designing superpowers in Human-Computer Interaction (HCI), often inspired by science fiction, has garnered increased attention. However, it is important to ask whether such superpower designs might have inherent negative side effects, especially considering that technological advances allow going beyond short demos to integrate these superpowers into everyday life. To understand the positive and negative side effects of superpower design, we created "EmoPals" and studied it in everyday life. EmoPals is a novel system inspired by telepathy, where one user's emotions are detected through a brain-computer interface and replicated on the other user's face through electrical muscle stimulation, therefore one user's happiness makes the other smile and vice versa. A 5-day field study with 12 participants suggests that EmoPals can strengthen emotional connections and facilitate empathy, however, it also highlights the negative side effects of amplifying negative emotions and social discomfort. We propose five design recommendations for designing superpowers that account for negative side effects. Ultimately, we aim to deepen our understanding of superpower design for everyday life.2025SLSiyi Liu et al.Electrical Muscle Stimulation (EMS)Brain-Computer Interface (BCI) & NeurofeedbackDIS
Exploring the Privacy and Security Challenges Faced by Migrant Domestic Workers in Chinese Smart HomesThe growing use of smart home devices poses considerable privacy and security challenges, especially for individuals like migrant domestic workers (MDWs) who may be surveilled by their employers. This paper explores the privacy and security challenges experienced by MDWs in multi-user smart homes through in-depth semi-structured interviews with 26 MDWs and 5 staff members of agencies that recruit and/or train domestic workers in China. Our findings reveal power imbalances in the relationships between MDWs and their employers and agencies, influenced by Chinese cultural and social factors (such as Confucianism and collectivism) as well as legal ones. Furthermore, the widespread and normalized use of surveillance technologies in China, particularly in public spaces, exacerbates these power imbalances, reinforcing a sense of constant monitoring and control. Drawing on our findings, we provide recommendations for domestic worker agencies and policymakers to address the privacy and security challenges faced by MDWs in Chinese smart homes.2025SHShijing He et al.King's College LondonPrivacy by Design & User ControlPrivacy Perception & Decision-MakingSmart Home Privacy & SecurityCHI
The Brain Knows What You Prefer: Using EEG to Decode AR Input PreferencesUnderstanding user input preferences is crucial in immersive environments, where input methods such as gestures and controllers are common. Traditional evaluation methods rely on post experience questionnaires, which don't capture real-time preferences. This study used brain signals to classify input preferences during Augmented Reality (AR) interactions. Thirty participants performed three interaction tasks (pointing, manipulation, and rotation) using hands or controllers. Their electroencephalogram (EEG) data were collected at varying task difficulties (low, medium, high) and phases (preparation, task, and completion). Machine learning was used to classify preferred and non-preferred input methods. Results showed that EEG signals effectively classify preferences with accuracies up to 86%, with the completion phase being the best indicator of preference. In addition, different input methods exhibited distinct EEG patterns. These findings highlight the potential of EEG signals for decoding real-time input preference in AR, offering insights for enhancing user experiences.2025KZKaining Zhang et al.University of South Australia, Empathic Computing LabBrain-Computer Interface (BCI) & NeurofeedbackAR Navigation & Context AwarenessCHI
Strollytelling: Coupling Animation with Physical Locomotion to Explore Immersive Data StoriesWith a growing interest in immersive data storytelling, there is an opportunity to explore story presentation and navigation techniques in virtual reality (VR) that can engage audiences as much as data story techniques have on conventional displays. We propose and explore “strolly”telling, a novel data storytelling technique that maps the story progression with the user/audience’s physical locomotion. Inspired by the conventional web-based technique for scrolling-based stories (i.e. scrollytelling), our technique tightly couples the user’s position in physical space to the animation frame of the data story. This technique leverages the natural tendency of humans to "walk and talk" while telling a story and requires users to engage with the content actively. This work defines strollytelling, design considerations, and a preliminary process for designing a strollytelling experience. A user study comparing strollytelling with virtual locomotion found that strollytelling was preferred by most participants and had higher self-reported immersion. We conclude with opportunities for strollytelling within the immersive data storytelling landscape.2025RJRADHIKA PANKAJ JAIN et al.University of South Australia, IVEData StorytellingInteractive Narrative & Immersive StorytellingCHI
InfoPrint: Embedding Interactive Information in 3D Prints Using Low-Cost Readily-Available Printers and MaterialsJiang 等人提出 InfoPrint 方法，利用低成本普通3D打印机和常规材料在打印物体中嵌入交互式信息，实现物理对象的数字化增强与可编程功能。2024WJWeiwei Jiang et al.Desktop 3D Printing & Personal FabricationCustomizable & Personalized ObjectsUbiComp
RadarHand: a Wrist-Worn Radar for On-Skin Touch based Proprioceptive GesturesWe introduce RadarHand, a wrist-worn wearable with millimetre wave radar that detects on-skin touch-based proprioceptive hand gestures. Radars are robust, private, small, penetrate materials, and require low computation costs. We first evaluated the proprioceptive and tactile perception nature of the back of the hand and found that tapping on the thumb is the least proprioceptive error of all the finger joints, followed by the index finger, middle finger, ring finger, and pinky finger in the eyes-free and high cognitive load situation. Next, we trained deep-learning models for gesture classification. We introduce two types of gestures based on the locations of the back of the hand: generic gestures and discrete gestures. Discrete gestures are gestures that start at specific locations and end at specific locations at the back of the hand, in contrast to generic gestures, which can start anywhere and end anywhere on the back of the hand. Out of 27 gesture group possibilities, we achieved 92% accuracy for a set of seven gestures and 93% accuracy for the set of eight discrete gestures. Finally, we evaluated RadarHand’s performance in real-time under two interaction modes: Active interaction and Reactive interaction. Active interaction is where the user initiates input to achieve the desired output, and reactive interaction is where the device initiates interaction and requires the user to react. We obtained an accuracy of 87% and 74% for active generic and discrete gestures, respectively, as well as 91% and 81.7% for reactive generic and discrete gestures, respectively. We discuss the implications of RadarHand for gesture recognition and directions for future works.2024MHMr Ryo Hajika et al.Vibrotactile Feedback & Skin StimulationFoot & Wrist InteractionUIST
The RayHand Navigation: A Virtual Navigation Method with Relative Position between Hand and Gaze-RayIn this paper, we introduce a novel Virtual Reality (VR) navigation method using gaze ray and hand, named RayHand navigation. It supports controlling navigation speed and direction by quickly indicating the initial direction using gaze and then using dexterous hand movement for controlling the speed and direction based on the relative position between the gaze ray and user’s hand. We conducted a user study comparing our approach to the head-hand and torso-leaning-based navigation methods, and also evaluated their learning effect. The results showed that the RayHand and head-hand navigations were less physically demanding than the torso-leaning navigation, and the RayHand supported rich navigation experience with high hedonic quality and solved the issue of the user unintentionally stepping out from the designated interaction area. In addition, our approach showed a significant improvement over time with a learning effect.2024SKSei Kang et al.Chonnam National UniversityFull-Body Interaction & Embodied InputEye Tracking & Gaze InteractionImmersion & Presence ResearchCHI
That's Rough! Encoding Data into Roughness for PhysicalizationWhile visual channels (e.g., color, shape, size) have been explored for visualizing data in data physicalizations, there is a lack of understanding regarding how to encode data into physical material properties (e.g., roughness, hardness). This understanding is critical for ensuring data is correctly communicated and for potentially extending the channels and bandwidth available for encoding that data. We present a method to encode ordinal data into roughness, validated through user studies. In the first study, we identified just noticeable differences in perceived roughness from this method. In the second study, we 3D-printed proof of concepts for five different multivariate physicalizations using the model. These physicalizations were qualitatively explored (N=10) to understand people's comprehension and impressions of the roughness channel. Our findings suggest roughness may be used for certain types of data encoding, and the context of the data can impact how people interpret roughness mapping direction.2024XDXiaojiao Du et al.University of South AustraliaData PhysicalizationVisualization Perception & CognitionCHI
Towards Applied Remapped Physical-Virtual Interfaces: Synchronization Methods for Resolving Control State ConflictsUser interfaces in virtual reality enable diverse interactions within the virtual world, though they typically lack the haptic cues provided by physical interface controls. Haptic retargeting enables flexible mapping between dynamic virtual interfaces and physical controls to provide real haptic feedback. This investigation aims to extend these remapped interfaces to support more diverse control types. Many interfaces incorporate sliders, switches, and knobs. These controls hold fixed states between interactions creating potential conflicts where a virtual control has a different state from the physical control. This paper presents two methods, ``manual'' and ``automatic'', for synchronizing physical and virtual control states and explores the effects of these methods on the usability of remapped interfaces. Results showed that interfaces without retargeting were the ideal configuration, but they lack the flexibility that remapped interfaces provide. Automatic synchronization was faster and more usable; however, manual synchronization is suitable for a broader range of physical interfaces.2023BMBrandon J Matthews et al.University of South Australia, University of South AustraliaForce Feedback & Pseudo-Haptic WeightMixed Reality WorkspacesImmersion & Presence ResearchCHI
ProxSituated Visualization: An Extended Model of Situated Visualization using Proxies for Physical Referents Existing situated visualization models assume the user is able to directly interact with the objects and spaces to which the data refers (known as physical referents). We review a growing body of work exploring scenarios where the user interacts with a proxy representation of the physical referent rather than immediately with the object itself. This introduces a complex mixture of immediate situatedness and proxies of situatedness that goes beyond the expressiveness of current models. We propose an extended model of situated visualization that encompasses Immediate Situated Visualization and ProxSituated (Proxy of Situated) Visualization. Our model describes a set of key entities involved in proxSituated scenarios and important relationships between them. From this model, we derive design dimensions and apply them to existing situated visualization work. The resulting design space allows us to describe and evaluate existing scenarios, as well as to creatively generate new conceptual scenarios.2023KSKadek Ananta Satriadi et al.University of South Australia, Monash UniversityInteractive Data VisualizationContext-Aware ComputingCHI
The Impact of Sharing Gaze Behaviours in Collaborative Mixed RealityIn a remote collaboration involving a physical task, visualising gaze behaviours may compensate for other unavailable communication channels. In this paper, we report on a 360° panoramic Mixed Reality (MR) remote collaboration system that shares gaze behaviour visualisations between a local user in Augmented Reality and a remote collaborator in Virtual Reality. We conducted two user studies to evaluate the design of MR gaze interfaces and the effect of gaze behaviour (on/off) and gaze style (bi-/uni-directional). The results indicate that gaze visualisations amplify meaningful joint attention and improve co-presence compared to a no gaze condition. Gaze behaviour visualisations enable communication to be less verbally complex therefore lowering collaborators’ cognitive load while improving mutual understanding. Users felt that bi-directional behaviour visualisation, showing both collaborator’s gaze state, was the preferred condition since it enabled easy identification of shared interests and task progress.2022AJAllison Jing et al.XR Collaboration; XR CollaborationCSCW
VRhook: A Data Collection Tool for VR Motion Sickness ResearchDespite the increasing popularity of VR games, one factor hindering the industry's rapid growth is motion sickness experienced by the users. Symptoms such as fatigue and nausea severely hamper the user experience. Machine Learning methods could be used to automatically detect motion sickness in VR experiences, but generating the extensive labeled dataset needed is a challenging task. It needs either very time consuming manual labeling by human experts or modification of proprietary VR application source codes for label capturing. To overcome these challenges, we developed a novel data collection tool, VRhook, which can collect data from any VR game without needing access to its source code. This is achieved by dynamic hooking, where we can inject custom code into a game's run-time memory to record each video frame and its associated transformation matrices. Using this, we can automatically extract various useful labels such as rotation, speed, and acceleration. In addition, VRhook can blend a customized screen overlay on top of game contents to collect self-reported comfort scores. In this paper, we describe the technical development of VRhook, demonstrate its utility with an example, and describe directions for future research.2022EWElliott Wen et al.Motion Sickness & Passenger ExperienceImmersion & Presence ResearchUIST
Emotion Recognition in Conversations using Brain and Physiological SignalsEmotions are complicated psycho-physiological processes that are related to numerous external and internal changes in the body. They play an essential role in human-human interaction and can be important for human-machine interfaces. Automatically recognizing emotions in conversation could be applied in many application domains like health-care, education, social interactions, and entertainment. Facial expressions, speech, and body gestures are primary cues that have been widely used for recognizing emotions in conversation. However, these cues can be ineffective as they cannot reveal underlying emotions when a person involuntarily or deliberately conceals their emotions. Researchers have shown that analyzing brain activity and physiological signals can lead to more reliable emotion recognition since they generally cannot be controlled. However, these body responses in emotional situations have been rarely explored in interactive tasks like conversations. This paper explores and discusses the performance and challenges of using brain activity and other physiological signals in recognizing emotions in a face-to-face conversation. We present an experimental setup for stimulating spontaneous emotions during a face-to-face conversation while recording brain and physiological activity. We then describe our analysis strategies for recognizing emotions using Electroencephalography (EEG), Photoplethysmography (PPG), and Galvanic Skin Responses (GSR) signals in a subject-dependent and subject-independent approach. Finally, we describe new directions for future research in conversational emotion recognition, and the limitations and challenges.2022NSNastaran Saffaryazdi et al.Brain-Computer Interface (BCI) & NeurofeedbackBiosensors & Physiological MonitoringIUI