Design Considerations for Human Oversight of AI: Insights from Co-Design Workshops and Work Design TheoryAs AI systems become increasingly capable and autonomous, domain experts’ roles are shifting from performing tasks themselves to overseeing AI-generated outputs. Such oversight is critical, as undetected errors can have serious consequences or undermine the benefits of AI. Effective oversight, however, depends not only on detecting and correcting AI errors but also on the motivation and engagement of the oversight personnel and the meaningfulness they see in their work. Yet little is known about how domain experts approach and experience the oversight task and what should be considered to design effective and motivational interfaces that support human oversight. To address these questions, we conducted four co-design workshops with domain experts from psychology and computer science. We asked them to first oversee an AI-based grading system, and then discuss their experiences and needs during oversight. Finally, they collaboratively prototyped interfaces that could support them in their oversight task. Our thematic analysis revealed four key user requirements: understanding tasks and responsibilities, gaining insight into the AI’s decision-making, contributing meaningfully to the process, and collaborating with peers and the AI. We integrated these empirical insights with the SMART model of work design to develop a framework of twelve design considerations with increased transferability compared to the identified user requirements. Our framework links interface characteristics and user requirements to the psychological processes underlying effective and satisfying work. Being grounded in work design theory and overlapping with existing guidelines for human–AI interaction, we expect these considerations to be applicable across domains and discuss how they go beyond existing guidelines for human-AI interaction to inform the design of engaging and meaningful interfaces that support human oversight of AI-based systems.2026CFCedric Faas et al.Saarland UniversityAI-Assisted Decision-Making & AutomationExplainable AI (XAI)Participatory DesignIUI
Understanding How Mobile Interactions Shape Grasp and Contact Patterns Beyond the TouchscreenThe way users hold a smartphone depends on the interaction task, yet little is known about the fingers' engagement with the device's surfaces beyond the touchscreen. Such an understanding not only opens up opportunities for novel on- and off-screen interactions, but also the device’s possible physical affordances. We present a study (N=23) that examines the hands' physical engagement with the smartphone beyond the touchscreen across nine mobile interactions. Grasps were annotated from photographs, and contact regions were captured using residual heat traces from grasping the device. Our findings show that fingers and palms adopt a variety of support roles and postures when engaging with the smartphone's back and side edges. The hand-contact maps reveal distinct patterns, differing in contact frequency and placement. This work contributes an empirical characterisation of hands' back and edge engagement, highlighting design opportunities for future smartphone usage extending beyond the touchscreen.2026CSCarolin Stellmacher et al.University of BremenOne-Handed Operation & Mobile GesturesTouch Target Selection & PointingCHI
Beyond Descriptions: A Generative Scene2Audio Framework for Blind and Low-Vision Users to Experience Vista LandscapesCurrent scene perception tools for Blind and Low Vision (BLV) individuals rely on spoken descriptions but lack engaging representations of visually pleasing distant environmental landscapes (Vista spaces). Our proposed Scene2Audio framework generates comprehensible and enjoyable nonverbal audio using generative models informed by psychoacoustics, and principles of scene audio composition. Through a user study with 11 BLV participants, we found that combining the Scene2Audio sounds with speech creates a better experience than speech alone, as the sound effects complement the speech making the scene easier to imagine. A mobile app “in-the-wild” study with 7 BLV users for more than a week further showed the potential of Scene2Audio in enhancing outdoor scene experiences. Our work bridges the gap between visual and auditory scene perception by moving beyond purely descriptive aids, addressing the aesthetic needs of BLV users.2026CGChitralekha Gupta et al.National University of SingaporeAudio Accessibility (Captions, Sign Language, Vibration)Emotion-Sensing WearablesVisual Impairment Technologies (Screen Readers, Tactile Graphics, Braille)CHI
A Systematic Review of Interaction Techniques for Mobile Virtual RealityWhile low-cost smartphone-based mobile VR (MVR) improves access to extended reality (XR) technology, it lacks the interaction capabilities of high-end devices. Following PRISMA 2020 methodology, we present a survey of both established and emerging MVR interaction techniques for travel, selection, manipulation, and system control. We reviewed literature from four databases published between 2011–2025 that reported evaluations of MVR interaction techniques. We filtered an initial set of 1041 publications to 64 articles and synthesized the current state of MVR interaction, focusing on cost-accessible approaches. We found many effective low-cost emerging selection and travel techniques, but low-cost object manipulation techniques remain problematic. Acoustic sensing offered superior 3D interaction performance than other sensing modalities while keeping cost low. Our findings inform a novel taxonomy of emerging MVR interaction techniques. We further present a toolkit supporting the design of cost-accessible XR interactions. Our findings underscore practical advantages of DIY approaches to future standalone XR applications developments.2026KGKristen Grinyer et al.Carleton UniversityAR Navigation & Context AwarenessImmersion & Presence ResearchContext-Aware ComputingCHI
HaptEx: Investigating Haptic Notification Channels for Exoskeletons Across Different Levels of ActuationExoskeletons are increasingly deployed in real-world contexts, where communicating critical system states or unexpected events is important for effective interaction. Haptic feedback offers a direct communication channel, integrating naturally with the actuated body region. Yet, it remains unclear how well haptic feedback is perceived while the body is being actuated. In a controlled study (N=24) with a shoulder exoskeleton, we compare four common haptic notification channels (poking, proprioceptive, thermal, vibrotactile) under different levels of actuation. Results show that poking was detected fastest, while thermal and proprioceptive notifications were most accurate and noticeable. Actuation levels affected error rates and noticeability, but not response times. Participants reported that thermal notifications aligned best with the actuation levels, producing a distinct sensation that blended naturally with movement. In contrast, proprioceptive notifications conveyed the strongest sense of urgency. We discuss design implications for leveraging haptic notifications to support embodied communication with exoskeletons.2026MMMarie Muehlhaus et al.Saarland Informatics CampusVibrotactile Feedback & Skin StimulationForce Feedback & Pseudo-Haptic WeightHaptic WearablesCHI
Forefeel the Move: Investigating Proprioceptive Feedback for Communicating Imminent Motions of Body-actuating SystemsSystems actuating the body can proactively assist users in diverse tasks. However, unexpected body actuation may pose safety risks. We propose proprioceptive feedback to inform users about an imminent actuation before the system takes control. In a user study, we compare different proprioceptive cues that either interrupt or augment user motion to convey (1) solely that a body actuation is imminent, (2) its direction, or (3) its target. To enable a controlled investigation, we confined the cues to one degree-of-freedom joints and implemented them in an elbow exoskeleton. The results show that all cues are highly noticeable, offering an integrated feedback channel; yet, their effectiveness in communicating direction and target differed: While cues that augmented user motion were more accurate and preferred, disruptive cues enabled faster but less accurate interpretations. Furthermore, our analysis revealed that proprioceptive feedback enhanced the expressiveness of the conveyed information and user's aspirations for adaptive feedback.2026MMMarie Muehlhaus et al.Saarland Informatics CampusForce Feedback & Pseudo-Haptic WeightHaptic WearablesVibrotactile Feedback & Skin StimulationCHI
EmbroForm: Digital Fabrication of Soft Freeform Objects with Machine Embroidered Pull-up StringsPull-up objects form 3D shapes by pulling a string routed through a 2D material, offering low-cost 2D fabrication and reversible transformation. However, existing approaches rely on origamic folding, which creates faceted, oftentimes rigid surfaces and requires manual pull-up string routing. We introduce EmbroForm, a digital fabrication pipeline for fully soft pull-up objects with organic, higher-fidelity shapes. Instead of folding, EmbroForm forms 3D shapes by seaming the boundaries of a flexible 2D patch unwrapped from the target. To enable this, we contribute a fabrication technique that automates the routing of sliding strings on flexible sheet materials with machine embroidery, which we extend on to design zig-zag lacings for seaming the boundaries. Then we introduce an end-to-end pipeline that, given a 3D mesh, creates an optimized 2D unwrapped patch and generates pull-up string routing paths for fabrication. We provide a design tool for customization and validate our approach with technical experiments and implemented application cases.2026YJYu Jiang et al.Saarland University, Saarland Informatics CampusShape-Changing Interfaces & Soft Robotic MaterialsCircuit Making & Hardware PrototypingCustomizable & Personalized ObjectsCHI
Privacy & Safety Challenges of On-Body Interaction TechniquesOn-body computing systems offer new forms of interaction, but while they are increasingly integrated into everyday contexts, their unique privacy and safety challenges remain understudied. This paper examines these challenges through a two-round interview study with $N = 15$ experts in human-computer interaction, and privacy and safety, using speculative scenarios and adversarial roleplaying to elicit insights. Our findings reveal risks specific to on-body interactions, including over-collection of sensitive data, unwanted inferences, harm to bystanders, and threats to bodily autonomy and psychological well-being. Importantly, in the on-body context, privacy and safety concerns are deeply interconnected and cannot be addressed in isolation. We contribute an empirically grounded characterization of these entangled challenges and derive eight actionable design guidelines to support safer, more privacy-aware, on-body systems. This work informs future research and design in ubiquitous computing by highlighting the need for proactive and integrated approaches to privacy and safety in trustworthy on-body computing.2026DGDañiel Gerhardt et al.CISPA Helmholtz Center for Information SecurityContext-Aware ComputingPrivacy by Design & User ControlPrivacy Perception & Decision-MakingCHI
Efficient Human-in-the-Loop Optimization via Priors Learned from User ModelsHuman-in-the-loop optimization identifies optimal interface designs by iteratively observing user performance. However, it often requires numerous iterations due to the lack of prior information. While recent approaches have accelerated this process by leveraging previous optimization data, collecting user data remains costly and often impractical. We present a conceptual framework, Human-in-the-Loop Optimization with Model-Informed Priors (HOMI), which augments human-in-the-loop optimization with a training phase where the optimizer learns adaptation strategies from diverse, synthetic user data generated with predictive models before deployment. To realize HOMI, we introduce Neural Acquisition Function+ (NAF+), a Bayesian optimization method featuring a neural acquisition function trained with reinforcement learning. NAF+ learns optimization strategies from large-scale synthetic data, improving efficiency in real-time optimization with users. We evaluate HOMI and NAF+ with mid-air keyboard optimization, a representative VR input task. Our work presents a new approach for more efficient interface adaptation by bridging in situ and in silico optimization processes.2026YLYi-Chi Liao et al.ETH ZürichMid-Air Haptics (Ultrasonic)Hand Gesture RecognitionImmersion & Presence ResearchCHI
User-reconfigured Haptics: Combining User-Reconfiguration and Visual Manipulations to Enhance Dynamic Passive Haptic Experiences for VRVirtual Reality (VR) depends on haptic feedback to create immersive experiences. Traditional passive proxies align physical props with their virtual counterparts but remain limited in scalability and expressiveness, or require bulky actuators to support reconfiguration. We introduce User-reconfigured Haptics, an approach that utilizes implicit user actions to reconfigure haptic interfaces to extend the gamut of VR haptic experiences. Modular 3D-printed cells are assembled into dynamic interfaces that express diverse haptic properties such as softness and weight. By masking physical reconfigurations with visual (re)mapping, user actions unnoticeably change haptic properties, resulting in user-driven, dynamic haptic experiences. User studies show that our design can provide distinguishable haptic experiences and is perceived as realistic and enjoyable in a VR task. We further showcase four applications: a fishing rod that changes weight and flexibility, a dynamic desktop of pressable buttons, a glove with adjustable squeezing, and a crossbow with variable pulling resistance.2026XWXinrong Wang et al.Saarland Informatics Campus (DFKI)Haptic WearablesImmersion & Presence ResearchShape-Changing Interfaces & Soft Robotic MaterialsCHI
Scene2Hap: Generating Scene-Wide Haptics for VR from Scene Context with Multimodal LLMsHaptic feedback contributes to immersive virtual reality (VR) experiences. However, designing such feedback at scale for all objects within a VR scene remains time-consuming. We present Scene2Hap, an LLM-centered system that automatically designs object-level vibrotactile feedback for entire VR scenes based on the objects' semantic attributes and physical context. Scene2Hap employs a multimodal large language model to estimate each object’s semantics and physical context, including its material properties and vibration behavior, from multimodal information in the VR scene. These estimated attributes are then used to generate or retrieve audio signals, subsequently converted into plausible vibrotactile signals. For more realistic spatial haptic rendering, Scene2Hap estimates vibration propagation and attenuation from vibration sources to neighboring objects, considering the estimated material properties and spatial relationships of virtual objects in the scene. Three user studies confirm that Scene2Hap successfully estimates the vibration-related semantics and physical context of VR scenes and produces realistic vibrotactile signals.2026AJArata Jingu et al.Saarland Informatics CampusMid-Air Haptics (Ultrasonic)Social & Collaborative VRImmersion & Presence ResearchCHI
Connected Material Experiences using Bimanual Vibrotactile Crosstalk in Virtual RealityPerceiving material properties such as elasticity, flexibility, and torsion is inherently bimanual, as we rely on the relative motion of our hands to form a unified sense of materiality. Yet, most vibrotactile material rendering approaches are limited to a single hand or finger. While prior work has explored bimanual haptic interfaces, most depend on specialized hardware for specific interactions. In this paper, we demonstrate design strategies to support bimanual material exploration through motion-coupled vibrotactile feedback. Our technique introduces variable crosstalk between the controllers' vibration to evoke connectedness, making two unconnected devices feel as though they manipulate a single object. The technique generalizes motion-coupled feedback approaches beyond previous single-point explorations. Through two user studies, we show that this approach (1) significantly enhances perceived connectedness and (2) conveys distinct material qualities such as elasticity and torsion. Finally, we present \textit{Dvihastīya}, an authoring tool for designing connected bimanual experiences in virtual reality.2026NSNihar Sabnis et al.Max Planck Institute for Informatics, Saarland Informatics CampusIn-Vehicle Haptic, Audio & Multimodal FeedbackImmersion & Presence ResearchShape-Changing Interfaces & Soft Robotic MaterialsCHI
Move with Style! Enhancing Avatar Embodiment in Virtual Reality through Proprioceptive Motion FeedbackIn virtual reality (VR), users slip into a variety of roles, represented by a rich diversity of avatars that each exhibit specific visual attributes and motion styles. While users can see their avatar's motion in VR, they usually cannot feel it. To enhance avatar embodiment, we propose active proprioceptive feedback that aligns users' physical movements with the expected motion style of their avatar, for instance, by mimicking the avatar's weight, typical motion speed or motion range. We introduce a conceptual space of relevant motion properties which enable designers to create expressive proprioceptive motion styles for avatars. We instantiate this concept with MotionStyler: a system for designing customized motion styles and rendering them in real-time with an arm-based exoskeleton that is synchronized with the VR avatar. Results from a survey confirmed the expressiveness of the proposed conceptual space. A user study demonstrated the system's capability to create diverse proprioceptive motion styles which enhance user's self-identification with their avatar and thereby positively contribute to avatar embodiment in VR.2025DWDavid Wagmann et al.Force Feedback & Pseudo-Haptic WeightIdentity & Avatars in XRUIST
Imaginary Joint: Proprioceptive Feedback for Virtual Body Extensions via Skin StretchVirtual body extensions such as a wing or tail have the potential to offer users new bodily experiences and capabilities in virtual and augmented reality. To use these extensions as naturally as one’s own body—particularly for body parts that are normally hard to see, such as a tail—it is essential to provide proprioceptive feedback that allows users to perceive the position, orientation, and force exerted by these parts, rather than relying solely on visual cues. In this study, we propose a novel approach by introducing an "Imaginary Joint" at the interface between the user's actual body and the virtual extension, delivering information about joint flexion and force through skin-stretch feedback. We present a wearable device for skin-stretch feedback and explore informing mappings that convey the bending rotation and torque of the Imaginary Joint. The final system presents both types of information simultaneously by superimposing these skin deformations. Results from a controlled experiment with users demonstrate that users could identify tail position and force without relying on visual cues, and do so more effectively than in the vibrotactile condition. Furthermore, the tail was perceived as more embodied than in a vibrotactile condition, resulting in a more naturalistic and intuitive sensation. Finally, we introduce several application scenarios, including Perception of Extended Bodies, Enhanced Bodily Expression, and Body-Mediated Communication, and discuss the potential for future extensions of this system.2025STShuto Takashita et al.Haptic WearablesShape-Changing Interfaces & Soft Robotic MaterialsDance & Body Movement ComputingUIST
eTactileKit: A Toolkit for Design Exploration and Rapid Prototyping of Electro-Tactile InterfacesElectro-tactile interfaces are becoming increasingly popular due to their unique advantages, such as delivering fast and localised tactile response, thin and flexible form factors, and the potential to create novel tactile experiences. However, insights from a formative study with typical designers highlighted the lack of resources, limited access to information and complexity of software and hardware tools. This establishes a high barrier to entry and limits the ability to rapidly prototype and experiment with electro-tactile interfaces. To address these challenges, we propose eTactileKit, a scalable and accessible toolkit providing end-to-end support for designing and prototyping electro-tactile interfaces. eTactileKit comprises a hardware platform and a software framework for designing, simulating and exploring electro-tactile stimuli. We evaluated the impact and usability of eTactileKit through a three-week long take-home study, which demonstrated increased accessibility, ease of use, and the toolkit's positive impact on design workflow. Additionally, we implemented a set of use cases to demonstrate the toolkit's practicality and effectiveness across various applications.2025PPPraneeth Bimsara Perera et al.Electrical Muscle Stimulation (EMS)Prototyping & User TestingUIST
Texergy: Textile-based Harvesting, Storing, and Releasing of Mechanical Energy for Passive On-Body ActuationHumans instinctively manipulate and "actuate" their clothing, for instance, to adapt to the environment or to modify aesthetics. However, such manual actuation remains inflexible and directly tied to user action. We introduce Texergy, a textile-based technical framework that decouples user input and actuated output to make passive on-body actuation interactive and programmable. Texergy achieves this by harvesting energy from user interactions with a set of input modules, storing it mechanically on the body in elastic materials, later releasing the energy on demand, and finally connecting to output end-effectors that realize the actuation. We present a fabrication approach based on almost entirely textile materials using laser-cutting and simple manual assembly to enable integration into clothing and easy prototyping. We report the results of technical experiments and provide a design tool to support customizing the actuation’s force and distance, type of harvesting, and deployment of Texergy mechanisms. We practically demonstrate the capabilities of Texergy with four applications, including a quick-release belt, a passive exosuit with dynamic assistance, a haptic feedback top powered by implicit user actions in VR, and a dance-driven shape-changing costume.2025YJYu Jiang et al.Force Feedback & Pseudo-Haptic WeightHaptic WearablesShape-Changing Interfaces & Soft Robotic MaterialsUIST
GestureCoach: Rehearsing for Engaging Talks with LLM-Driven Gesture RecommendationsThis paper introduces GestureCoach, a system designed to help speakers deliver more engaging talks by guiding them to gesture effectively during rehearsal. GestureCoach combines an LLM-driven gesture recommendation model with a rehearsal interface that proactively cues speakers to gesture appropriately. Trained on experts’ gesturing patterns from TED talks, the model consists of two modules: an emphasis proposal module, which predicts when to gesture by identifying gesture-worthy text segments in the presenter notes, and a gesture identification module, which determines what gesture to use by retrieving semantically appropriate gestures from a curated gesture database. Results of a model performance evaluation and user study (N=30) show that the emphasis proposal module outperforms off-the-shelf LLMs in identifying suitable gesture regions, and that participants rated the majority of these predicted regions and their corresponding gestures as highly appropriate. A subsequent user study (N=10) showed that rehearsing with GestureCoach encouraged speakers to gesture and significantly increased gesture diversity, resulting in more engaging talks. We conclude with design implications for future AI-driven rehearsal systems.2025ARAshwin Ram et al.Hand Gesture RecognitionHuman-LLM CollaborationCreative Collaboration & Feedback SystemsUIST
Learn, Explore and Reflect by Chatting: Understanding the Value of an LLM-Based Voting Advice Application ChatbotVoting advice applications (VAAs), which have become increasingly prominent in European elections, are seen as a successful tool for boosting electorates' political knowledge and engagement. However, VAAs' complex language and rigid presentation constrain their utility to less-sophisticated voters. While previous work enhanced VAAs' click-based interaction with scripted explanations, a conversational chatbot's potential for tailored discussion and deliberate political decision-making remains untapped. Our exploratory mixed-method study investigates how LLM-based chatbots can support voting preparation. We deployed a VAA chatbot to 331 users before Germany's 2024 European Parliament election, gathering insights from surveys, conversation logs, and 10 follow-up interviews. Participants found the VAA chatbot intuitive and informative, citing its simple language and flexible interaction. We further uncovered VAA chatbots' role as a catalyst for reflection and rationalization. Expanding on participants' desire for transparency, we provide design recommendations for building interactive and trustworthy VAA chatbots.2025JZJianlong Zhu et al.Conversational ChatbotsHuman-LLM CollaborationAI Ethics, Fairness & AccountabilityCUI
Exploring LLMs for Automated Generation and Adaptation of QuestionnairesEffective questionnaire design improves the validity of the results, but creating and adapting questionnaires across contexts is challenging due to resource constraints and limited expert access. Recently, the emergence of LLMs has led researchers to explore their potential in survey research. In this work, we focus on the suitability of LLMs in assisting the generation and adaptation of questionnaires. We introduce a novel pipeline that leverages LLMs to create new questionnaires, pretest with a target audience to determine potential issues and adapt existing standardized questionnaires for different contexts. We evaluated our pipeline for creation and adaptation through two studies on Prolific, involving 238 participants from the US and 118 participants from South Africa. Our findings show that participants found LLM-generated text clearer, LLM-pretested text more specific, and LLM-adapted questions slightly clearer and less biased than traditional ones. Our work opens new opportunities for LLM-driven questionnaire support in survey research.2025DADivya Mani Adhikari et al.Human-LLM CollaborationCUI
Towards Trustable Intelligent Clinical Decision Support Systems: A User Study with OphthalmologistsIntegrating Artificial Intelligence (AI) into Clinical Decision Support Systems (CDSS) presents significant opportunities for improving healthcare delivery, particularly in fields like ophthalmology. This paper explores the usability and trustworthiness of an AI-driven CDSS designed to assist ophthalmologists in treating diabetic retinopathy and age-related macular degeneration. Therefore, we created a CDSS and evaluated its impact on efficiency, informedness, and user experience through task-based semi-structured interviews and questionnaires with 11 ophthalmologists. The usability of the CDSS was rated highly, with a SUS of 81.75. Additionally, results show that participants felt like the CDSS would improve their efficiency and informedness with one major aspect being integrating Electronic Health Records (EHR) and Optical Coherence Tomography (OCT) data into a single interface. Additionally, we explored aspects of the trustworthiness of AI components, specifically OCT segmentation, treatment recommendation, and visual acuity forecasting. Through thematic analysis, we identified key factors influencing trustworthiness and clinical adoption. Results show that a larger degree of abstraction from input to output of a model correlates with decreased trust. From our findings, we propose two guidelines for designing trustworthy CDSS.2025RLRobert Andreas Leist et al.Explainable AI (XAI)Telemedicine & Remote Patient MonitoringIUI