Framing Water: Exploring Tensions between Social Norms and Environmental Sustainability through a Data Physicalization GameMountain areas are especially vulnerable to climate change. In recent years, intermittent droughts have forced many Alpine huts to close early or rely on cable cars or helicopters to import water. To raise awareness of water scarcity among hut visitors, we developed Framing Water, a reflective game based on the data physicalization of each visitor's water consumption during an overnight stay. The game requires players to select their most essential water-using activities without exceeding a fixed limit. Yet, it is designed not to provide a univocal answer about the right choices to make but to spark reflection and dialogue around trade-offs in daily practices. We evaluated it with 56 participants and found that water-use decisions are influenced by individual needs, values, and social norms. This work contributes to Sustainable HCI by showing how playful, data-driven artifacts can foster reflection and negotiation of resource use in response to climate challenges.2026EMEleonora Mencarini et al.Fondazione Bruno KesslerData PhysicalizationSustainable HCIEcological Design & Green ComputingCHI
Looking inside the VR Music Scene: Mapping Platforms, Events and PeopleMusic is increasingly performed and experienced in Social Virtual Reality (Social VR), from VRChat raves to high-production concerts on bespoke platforms. Yet Human-Computer-Interaction (HCI) research still focuses mainly on building new VR systems rather than examining the communities that already create and sustain these practices. We present a cultural mapping of the Social VR music scene based on 84 survey responses, 27 interviews, and 17 event observations with diverse stakeholders, including audience members, musicians, developers, platform owners, and event organisers. We found that the scene operates as a fragmented cross-platform ecosystem sustained by user-generated infrastructure and continuous community labour. The bottom-up organisation produces role fluidity with individuals dynamically shifting between roles as performers, world builders, organisers, and audience members. However, the openness that enables this creativity also creates tensions between expectations of free access and the financial and emotional labour required to keep events running. Taken together, our findings reveal the vibrant cultural practices that continue to flourish in Social VR, even as corporate narratives declare the ``metaverse'' dead.2026SPSophia Ppali et al.CYENS Centre of ExcellenceSocial & Collaborative VRImmersion & Presence ResearchMusic Composition & Sound Design ToolsCHI
Plug the Guitar In Before You Put the Headset On: Co-designing a Mixed Reality Collaborative Application with Professional MusiciansMixed reality (MR) technologies hold promise for distributed musical collaboration, yet the needs of professional musicians remain underexplored. We conducted a set of co-design workshops with 24 professional musicians, organized into six four-piece bands, to envision future MR systems for networked music performance. Using a three-phase process (hands-on MR experience, individual brainwriting, and collaborative consensus-building), we identified core requirements that challenge current MR paradigms. Professional musicians operate from an "Audio-First'' framework where sonic experience constitutes the primary reality, contradicting visual-first MR approaches. Five unanimous requirements emerged: audio-first workflow, support for non-verbal communication, personalized audio control, spatial layout customization, and plug-and-play simplicity. Other high-consensus themes included real-time voice communication and realistic avatar representation. Taken together, our findings reveal that professional creative collaboration demands fundamentally different design priorities than traditional office-centered collaborative MR applications, requiring systems that support established workflows and nonverbal communication channels while prioritizing workflow preservation over visual immersion.2026ABAlberto Boem et al.University of TrentoSocial & Collaborative VRMixed Reality WorkspacesMusic Composition & Sound Design ToolsCHI
Investigating the Use and Perception of Blocking Feature in Social Virtual Reality SpacesThis study explores the use of `blocking' as a safety tool in social VR platforms, a feature widely implemented to protect users from harassment. Despite its prevalence, little research investigates the reasons users block others and how they perceive the feature. To address this gap, we analyzed discussions from r/VRChat, r/AltspaceVR, r/HorizonWorld, and r/RecRoom, using thematic analysis. Our findings reveal that users block others for various reasons, extending beyond mere disruption avoidance to include personal preferences of avatars and social management such as blocking a friend for a disagreement. Furthermore, the study highlights the empowering roles and the limitations of blocking as a solution to harassment. The study contributes to understanding user behavior in social VR and offers insights for developing more effective safety tools in these immersive environments.2025QCQijia Chen et al.Harassment & Micro-AggressionsCSCW
“React”, “Command”, or “Instruct”? Teachers’ Mental Models on End-User DevelopmentThis paper presents findings from a thinking-aloud protocol exploring mental models in 28 elementary school math teachers during their initial attempt at composing and testing trigger-action rules for a smart tangible educational device. In the study, two sets of event-driven primitives were implemented in an End-User Development platform for guiding teachers with no programming experience in defining new functions of the device: "concrete", based on actual actions performed on the device, and "abstract", based on general definitions of events/actions. With a thematic analysis, we identified three different metaphors that drive participants' interaction with the device. We discuss how the metaphors influenced performance and how the order of exposition to the two primitive sets impacted their grasping of the trigger-action logic. Our findings suggest the importance of guiding teachers in assuming effective metaphors for performing End-User Development tasks, to empower them to adopt an active role toward digital devices in education.2025MAMargherita Andrao et al.University of Trento, Department of Psychology and Cognitive Science; Fondazione Bruno Kessler (FBK)Augmentative & Alternative Communication (AAC)Programming Education & Computational ThinkingIntelligent Tutoring Systems & Learning AnalyticsCHI
Unpacking Trust Dynamics in the LLM Supply Chain: An Empirical Exploration to Foster Trustworthy LLM Production & UseResearch on trust in AI is limited to several trustors (e.g., end-users) and trustees (especially AI systems), and empirical explorations remain in laboratory settings, overlooking factors that impact trust relations in the real world. Here, we broaden the scope of research by accounting for the supply chains that AI systems are part of. To this end, we present insights from an in-situ, empirical, study of LLM supply chains. We conducted interviews with 71 practitioners, and analyzed their (collaborative) practices using the lens of trust drawing from literature in organizational psychology. Our work reveals complex trust dynamics at the junctions of the chains, with interactions between diverse technical artifacts, individuals, or organizations. These junctions might constitute terrain for uncalibrated reliance when trustors lack supply chain knowledge or power dynamics are at play. Our findings bear implications for AI researchers and policymakers to promote AI governance that fosters calibrated trust.2025ABAgathe Balayn et al.Delft University of Technology, Software TechnologyAI Ethics, Fairness & AccountabilityAlgorithmic Transparency & AuditabilityCHI
Do Your Expectations Match? A Mixed-Methods Study on the Association Between a Robot's Voice and AppearanceBoth physical appearance and voice can elicit mental images of what someone and/or something should sound and look like. This is particularly relevant for human-robot interaction design and research since any voice can be added to a robot. Therefore, it is important to give robots voices that match users' expectations. In this paper, we examined the voice-appearance association by asking participants to match a robot image with a voice (Experiment 1, N = 24), and vice versa, a voice with a robot image (Experiment 2, N = 24), in two mixed-methods studies. We looked at participants' differences that could influence the voice-robot association (gender and nationality) and at voice and robot features that could influence participants' voice preferences (voice gender, pitch and robot's appearance). Results show that nationality influenced participants' association with a robot image after hearing its voice. Furthermore, a content analysis identified that when creating a voice mental image, participants looked at robots' gendered characteristics and height and they paid special attention to human-like and gender-specific cues in a voice when forming a mental image of a robot. Sociological differences also emerged, with Swedish participants suggesting the use of gender-neutral voices to avoid strengthening existing stereotypes, and Italians saying the opposite. Our work highlights the importance of individual differences in the robot voice-appearance association and the importance of involving the end user in designing the voice.2024MCMartina De Cet et al.Agent Personality & AnthropomorphismSocial Robot InteractionCUI
MR Object Identification and Interaction: Fusing Object Situation Information from Heterogeneous Sources"The increasing number of objects in ubiquitous computing environments creates a need for effective object detection and identification mechanisms that permit users to intuitively initiate interactions with these objects. While multiple approaches to such object detection -- including through visual object detection, fiducial markers, relative localization, or absolute spatial referencing -- are available, each of these suffers from drawbacks that limit their applicability. In this paper, we propose ODIF, an architecture that permits the fusion of object situation information from such heterogeneous sources and that remains vertically and horizontally modular to allow extending and upgrading systems that are constructed accordingly. We furthermore present BLEARVIS, a prototype system that builds on the proposed architecture and integrates computer-vision (CV) based object detection with radio-frequency (RF) angle of arrival (AoA) estimation to identify BLE-tagged objects. In our system, the front camera of a Mixed Reality (MR) head-mounted display (HMD) provides a live image stream to a vision-based object detection module, while an antenna array that is mounted on the HMD collects AoA information from ambient devices. In this way, BLEARVIS is able to differentiate between visually identical objects in the same environment and can provide an MR overlay of information (data and controls) that relates to them. We include experimental evaluations of both, the CV-based object detection and the RF-based AoA estimation, and discuss the applicability of the combined RF and CV pipelines in different ubiquitous computing scenarios. This research can form a starting point to spawn the integration of diverse object detection, identification, and interaction approaches that function across the electromagnetic spectrum, and beyond." https://doi.org/10.1145/36108792023JSJannis Strecker et al.Mixed Reality WorkspacesContext-Aware ComputingUbiComp
SensCon: Embedding Physiological Sensing into Virtual Reality ControllersVirtual reality experiences increasingly use physiological data for virtual environment adaptations to evaluate user experience and immersion. Previous research required complex medical-grade equipment to collect physiological data, limiting real-world applicability. To overcome this, we present SensCon for skin conductance and heart rate data acquisition. To identify the optimal sensor location in the controller, we conducted a first study investigating users' controller grasp behavior. In a second study, we evaluated the performance of SensCon against medical-grade devices in six scenarios regarding user experience and signal quality. Users subjectively preferred SensCon in terms of usability and user experience. Moreover, the signal quality evaluation showed satisfactory accuracy across static, dynamic, and cognitive scenarios. Therefore, SensCon reduces the complexity of capturing and adapting the environment via real-time physiological data. By open-sourcing SensCon, we enable researchers and practitioners to adapt their virtual reality environment effortlessly. Finally, we discuss possible use cases for virtual reality-embedded physiological sensing.2023FCFrancesco Chiossi et al.Immersion & Presence ResearchBiosensors & Physiological MonitoringContext-Aware ComputingMobileHCI
Complex Daily Activities, Country-Level Diversity, and Smartphone Sensing: A Study in Denmark, Italy, Mongolia, Paraguay, and UKSmartphones enable understanding human behavior with activity recognition to support people’s daily lives. Prior studies focused on using inertial sensors to detect simple activities (sitting, walking, running, etc.) and were mostly conducted in homogeneous populations within a country. However, people are more sedentary in the post-pandemic world with the prevalence of remote/hybrid work/study settings, making detecting simple activities less meaningful for context-aware applications. Hence, the understanding of (i) how multimodal smartphone sensors and machine learning models could be used to detect complex daily activities that can better inform about people’s daily lives, and (ii) how models generalize to unseen countries, is limited. We analyzed in-the-wild smartphone data and ~216K self-reports from 637 college students in five countries (Italy, Mongolia, UK, Denmark, Paraguay). Then, we defined a 12-class complex daily activity recognition task and evaluated the performance with different approaches. We found that even though the generic multi-country approach provided an AUROC of 0.70, the country-specific approach performed better with AUROC scores in [0.79-0.89]. We believe that research along the lines of diversity awareness is fundamental for advancing human behavior understanding through smartphones and machine learning, for more real-world utility across countries.2023KALakmal Meegahapola et al.École Polytechnique Fédérale de LausanneHuman Pose & Activity RecognitionContext-Aware ComputingCHI
On the state of reporting in crowdsourcing experiments and a checklist to aid current practicesCrowdsourcing is being increasingly adopted as a platform to run studies with human subjects. Running a crowdsourcing experiment involves several choices and strategies to successfully port an experimental design into an otherwise uncontrolled research environment, e.g., sampling crowd workers, mapping experimental conditions to micro-tasks, or ensure quality contributions. While several guidelines inform researchers in these choices, guidance of how and what to report from crowdsourcing experiments has been largely overlooked. If under-reported, implementation choices constitute variability sources that can affect the experiment's reproducibility and prevent a fair assessment of research outcomes. In this paper, we examine the current state of reporting of crowdsourcing experiments and offer guidance to address associated reporting issues. We start by identifying sensible implementation choices, relying on existing literature and interviews with experts, to then extensively analyze the reporting of 171 crowdsourcing experiments. Informed by this process, we propose a checklist for reporting crowdsourcing experiments.2021JRJorge Ramirez et al.Methods and Design ApproachesCSCW
Multimodal Emotion Recognition of Hand-Object InteractionIn this paper, we investigate whether information related to touches and rotations impressed to an object can be effectively used to classify the emotion of the agent manipulating it. We specifically focus on sequences of basic actions (e.g., grasping, rotating), which are constituents of daily interactions. We use the iCube, a 5 cm cube covered with tactile sensors and embedded with an accelometer, to collect a new dataset including 11 persons performing action sequences associated with 4 emotions: anger, sadness, excitement and gratitude. Next, we propose 17 high-level hand-crafted features based on the tactile and kinematics data derived from the iCube. Twelve of these features vary significantly as a function of the emotional context in which the action sequence was performed.In particular, a larger surface of the object is engaged in physical contact for anger and excitement, than for sadness. Furthermore, the average duration of interactions labelled as sad, is longer than for the remaining 3 emotions. More rotations are performed for anger and excitement than for sadness and gratitude. The accuracy of a classification experiment in the case of four emotions reaches 0.75. This result shows that the emotion recognition during hand-object interactions is possible and it may foster development of new intelligent user interfaces.2021RNRadoslaw Niewiadomski et al.In-Vehicle Haptic, Audio & Multimodal FeedbackHuman Pose & Activity RecognitionIUI
Relationship Between Visual Complexity and Aesthetics of WebpagesSubstantial HCI research investigated the relationship between webpage complexity and aesthetics, but without a definitive conclusion. Some research showed an inverse linear correlation, some other showed an inverted u-shaped curve, while the rest showed no relationship at all. Such a lack of clarity complicates hypothesis formulation and result interpretation for future research, and lowers the reliability and generalizability of potential advice for Web design practice. We re-collected complexity and aesthetics ratings for five datasets previously used in webpage aesthetics and complexity research. The results were mixed, but suggested an inverse linear relationship with a weaker u-shaped sub-component. A subsequent visual inspection of revealed several confounding factors that may have led to the mixed results, including some webpages looking broken or archaic. The second data collection showed that accounting for these factors generally eliminates the u-shaped tendency of the complexity-aesthetics relationship, at least, for a relatively homogeneous sample of English-speaking participants.2020AMAliaksei Miniukovich et al.University of TrentoInteractive Data VisualizationUser Research Methods (Interviews, Surveys, Observation)CHI
Guideline-Based Evaluation of Web ReadabilityEffortless reading remains an issue for many Web users, despite a large number of readability guidelines available to designers. This paper presents a study of manual and automatic use of 39 readability guidelines in webpage evaluation. The study collected the ground-truth readability for a set of 50 webpages using eye-tracking with average and dyslexic readers (n = 79). It then matched the ground truth against human-based (n = 35) and automatic evaluations. The results validated 22 guidelines as being connected to readability. The comparison between human-based and automatic results also revealed a complex framework: algorithms were better or as good as human experts at evaluating webpages on specific guidelines – particularly those about low-level features of webpage legibility and text formatting. However, multiple guidelines still required a human judgment related to understanding and interpreting webpage content. These results contribute a guideline categorization laying the ground for future design evaluation methods.2019AMAliaksei Miniukovich et al.University of TrentoExplainable AI (XAI)Recommender System UXCognitive Impairment & Neurodiversity (Autism, ADHD, Dyslexia)CHI
FontMatcher: Font Image Paring for Harmonious Digital Graphic DesignOne of the important aspects in graphic design is choosing the font of the caption that matches aesthetically the associated image. To obtain a good match, users would exhaustively examine a long font list requiring them a substantial effort. This paper presents FontMatcher, which supports users to design digital graphic works harmoniously pairing fonts with an image. The system provides three features, recommendation, explaination and feedback. If a warm feeling image is given as input, the system recommends warm feeling fonts, and then explains what is the distinguishing features of the recommendation, e.g. a cursive shape. Users can also provide feedback to find fonts which correspond to their intention. The evaluation results show that the recommended fonts scored better than selected fonts by novices and provides competing results with the ones chosen by experienced graphic designers. The explanations help increasing the reliability of the recommended results.2018SCSaemi Choi et al.Generative AI (Text, Image, Music, Video)Visualization Perception & CognitionGraphic Design & Typography ToolsIUI
Combining Crowd and Machines for Multi-predicate Item ScreeningThis paper discusses how crowd and machine classifiers can be efficiently combined to screen items that satisfy a set of predicates. We show that this is a recurring problem in many domains, present machine-human (hybrid) algorithms that screen items efficiently and estimate the gain over human-only or machine-only screening in terms of performance and cost. We further show how, given a new classification problem and a set of classifiers of unknown accuracy for the problem at hand, we can identify how to manage the cost-accuracy trade off by progressively determining if we should spend budget to obtain test data (to assess the accuracy of the given classifiers), or to train an ensemble of classifiers, or whether we should leverage the existing machine classifiers with the crowd, and in this case how to efficiently combine them based on their estimated characteristics to obtain the classification. We demonstrate that the techniques we propose obtain significant cost/accuracy improvements with respect to the leading classification algorithms.2018EKEvgeny Krivosheev et al.Classification and LabelsCSCW