Enhancing Generative AI Image Refinement with Scribbles and Annotations: A Comparative Study of Multimodal PromptsGenerative AI (GenAI) image tools are increasingly used in design practice, enabling rapid ideation but offering limited support for refinement tasks such as adjusting layout, scale, or visual attributes. While text prompts and inpainting allow localized edits, they often remain inefficient or ambiguous for precise, in-context, and iterative refinement---motivating the exploration of alternative methods. This work examines how pen-based scribbles and annotations can enhance GenAI image refinement. A formative study with seven professional designers informed a prototype supporting three input modalities: text-only, visual-only, and combined prompting. A within-subjects study with 30 designers and design students compared these modalities across closed- and open-ended tasks, evaluating expressiveness, efficiency, workload, user experience, iteration, and multimodal strategies. Visual prompts improved clarity and speed for spatial edits while reducing workload, whereas text remained effective for semantic and global changes. The combined modality received the highest overall ratings, enabling complementary use, balancing spatial precision with semantic detail, and supporting smoother iteration. Task-specific preferences also emerged: adding new objects often required both modalities, while moving or modifying elements was typically handled through visual input. This work contributes (1) an empirical comparison of multimodal prompting for GenAI refinement, (2) a prototype integrating scribbles and annotations, and (3) insights into designers' multimodal strategies to inform future GenAI interfaces that better support refinement in GenAI-supported design workflows.2026HPHyerim Park et al.BMW GroupGenerative AI (Text, Image, Music, Video)Creative Collaboration & Feedback SystemsGraphic Design & Typography ToolsIUI
The Role of Personality of Conversational Virtual Avatars on Proxemic Behaviour during Indoor NavigationAs LLM-based Conversational Avatars increasingly act as collaborators in hybrid indoor navigation, understanding how their personality traits influence human-avatar proxemic behavior is becoming crucial. Prior work has largely examined personality effects in static or one-sided interactions such as sitting, standing, or approaching. However, there is a gap in research on how avatar personality and motion-related factors (e.g., walking speed) shape proxemics when both the human and avatar are in motion. To address this, we developed an AR indoor navigation system featuring a Conversational Virtual Avatar (CVA) with three distinct personalities: Dominant, Warm, and Conscientious. The CVA guides users to destinations within the environment. In a between-subjects study ($N$=27), we found statistically significant effects of avatar personality and walking speed on proxemic behavior. Our work contributes to a broader understanding of the role of personality and walking speed of a CVA on human-avatar proxemic behaviour during navigation.2026RBRishab Bhattacharyya et al.TU BerlinIdentity & Avatars in XRAR Navigation & Context AwarenessAgent Personality & AnthropomorphismCHI
With, not For: Co-Designing a Patient-Facing AI Companion Concept for the Emergency Department Waiting AreaEmergency departments (EDs) often experience overcrowding, staff shortages, and long waiting times, which put a strain on clinicians and negatively impact patients' experiences. Although AI is commonly proposed to optimize clinical workflows, the perspectives of patients are often overlooked in AI design. We report on a four-phase study at a university hospital combining preparatory fieldwork, co-creation workshops, design phase, and storyboard-guided interviews. Our findings reveal a misalignment: clinicians framed the contribution of AI around triage efficiency, while patients emphasized reassurance, empathy, and guidance. Addressing both needs, we developed and evaluated a concept of an ephemeral AI companion for the ED waiting area, designed to provide orientation, support reflection, and prepare patients for consultations without substituting human contact. We contribute: empirical evidence of patients’ needs, concerns, and expectations for AI support, design principles for an ephemeral AI companion concept, and findings from a study conducted with patients during their ED waits.2026JKJacobe Klein et al.Human-Centered-Computing, Freie Universität BerlinAI-Assisted Decision-Making & AutomationMental Health Apps & Online Support CommunitiesTelemedicine & Remote Patient MonitoringCHI
The (Anti-)Affordance Problem: Effects of Physical Context on Collaborator Placement in Augmented Reality MeetingsWhile Augmented Reality (AR) promises to transform remote collaboration, many aspects remain underexplored, particularly where to place remote avatars in messy, everyday environments. Two mixed-methods within-subjects studies examined avatar placement preferences during cooperative (brainstorming) and competitive (negotiation) tasks between participant pairs, focusing on the influence of physical objects (chairs, box, tree) on user preferences. Results showed a strong preference for frontal or slightly off-centre avatar placements, independent of task type. Participants preferred avatar placements that mirrored real-life behaviour, with chairs inviting placements and the tree deterring them. Notably, the large and visually simple box elicited mixed reactions, being viewed alternately as an obstacle to avoid when placing avatars or as an inviting physical anchor for them, despite causing a clear physicality conflict. We term this the "(Anti-)Affordance Problem", highlighting the complexity of avatar placement within physical contexts, and the necessity for AR collaboration platforms to respond to real-world constraints, offering flexibility in avatar placements to accommodate diverse user preferences.2026DDDiego Drago et al.University of GlasgowAR Navigation & Context AwarenessImmersion & Presence ResearchMixed Reality WorkspacesCHI
Anticipating Physical Processes in VR: Environment Type and Scale Alter Temporal ExpectationsAccurate temporal expectations support interaction in virtual reality (VR), yet it remains unclear whether the internal models that guide such expectations in the real world transfer unchanged to immersive VR. We report two experiments examining expected durations of gravity-driven motion across real and virtual environments. In Experiment 1, participants imagined a ball rolling down ramps in a physical lab, a 1:1 VR replica, and an up-scaled VR room and produced the time the imagined process would take. Results revealed systematic distortions: durations were underestimated in VR relative to the physical lab, and larger virtual spaces elicited longer durations. Experiment 2 assessed whether participants incorporated gravity laws into their simulations. Although gravitational acceleration was consistently underestimated, it was incorporated in both real and virtual environments. Our findings show that VR and its spatial scale bias temporal expectations, with implications for the design of temporally coherent and physically plausible VR experiences.2026MRMartin Riemer et al.Technical University BerlinImmersion & Presence ResearchAR Navigation & Context AwarenessCHI
The Role of Expertise in Effectively Moderating Harmful Social Media ContentSocial media platforms played a significant role in spreading genocidal content in the 2020-2022 Tigray war, where the deadliest genocide of the 21st century was committed. While linguistic expertise is clearly needed to adequately moderate such content, we ask: What additional expertise is needed? Why and to what extent do experts disagree on what constitutes harmful content, and what is the best way to resolve these disagreements? What do social media platforms do instead? We examine these questions through a 4 month study with 7 experts labeling 340 X (formerly Twitter) posts, and by interviewing 15 commercial content moderators. We find in-depth cultural knowledge and dialects to be most important for accurate hate speech annotation – knowledge which social media platforms do not prioritize. Even amongst experts, disagreements are high (71%), dropping to 40% after deliberation meetings. Based on these results, we present 7 recommendations to improve hate speech annotation and moderation practices.2025NANuredin Ali Abdelkadir et al.University of Minnesota, Computer Science and Engineering; The Distributed AI Research InstituteAI Ethics, Fairness & AccountabilityContent Moderation & Platform GovernanceMisinformation & Fact-CheckingCHI
Lost in Moderation: How Commercial Content Moderation APIs Over- and Under-Moderate Group-Targeted Hate Speech and Linguistic VariationsCommercial content moderation APIs are marketed as scalable solutions to combat online hate speech. However, the reliance on these APIs risks both silencing legitimate speech, called over-moderation, and failing to protect online platforms from harmful speech, known as under-moderation. To assess such risks, this paper introduces a framework for auditing black-box NLP systems. Using the framework, we systematically evaluate five widely used commercial content moderation APIs. Analyzing five million queries based on four datasets, we find that APIs frequently rely on group identity terms, such as ``black'', to predict hate speech. While OpenAI's and Amazon's services perform slightly better, all providers under-moderate implicit hate speech, which uses codified messages, especially against LGBTQIA+ individuals. Simultaneously, they over-moderate counter-speech, reclaimed slurs and content related to Black, LGBTQIA+, Jewish, and Muslim people. We recommend that API providers offer better guidance on API implementation and threshold setting and more transparency on their APIs' limitations. \noindent \textit{\textbf{Warning}: This paper contains offensive and hateful terms and concepts. We have chosen to reproduce these terms for reasons of transparency.}2025DHDavid Hartmann et al.Weizenbaum Institute Berlin, Data, Algorithmic Systems and Ethics; Technical University Berlin, Internet and SocietyAI Ethics, Fairness & AccountabilityAlgorithmic Transparency & AuditabilityDark Patterns RecognitionCHI
The Making of Performative Accuracy in AI Training: Precision Labor and Its ConsequencesAccuracy and precision are central values in the AI communities and the technology sector. This paper provides empirical evidence on the construction and organizational management of technical accuracy, demonstrating how technology companies' preoccupation with such values leads to harm. Drawing on nine months of multi-sited ethnographic fieldwork in China, we document how AI trainers' everyday work practices, challenges, and harms stem from clients' demands for high levels of technical accuracy. We introduce the concept of precision labor to unpack the labor dimension of constructing and performing accuracy in AI training. This concept highlights the hidden and excessive labor required to reconcile the ambiguity and uncertainty involved in this process. We argue that precision labor offers a new lens to illuminate three critical aspects of AI training: 1) the negative health and financial impacts of hidden and excessive labor on AI workers; 2) emerging harms, including workers' subordinate roles to machines and financial precarity; and 3) a conceptual contribution to contexts beyond AI training. This contribution re-centers arbitrariness in technical production, highlights the excessive demands of precision labor, and examines the legitimization of labor and harm. Our study also contributes to existing scholarship on the prevailing values and invisible labor in AI production, underscoring accuracy as performative rather than self-evident and unambiguous. A precision labor lens challenges the legitimacy and sustainability of relentlessly pursuing technical accuracy, raising new questions about its consequences and ethical implications. We conclude by proposing recommendations and alternative approaches to enhance worker agency and well-being.2025BZBen Zefeng Zhang et al.University of MichiganAI-Assisted Decision-Making & AutomationPrivacy by Design & User ControlTechnology Ethics & Critical HCICHI
Who is Trusted for a Second Opinion? Comparing Collective Advice from a Medical AI and Physicians in Biopsy Decisions After Mammography ScreeningArtificial Intelligence (AI) is increasingly integrated into clinical practice, but its influence on patient decision-making, particularly when AI and physicians disagree, remains unclear. To examine collective advice, we investigated a breast cancer screening scenario using (1) a qualitative interview study (N=9) and (2) a quantitative experiment (N=339) where participants received either consistent or conflicting biopsy recommendations. Qualitative findings include the need for empathetic care, the importance of patient autonomy, and a desire for a four-eyes principle. Quantitative findings accordingly show that patients generally trust physicians more than AI but still tend to follow AI recommendations due to risk aversion. When both advised a biopsy, 99% adhered; if both advised against it, 25% still proceeded. In conflicting scenarios, 97% followed the physician’s advice, whereas 66% followed the AI if it recommended the biopsy. These results underscore the need for careful interaction design of collective healthcare advice to prevent unnecessary healthcare procedures.2025HDHenrik Detjen et al.Fraunhofer Institute for Digital Medicine MEVISAI-Assisted Decision-Making & AutomationAI Ethics, Fairness & AccountabilityPrivacy by Design & User ControlCHI
PASTEL: Privacy-Preserving Federated Learning in Edge ComputingElhattab 等人提出 PASTEL 隐私保护联邦学习方案，应用于边缘计算环境。2024FEFatima Elhattab et al.Generative AI (Text, Image, Music, Video)Privacy by Design & User ControlUbiComp
Predicting the Limits: Tailoring Unnoticeable Hand Redirection Offsets in Virtual Reality to Individuals’ Perceptual BoundariesMany illusion and interaction techniques in Virtual Reality (VR) rely on Hand Redirection (HR), which has proved to be effective as long as the introduced offsets between the position of the real and virtual hand do not noticeably disturb the user experience. Yet calibrating HR offsets is a tedious and time-consuming process involving psychophysical experimentation, and the resulting thresholds are known to be affected by many variables---limiting HR's practical utility. As a result, there is a clear need for alternative methods that allow tailoring HR to the perceptual boundaries of individual users. We conducted an experiment with 18 participants combining movement, eye gaze and EEG data to detect HR offsets Below, At, and Above individuals' detection thresholds. Our results suggest that we can distinguish HR At and Above from no HR. Our exploration provides a promising new direction with potentially strong implications for the broad field of VR illusions.2024MFMartin Feick et al.Full-Body Interaction & Embodied InputEye Tracking & Gaze InteractionBrain-Computer Interface (BCI) & NeurofeedbackUIST
Ghost Readers of the Nile: Decrypting Password Sharing Habits in Chatting Applications among Egyptian WomenPassword sharing is a convenient means to access shared resources, save on subscription costs, provide emergency access, and avoid forgetting vital account details. However, it also raises significant privacy concerns, especially in digital communication contexts where content may be inadvertently exposed to unintended recipients. In this paper, we investigate this duality, using a survey of 86 Egyptian women to understand their sharing behavior and the design and evaluation of a chat application used by 60 participants. This application issues warnings based on content sensitivity, leading to increased user awareness about privacy risks. Our findings indicate that, while many participants initially shared passwords, they were surprised to discover others doing the same. Furthermore, our application effectively reduced password sharing, reflecting improved awareness of associated risks. This research acknowledges the cultural aspects of password sharing while striving to enhance the experience, enabling participants to make informed choices that enhance their information control.2024MSMennatallah Saleh et al.Privacy by Design & User ControlPasswords & AuthenticationMobileHCI
Responding to Generative AI Technologies with Research-through-Design: The Ryelands AI Lab as an Exploratory StudyGenerative AI technologies demand new practical and critical competencies, which call on design to respond to and foster these. We present an exploratory study guided by Research-through-Design, in which we partnered with a primary school to develop a constructionist curriculum centered on students interacting with a generative AI technology. We provide a detailed account of the design of and outputs from the curriculum and learning materials, finding centrally that the reflexive and prolonged 'hands-on' approach led to a co-development of students' practical and critical competencies. From the study, we contribute guidance for designing constructionist approaches to generative AI technology education; further arguing to do so with 'critical responsivity.' We then discuss how HCI researchers may leverage constructionist strategies in designing interactions with generative AI technologies; and suggest that Research-through-Design can play an important role as a 'rapid response methodology' capable of reacting to fast-evolving, disruptive technologies such as generative AI.2024JBJesse Josua Benjamin et al.Generative AI (Text, Image, Music, Video)Programming Education & Computational ThinkingParticipatory DesignDIS
Explaining It Your Way - Findings from a Co-Creative Design Workshop on Designing XAI Applications with AI End-Users from the Public SectorHuman-Centered AI prioritizes end-users' needs like transparency and usability. This is vital for applications that affect people's everyday lives, such as social assessment tasks in the public sector. This paper discusses our pioneering effort to involve public sector AI users in XAI application design through a co-creative workshop with unemployment consultants from Estonia. The workshop's objectives were identifying user needs and creating novel XAI interfaces for the used AI system. As a result of our user-centered design approach, consultants were able to develop AI interface prototypes that would support them in creating success stories for their clients by getting detailed feedback and suggestions. We present a discussion on the value of co-creative design methods with end-users working in the public sector to improve AI application design and provide a summary of recommendations for practitioners and researchers working on AI systems in the public sector.2024KWKatharina Weitz et al.University of AugsburgExplainable AI (XAI)Participatory DesignCHI
Experiencing Dynamic Weight Changes in Virtual Reality Through Pseudo-Haptics and Vibrotactile FeedbackVirtual reality (VR) objects react dynamically to users' touch interactions in real-time. However, experiencing changes in weight through the haptic sense remains challenging with consumer VR controllers due to their limited vibrotactile feedback. While prior works successfully applied pseudo-haptics to perceive absolute weight by manipulating the control-display (C/D) ratio, we continuously adjusted the C/D ratio to mimic weight changes. Vibrotactile feedback additionally emphasises the modulation in the virtual object's physicality. In a study (N=18), we compared our multimodal technique with pseudo-haptics alone and a baseline condition to assess participants' experiences of weight changes. Our findings demonstrate that participants perceived varying degrees of weight change when the C/D ratio was adjusted, validating its effectiveness for simulating dynamic weight in VR. However, the additional vibrotactile feedback did not improve weight change perception. This work extends the understanding of designing haptic experiences for lightweight VR systems by leveraging perceptual mechanisms.2024CSCarolin Stellmacher et al.University of BremenVibrotactile Feedback & Skin StimulationForce Feedback & Pseudo-Haptic WeightCHI
Imagination vs. Reality: Investigating the Acceptance and Preferred Anthropomorphism in Service HRI While the use of robots in public spaces is increasing, still few studies explore the resulting everyday human-robot interactions (HRI). The present study sought to bridge the disparity between real-world interactions and the frequently examined hypothetical interactions. To do so, we investigate the imagined and actual interaction with an ice cream serving robot. In two studies and an exploratory study comparison, we investigated user acceptance and preference for the degree of anthropomorphic appearance. Although a typical human service task was taken over by a robot, an industrial robot was preferred according to participant’s ratings in both studies. Moreover, both studies demonstrated that robot enthusiasm significantly relates to participants' acceptance of the robot for the task. Besides these commonalities, the results showed also that while humans were preferred over robots in the imagined setting, no clear preference was found in the real-life setting. Additional analyses compared the free text answers of the two studies and provided insights into participants' general attitudes toward robots in the workforce. In line with the higher preferences for humans over robots in the imagined setting, considerably more participants mentioned a better customer experience with humans as important in the imagined study compared to the participants who actually interacted with the robot. The studies strikingly demonstrated that imaginary settings yield similar outcomes to those where participants physically engage with the robot in certain aspects, such as their preference for anthropomorphism. However, this phenomenon does not appear to hold for other facets, such as their favored service agent.2024KWKatharina Wzietek et al.Agent Personality & AnthropomorphismAI Ethics, Fairness & AccountabilitySocial Robot InteractionHRI
Exploring Millions of User Interactions with ICEBOAT: Big Data Analytics for Automotive User InterfacesUser Experience (UX) professionals need to be able to analyze large amounts of usage data on their own to make evidence-based design decisions. However, the design process for In-Vehicle Information Systems (IVISs) lacks data-driven support and effective tools for visualizing and analyzing user interaction data. Therefore, we propose ICEBOAT, an interactive visualization tool tailored to the needs of automotive UX experts to effectively and efficiently evaluate driver interactions with IVIS. ICEBOAT visualizes telematics data collected from production line vehicles, allowing UX experts to perform task-specific analyses. Following a mixed methods User-Centered Design (UCD) approach, we conducted an interview study (N=4) to extract the domain specific information and interaction needs of automotive UX experts and used a co-design approach (N=4) to develop an interactive analysis tool. Our evaluation (N=12) shows that ICEBOAT enables UX experts to efficiently generate knowledge that facilitates data-driven design decisions.2023PEPatrick Ebel et al.Head-Up Display (HUD) & Advanced Driver Assistance Systems (ADAS)Interactive Data VisualizationAutoUI
The God-I-Box: Iteratively Provotyping Technology-Mediated Worship ServicesThe COVID-19 pandemic accelerated the development of alternative formats for religious rituals, such as Protestant online worship services. However, current design approaches focus on problem-solving, and the resulting online solutions merely imitate the offline status quo. To overcome these limitations, we suggest adopting a provotype approach that allows for a more holistic, open-ended dialogue with those affected. We iteratively developed a first provotype in response to tensions found in observation-based field research, aiming to test whether and how it can trigger productive impulses for exploring future technology-mediated worship services based on existing experiences and perspectives. The resulting God-I-Box exaggerates individuality and allows congregants to act almost like liturgists. An analysis of congregants' and pastors' (online) first encounters with the God-I-Box revealed three reaction modes: spontaneous emotions, reflective coping, and exploratory imagination. We conclude with reflections and recommendations for provocative research and design in this context and beyond.2023SWSara Wolf et al.Mental Health Apps & Online Support CommunitiesDesign FictionDIS
Towards an Implicit Metric of Sensory-Motor Accuracy: Brain Responses to Auditory Prediction Errors in PianistsDuring listening to music, the brain expects specific acoustic events based on learned musical rules. During music performance expectancy is additionally created based on motor action by linking keypresses to their sounds. We investigated EEG (Electroencephalography) signals to auditory expectancy violations in piano performance and perception. In our study, pianists experienced manipulations of different acoustic features, such as pitch and loudness, during playing and listening to piano sequences. We found that manipulations during performance elicited deflections with stronger amplitudes compared to manipulations during perception indicating that the action of producing sounds strengthens auditory expectancy. Loudness manipulations, violating musical regularity, elicited deflections with smaller latencies compared to pitch manipulations, which violate harmonic expectancy, suggesting that the brain processes expectancy violations of distinct acoustic features in a different way. These EEG signatures may prove useful for applications in intelligent music interfaces by providing information about sensory-motor accuracy.2023EPElisabeth Pangratz et al.Brain-Computer Interface (BCI) & NeurofeedbackBiosensors & Physiological MonitoringC&C
Mobilizing Social Media Data: Reflections of a Researcher Mediating between Data and OrganizationThis paper examines the practices involved in mobilizing social media data from their site of production to the institutional context of non-profit organizations. We report on nine months of fieldwork with a transnational and intergovernmental organization using social media data to understand the role of grassroots initiatives in Mexico, in the unique context of the COVID-19 pandemic. We show how different stakeholders negotiate the definition of problems to be addressed with social media data, the collective creation of ground-truth, and the limitations involved in the process of extracting value from data. The meanings of social media data are not defined in advance; instead, they are contingent on the practices and needs of the organization that seeks to extract insights from the analysis. We conclude with a list of reflections and questions for researchers who mediate in the mobilization of social media data into non-profit organizations to inform humanitarian action.2023AGAdriana Alvarado Garcia et al.IBM Research, Georgia Institute of TechnologySocial Platform Design & User BehaviorCommunity Engagement & Civic TechnologyCHI