Somatic Drawing Tool: 3D Body Sheet as Material for Articulating Synaesthetic ExperiencesWe present a participatory co-design cycle that studied synaesthesia, a cross-sensory phenomenon where one stimulus automatically triggers another perception. We worked with a community of synaesthetes to develop and evaluate an interactive technology to depict and communicate synaesthetic experience. By working with qualitative methods to understand experience, we accommodate the highly individual nature and support otherwise limited sensory vocabulary and lack of shared understanding of the condition. The first study involved a series of participatory workshops that explored text, creative tools, interactive technologies, and sensory materials in conveying the synaesthetic experience. Insights from these workshops informed the development of the Somatic Drawing Tool that allows users to depict their experiences. Within the tool, we deployed the 3D Body Sheet, an immersive extension of techniques drawn from soma design. We engaged users in an initial evaluation of the system and took on board their feedback in a subsequent cycle of iterative development. This resulted in a bespoke system that could be personalised to diverse types of individual synaesthetic experience. This enables nuanced articulation, providing a supportive environment for externalising and reflecting sensations.2026MWMamoru Watanabe et al.University of BristolDigital Art Installations & Interactive PerformanceTangible User Interface DesignPhysical-Digital Hybrid InteractionCHI
Feeling the Flavour: Exploring Children's Touch–Taste Correspondences and Willingness to Try Unknown Foods for Child–Food Interaction DesignHow can we leverage taste expectations to create novel food-based experiences for children? Eating is an embodied process that jointly engages with multiple senses. Cross-sensory correspondences may offer educational and recreational opportunities to design interactive applications that encourage diversifying encounters with food. We present a study with 64 children (ages 10–11) who explored eight textured materials hidden inside mystery "food'' boxes and reported both their expected tastes and willingness to eat. Our findings provide evidence of touch–taste cross-sensory correspondences in children — sweetness with weak-hard-brittle and strong-soft-brittle materials, and saltiness with a weak-soft brittle material — and how these mappings influenced children's openness to unknown foods. These results provide empirical grounding for cross-sensory interaction design with children, demonstrating how texture could scaffold curiosity and learning. We outline design implications for cross-sensory food interfaces, non-edible public exhibits, and playful educational technologies that could broaden eating experiences and enable new forms of virtual food interaction.2026PLPriscilla Y. Lo et al.University of BristolOlfactory Display & Smell InteractionMultisensory Fusion ExperienceFood Culture & Food InteractionCHI
triMorph: Bridging Shape-Change and Cross-Sensory Correspondences for Haptic InteractionCross-sensory correspondences provide opportunities for designing rich sensory HCI, with prior work showing that features such as roundness and sharpness are systematically linked to language, color, sound, and emotion. Yet two challenges remain: few technologies can dynamically transition between these features, and little is known about the thresholds at which a form is judged as sufficiently rounded or spiky to realize these cross-sensory effects. We present triMorph, a pneumatic shape-changing interface capable of smoothly morphing between spiky, flat, and rounded configurations. In a psychophysical study with 30 participants, we quantified perceptual accuracy and precision in mapping triMorph shapes to visual-linguistic categories and examined shape–color and shape–emotion correspondences. Results reveal threshold values for reliable categorization, with rounded shapes linked to pleasant emotions and lighter colors, and spiky shapes to arousal and darker tones. Our findings provide empirical foundations and design guidelines for grounding shape-changing artifacts more firmly in cross-sensory cognition.2026ZFZhuzhi FAN et al.University of BristolShape-Changing Interfaces & Soft Robotic MaterialsMultisensory Fusion ExperienceAffective Feedback & Emotion Regulation InterfacesCHI
Beyond Accuracy: Auditing Allocative Harms in Facial-Gesture Recognition for People with Motor ImpairmentsCamera-based facial-gesture interfaces offer hands-free access for people with motor impairments (PwM), yet most recognition models are trained on able-bodied data and implicitly assume normative motor control and proprioception. We conducted a mixed-methods empirical study of 37 above-neck gestures performed by 11 PwM and 11 non-impaired participants. Results reveal systematic mismatches between user intention and model recognition in the PwM group, stemming from diverse patterns of body perception and control and leading to allocative harms. These mismatches concentrated in low-amplitude, asymmetric, and directional gestures. Building on these findings, we introduce FairGesture, a diagnostic auditing method for quantifying and interpreting such mismatches. FairGesture combines (1) the Perception Gap metric, (2) trajectory-based motion analysis, and (3) an analysis of user sensorimotor feedback, exploring the reasons behind these mismatches. The work reframes accuracy in gesture recognition as a problem of sensorimotor alignment, advancing user-centred evaluation and inclusive model design.2026SZSiyu Zhang et al.University of BristolMotor Impairment Assistive Input TechnologiesHand Gesture RecognitionHuman Pose & Activity RecognitionCHI
Let's Make a Community [of Practice]: Using Community-Based Participatory Design to Support InterdependenceIn line with the shift toward dementia-friendly communities, HCI research is increasingly exploring holistic ways to support people living with dementia as active community members. However, less is known about how existing community networks influence technology design and effectiveness. This paper analyses Community-Based Participatory Design (CBPD) workshops conducted with individuals living with dementia, their spouses, and program coordinators through Wenger's Communities of Practice Framework. We demonstrate how participants' interactions created a community of practice through their: engagement in creating mutual meaning, alignment around a common purpose, and imagination in envisioning new possibilities for inclusion. Our findings highlight the agency of participants as they worked to create inclusive experiences for themselves and others. Our findings additionally demonstrated how community boundaries create systemic barriers that lead to non-participation. We argue Wenger’s framework offers a roadmap for designing technologies that foster person-to-person interdependence, helping to build more genuinely inclusive communities.2026ECElaine Czech et al.University of BristolParticipatory DesignElderly Care & Dementia SupportCHI
Rough Meanings: Cross-sensory correspondences linking surface textures with sound symbolism, colours, and emotionsSurface textures play a critical role in shaping interaction with tangible and multisensory technologies, yet little is known about how their microstructural features influence cognitive and affective responses - factors central to interface design. We investigated this through cross-sensory correspondences of textures systematically varying in roundness and size. Thirty participants explored 3D-printed textures under visuo-tactile and tactile-only conditions, rating them on visuo-linguistic association, roughness, colour, and emotion. Rounded textures were often linked with Bouba and pleasantness, whereas pointed textures were associated with Kiki, higher arousal, and warmer colours. Visual access also influenced exploratory behaviour, reflected in applied normal force \del{and duration}. These findings demonstrate how microstructural tactile cues shape cross-sensory and affective associations. We propose cross-sensory correspondences as a methodological framework for designing microstructural features of surfaces textures, which could open up new design opportunities for pseudo-haptic feedback, texture-rich tangible interfaces, and coherent multisensory experiences in VR/AR.2026MLMin Susan Li et al.University of BristolShape-Changing Interfaces & Soft Robotic MaterialsPhysical-Digital Hybrid InteractionMultisensory Fusion ExperienceCHI
Touching Emotions, Smelling Shapes: Exploring Tactile, Olfactory and Emotional Cross-sensory Correspondences in Preschool-aged Children Multisensory design principles are increasingly seen as central in the design of technologies for learning, communication, and affective regulation. Multisensory integration, the process by which we combine information from different senses, develops rapidly in the preschool years, shaping processes of perception and sense-making. In particular, this may impact cross-sensory correspondence, how perceptions in different sensory modalities influence one another, a key issue in multisensory design. To date, little is known about cross-sensory correspondences in preschool-aged children (2-4 years). We present a study with 26 preschoolers examining smell–touch–emotion correspondences through playful cross-sensory tasks. We found significant correspondences between sensory modalities and between sensory modalities and affective judgements. Further analysis revealed association strategies underpinning these mappings. We contribute empirical insights into cross-sensory correspondences in early childhood, design guidelines for creating sensory interfaces that align with how preschoolers integrate sensory input, and a replicable method for probing cross-sensory cognition in this age group.2026TRTegan Joy Roberts-Morgan et al.University of BristolOlfactory Display & Smell InteractionMultisensory Fusion ExperienceTangible Interaction in EducationCHI
"Put Your Hands Up": How Joint Attention Is Initiated Between Blind Children And Their Sighted PeersInitiating joint attention (JA) is a fundamental first step in social interactions. In sighted individuals, it relies predominantly on visual cues, such as gaze and hand gestures. These features can reduce opportunities for blind and visually impaired (BVI) and sighted people to interact. Understanding the strategies to navigate these challenges is necessary to develop technology that can facilitate more inclusive JA. To address this, we conducted a longitudinal case study of five children with mixed visual abilities engaging in activities rich with JA opportunities. In a teacher-led classroom, the children experimented with the use of an AI-powered headset designed to support BVI people in social situations. Interaction analysis established that situational complexity affects the children’s responses to initiation attempts. Furthermore, the headset adds to this complexity, affecting the frequency and reactions to attempts to initiate JA. The findings informed the creation of a JA initiation framework and suggestions for future design.2025KJKatherine Jones et al.University of BristolCognitive Impairment & Neurodiversity (Autism, ADHD, Dyslexia)Augmentative & Alternative Communication (AAC)Universal & Inclusive DesignCHI
"It Helps Us Express Our Feelings Without Having To Say Anything": Exploring Accompanying Social Play Things Designed With and For Neurodiverse Groups of ChildrenSocial play is crucial for children's well-being and development. However, many social play technologies fail to address the specific characteristics and needs of neurodiverse play and often overlook divergent play styles. To address this, we first conducted a co-design study with a neurodiverse group of 7 children (Age 7-8) and, based on insights from these sessions, then developed a prototype, ChromaConnect, that allowed children to express their play style to one another during play. To evaluate ChromaConnect's ability to support neurodiverse social play in different contexts, we observed children using it in both structured and unstructured play settings. Our findings show that ChromaConnect enabled children to create a common language of play, made divergent play modes more visible, and facilitated explicit expression of social play initiation. We discuss how these findings could be used to design `accompanying social play things' that are more inclusive of neurodiverse play characteristics and divergent play styles.2025BMBrooke Morris et al.University of Bristol, School of Computer ScienceCognitive Impairment & Neurodiversity (Autism, ADHD, Dyslexia)Universal & Inclusive DesignSpecial Education TechnologyCHI
Understanding Break-ability through Screen-based AffordancesCan J.J. Gibson’s concept of affordances be empirically examined using screen-based technology? We show how screen-based affordances can be examined through the use case of perceptual toughness, i.e. the break-ability of a virtual object. We present two user experiments (n=72, n=66) examining break-ability through a novel ’Perceptual Impact Testing’ methodology and online screen-based 3D virtual environment. We show that judgements of break-ability are systematically distorted when a perceiver’s virtual ‘Point of Observation’ or virtual environment’s ‘Horizonal Geometry’ are manipulated. These statistically significant results provide evidence that: 1) direct perception can account for perceptual distortions of break-ability; 2) Gibsonian affordances can be empirically examined through screen-based interactions.2025RGRichard Grafton et al.University of BristolFull-Body Interaction & Embodied InputVisualization Perception & CognitionCHI
"I Don't Really Get Involved In That Way": Investigating Blind and Visually Impaired Individuals’ Experiences of Joint Attention with Sighted PeopleJoint attention (JA) is a crucial component of social interaction, relying heavily on visual cues like eye gaze and pointing. This creates barriers for blind and visually impaired people (BVI) to engage in JA with sighted peers. Yet, little research has characterised these barriers or the strategies BVI people employ to overcome them. We interviewed ten BVI adults to understand JA experiences and analysed videos of four BVI children with eight sighted partners engaging in activities conducive to JA. Interviews revealed that lack of JA feedback is perceived as voids that block engagement, exacerbated in group settings, with an emphasis on oneself to fill those voids. Video analysis anchored the absence of the person element within typical JA triads, suggesting a potential for technology to foster alternative dynamics between BVI and sighted people. We argue these findings could inform technology design that supports more inclusive JA interactions.2024KJKatherine Jones et al.University of BristolVisual Impairment Technologies (Screen Readers, Tactile Graphics, Braille)Universal & Inclusive DesignCHI
Understanding Neurodiverse Social Play Between Autistic and Non-Autistic ChildrenSocial play supports children to develop essential life skills and foster friendships. However, autistic and non-autistic children often do not have equal opportunities to engage in social play. Previous research to improve these opportunities tends to invoke social skill interventions solely for autistic children or is focused on designing for only one group, rather than considering the interactions or needs of all children in neurodiverse groups. In order to understand the different experiences of children during social play, we conducted interviews with 6 professionals who support neurodiverse social play and undertook observation sessions of 36 autistic and non-autistic children during unstructured social play. Our findings move beyond the existing characterizations of autistic social play and build upon the double empathy problem to capture and consider the needs of all children in neurodiverse playgroups. We argue these findings could be used to inform future neurodiverse social play technology design in HCI.2024BMBrooke Morris et al.University of BristolCognitive Impairment & Neurodiversity (Autism, ADHD, Dyslexia)Universal & Inclusive DesignCollaborative Learning & Peer TeachingCHI
How does HCI Understand Human Agency and Autonomy?Human agency and autonomy have always been fundamental concepts in HCI. New developments, including ubiquitous AI and the growing integration of technologies into our lives, make these issues ever pressing, as technologies increase their ability to influence our behaviours and values. However, in HCI understandings of autonomy and agency remain ambiguous. Both concepts are used to describe a wide range of phenomena pertaining to sense-of-control, material independence, and identity. It is unclear to what degree these understandings are compatible, and how they support the development of research programs and practical interventions. We address this by reviewing 30 years of HCI research on autonomy and agency to identify current understandings, open issues, and future directions. From this analysis, we identify ethical issues, and outline key themes to guide future work. We also articulate avenues for advancing clarity and specificity around these concepts, and for coordinating integrative work across different HCI communities.2023DBDan Bennett et al.University of BristolPrivacy by Design & User ControlTechnology Ethics & Critical HCICHI
Using Virtual Reality and Co-design to Study the Design of Large-scale Shape-Changing InterfacesLarge-scale shape-changing interfaces (SCIs) such as shape-changing walls offer opportunities for enhancing user experiences within buildings, e.g., for navigation. However, due to the embryonic nature of SCI technologies, designing and explaining the shape features that are beneficial to users is challenging. Previous work used virtual platforms (2D video or Projected Augmented Reality) to design SCI. This paper explores how Virtual Reality (VR) can provide an immersive experience that can help in designing large-scale SCI. We follow a co-design approach in which we use VR to obtain users’ impressions of shape-changing walls. Then, we conduct co-design sessions to understand how shape-changing walls can be designed to become ambient and blend with the environment. We report our results to guide the design of shape-changing walls as well as discuss how our approach can provide valuable insights into how a VR experience, prior to design, and can help in the design process.2023LALuluah Albarrak et al.University of BristolShape-Changing Interfaces & Soft Robotic MaterialsPrototyping & User TestingCHI
Multifractal Mice: Inferring Task Engagement and Dimensions of Readiness-to-hand from Hand MovementThe philosophical construct readiness-to-hand describes focused, intuitive, tool use, and has been linked to tool-embodiment and immersion. The construct has been influential in HCI and design for decades, but researchers currently lack appropriate measures and tools to investigate it empirically. To support such empirical work we investigate the possibility of operationalising readiness-to-hand in measurements of multfractality in movement, building on recent work in cognitive science. We conduct two experiments (N=44, N=30) investigating multifractality in mouse movements during a computer game, replicating prior results and contributing new findings. Our results show that multifractality correlates with dimensions associated with readiness-to-hand, including skill and task-engagement, during tool breakdown, task learning and normal play. We describe future possibilities for the application of these methods in HCI, supporting such work by sharing scripts and data, and introducing a new data-driven approach to parameter selection.2022DBDan Bennett et al.University of BristolVisualization Perception & CognitionComputational Methods in HCICHI
It's Touching: Understanding Touch-Affect Association in Shape-Change with Kinematic FeaturesWith the proliferation of shape-change research in affective computing, there is a need to deepen understandings of affective responses to shape-change display. Little research has focused on affective reactions to tactile experiences in shape-change, particularly in the absence of visual information. It is also rare to study response to the shape-change as it unfolds, isolated from a final shape-change outcome. We report on two studies on touch-affect associations, using the crossmodal ``Bouba-Kiki'' paradigm, to understand affective responses to shape-change as it unfolds. We investigate experiences with a shape-change gadget, as it moves between rounded (``Bouba'') and spiky (``Kiki'') forms. We capture affective responses via the circumplex model, and use a motion analysis approach to understand the certainty of these responses. We find that touch-affect associations are influenced by both the size and the frequency of the shape-change and may be modality-dependent, and that certainty in affective associations is influenced by association-consistency.2022FFFeng Feng et al.University of BristolShape-Changing Interfaces & Soft Robotic MaterialsVisualization Perception & CognitionCHI
Feeling Colours: Investigating Crossmodal Correspondences Between 3D Shapes, Colours and EmotionsWith increasing interest in multisensory experiences in HCI there is a need to consider the potential impact of crossmodal correspondences (CCs) between sensory modalities on perception and interpretation. We investigated CCs between active haptic experiences of tangible 3D objects, visual colour and emotion using the "Bouba/Kiki" paradigm. We asked 30 participants to assign colours and emotional categories to 3D-printed objects with varying degrees of angularity and complexity. We found tendencies to associate high degrees of complexity and angularity with red colours, low brightness and high arousal levels. Less complex round shapes were associated with blue colours, high brightness and positive valence levels. These findings contrast previously reported crossmodal effects triggered by 2D shapes of similar angularity and complexity, suggesting that designers cannot simply extrapolate potential perceptual and interpretive experiences elicited by 2D shapes to seemingly similar 3D tangible objects. Instead, we propose a design space for creating tangible multisensory artefacts that can trigger specific emotional percepts and discuss implications for exploiting CCs in the design of interactive technology.2021ALAnan Lin et al.University of BristolShape-Changing Interfaces & Soft Robotic MaterialsVisualization Perception & CognitionSTEM Education & Science CommunicationCHI
Exploring the Design of History-Enriched Floor Interfaces for Asynchronous Navigation SupportEnvironmental cues influence our spatial behaviour when we explore unfamiliar spaces. Research particularly shows that the presence/actions of other people affects our navigation decisions. Here we examine how such social information can be integrated digitally into the environment to support navigation in indoor public spaces. We carried out a study (n=12) to explore how to represent traces of navigation behaviour. We compared 6 floor visualisations and examined how they affect participants' navigational choices. Results suggest that direct representations such as footprints are most informative. To investigate further how such visualisation could work in practice, we implemented an interactive floor system and used it as probe during one-to-one design sessions (n=26). We particularly focused on four design challenges: the overall visual representation, representation of multiple people, designing more prominent visualisations and the incorporation of non-identifying information. Our results provide insights for designers looking to develop history-enriched floor interfaces.2020LALuluah Albarrak et al.Geospatial & Map VisualizationPrototyping & User TestingDIS
Review of Quantitative Empirical Evaluations of Technology for People with Visual ImpairmentsAddressing the needs of visually impaired people is of continued interest in Human Computer Interaction (HCI) research. Yet, one of the major challenges facing researchers in this field continues to be how to design adequate quantitative empirical evaluation for these users in HCI. In this paper, we analyse a corpus of 178 papers on technologies designed for people with visual impairments, published since 1988, and including at least one quantitative empirical evaluation (243 evaluations in total). To inform future research in this area, we provide an overview, historic trends and a unified terminology to design and report quantitative empirical evaluations. We identify open issues and propose a set of guidelines to address them. Our analysis aims to facilitate and stimulate future research on this topic.2020EBEmeline Brulé et al.University of SussexVisual Impairment Technologies (Screen Readers, Tactile Graphics, Braille)User Research Methods (Interviews, Surveys, Observation)Prototyping & User TestingCHI
Robots for Inclusive Play: Co-designing an Educational Game With Visually Impaired and sighted ChildrenDespite being included in mainstream schools, visually impaired children still face barriers to social engagement and participation. Games could potentially help, but games that cater for both visually impaired and sighted players are scarce. We used a co-design approach to design and evaluate a robot-based educational game that could be inclusive of both visually impaired and sighted children in the context of mainstream education. We ran a focus group discussion with visual impairment educators to understand barriers to inclusive play. And then a series of co-design workshops to engage visually impaired and sighted children and educators in learning about robot technology and exploring its potential to support inclusive play experiences. We present design guidelines and an evaluation workshop of a game prototype, demonstrating group dynamics conducive to collaborative learning experiences, including shared goal setting/execution, closely coupled division of labour, and interaction symmetry.2020OMOussama Metatla et al.University of BristolAccessible GamingSerious & Functional GamesSpecial Education TechnologyCHI