Evaluating Interfaces for Non-Driving Related Tasks While Operating an E-scooterMicromobility vehicles, such as e-scooters, provide ecological and financial benefits over automotive transportation. However, as with car drivers, micromobility users often perform non-driving related tasks (NDRTs), interacting with stereo controls or navigation tasks, which can lead to accidents. It remains unclear what control interfaces are appropriate and safe for micromobility. We evaluated six interface modalities for NDRTs and conducted a within-subjects study with 35 participants (yielding n=210 observations) in an e-scooter simulator to compare modality safety and preferences. Our results align with existing work on gaze and tactility in the automotive NDRTs context. However, unique to e-scooters, interfaces that required users to alter their grip on the handlebars were less preferred as they compromised stability. Social comfort also emerged as a critical factor due to concerns about public visibility. This work aims to encourage the design of safer, more socially acceptable interfaces for e-scooters and other emerging micromobility vehicles.2025KTKenshikimyo Terao et al.In-Vehicle Haptic, Audio & Multimodal FeedbackMicromobility (E-bike, E-scooter) InteractionAutoUI
Socially Adaptive Autonomous Vehicles: Effects of Contingent Driving Behavior on Drivers' ExperiencesSocial scientists have argued that autonomous vehicles (AVs) need to act as effective social agents; they have to respond implicitly to other drivers' behaviors as human drivers would. In this paper, we investigate how contingent driving behavior in AVs influences human drivers' experiences. We compared three algorithmic driving models: one trained on human driving data that responds to interactions (a familiar contingent behavior) and two artificial models that intend to either always-yield or never-yield regardless of how the interaction unfolds (non-contingent behaviors). Results show that a familiar contingent behavior significantly reduces drivers' hesitance and stress when interacting with AVs. The direct relationship between familiar contingency and positive experience indicates that AVs should incorporate socially familiar driving patterns through contextually-adaptive algorithms to improve the chances of successful deployment and acceptance in mixed human-AV traffic environments.2025CYChishang "Mario" Yang et al.Automated Driving Interface & Takeover DesignAI-Assisted Decision-Making & AutomationAutoUI
Simulating Multiple Road User Perspectives on Autonomous Vehicle BehaviorsThis work presents a system and a study in which we have multiple road users interact simultaneously with an autonomous vehicle (AV) in a virtual reality (VR) environment. We go beyond studying dyadic interactions (e.g., AV-pedestrian or AV-driver) to involve a pedestrian, a human driver, and an AV passenger all jointly interacting with an AV in the same VR scenario. We probed multiple user perspectives with two different prototypes of AV behavior strategies in ambiguous stop-sign intersections. An efficient AV attempts to enter the intersection as soon as it can without collision, while a prosocial AV waits for other road users to pass before proceeding. We recruited 16 three-person groups (N=48), where half interacted with the first AV type and the other half interacted with the second AV type in four different traffic configurations. Our investigation demonstrates that road users in different roles can have diverging preferences and trust levels in the same AV behavior when making joint decisions. Finally, we discuss how our methods and findings can be used to guide further explorations for AV interaction research with multiple agents in different roles.2025JJJiHyun Jeong et al.Automated Driving Interface & Takeover DesignExternal HMI (eHMI) — Communication with Pedestrians & CyclistsTeleoperated DrivingAutoUI
The People Behind the Robots: How Wizards Wrangle Robots in Public DeploymentsIn the Wizard-of-Oz study paradigm, human "wizards" perform not-yet-implemented system behavior, simulating, among others, how autonomous robots could interact in public to see how unwitting bystanders respond. This paper analyzes a 60-minute video recording of two wizards in a public plaza who are operating two trash-collecting robots within their line of sight. We take an ethnomethodology and conversation analysis perspective to scrutinize interactions between the wizards and the people in the plaza, focusing on critical instances where one robot gets stuck and requires collaborative intervention by the wizards. Our analysis unpacks how the wizards deal with emergent problems by pushing one robot into the other, how they manage front and backstage interactions, and how they monitor the location of each other's robots. We discuss how scrutinizing the work of wizards can inform explorative Wizard-of-Oz paradigms, the design of multi-agent robot systems, and the operation of urban robots from a distance.2025HPHannah RM Pelikan et al.Linköping University, Department of Culture and SocietySocial Robot InteractionTeleoperation & TelepresenceCHI
Decoding Driver Intention Cues: Exploring Non-verbal Communication for Human-Centered Automotive InterfacesIn emerging "driver-less" automated vehicles (AVs), the intuitive communication that exists between human drivers and passengers no longer exists, which can lead to reduced trust and acceptance in passengers if they are unclear about what the AV intends to do. This paper contributes the foundational understanding of how passengers naturally decode drivers' non-verbal cues about their intended action to inform intuitive Human-Machine Interface (HMI) designs that try to emulate those cues. Our study investigates what cues passengers perceive, their saliency, and interpretation through a mixed-method approach combining field observations, experience sampling, and auto-confrontation interviews with 30 driver-passenger pairs. Analysis of posture, head/eye movements, and vestibular sensations revealed four categories of intention cues: awareness, interaction, vestibular, and habitual. These findings provide empirical foundations for designing AV interfaces that mirror natural human communication patterns. We discuss implications for designing anthropomorphic HMIs that could enhance trust, predictability, and user experience in AVs.2025MFMohammad Faramarzian et al.Queensland University of Technology (QUT), Centre for Accident Research and Road Safety - Queensland (CARRS-Q)Automated Driving Interface & Takeover DesignIn-Vehicle Haptic, Audio & Multimodal FeedbackCHI
The Robotability Score: Enabling Harmonious Robot Navigation on Urban StreetsThis paper introduces the Robotability Score (R), a novel metric that quantifies the suitability of urban environments for autonomous robot navigation. Through expert interviews and surveys, we identify and weigh key features contributing to R for wheeled robots on urban streets. Our findings reveal that pedestrian density, crowd dynamics and pedestrian flow are the most critical factors, collectively accounting for 28% of the total score. Computing robotability across New York City yields significant variation; the area of highest R is 3.0 times more "robotable'' than the area of lowest R. Deployments of a physical robot on high and low robotability areas show the adequacy of the score in anticipating the ease of robot navigation. This new framework for evaluating urban landscapes aims to reduce uncertainty in robot deployment while respecting established mobility patterns and urban planning principles, contributing to the discourse on harmonious human-robot environments.2025MFMatthew Franchi et al.Cornell Tech, Computer ScienceHuman-Robot Collaboration (HRC)Smart Cities & Urban SensingCHI
Understanding Farmers' Data Collection Practices on Small-to-Medium Farms for the Design of Future Farm Management Information SystemsFarm Management Information Systems (FMIS) integrate data from a variety of sources, including sensors, for the purpose of enabling farmers to interpret past activity and predict future performance. FMIS is traditionally designed for and used by large farms, given their capital and need for automation and to scale up. This paper examines the current data collection practices on small and medium farms so that FMIS systems can be better designed to their needs. Our empirical research comprises interviews conducted during 10 farm visits. Our semi-structured interviews incorporated questions about daily activities, points of decision-making, data sharing, and incentives for data collection. We analyzed the interviews by focusing on possible obstacles to adopting expanding digital data collection practices and how expanded data collection might help fulfill farmers' goals and motivations. We found that farmers use their own bespoke data collection techniques instead of or in parallel to more formalized methods and often hold key observations and hypotheses in their heads rather than committing them to any data collection system at all. Key barriers to FMIS adoption include technology skepticism, technical hurdles and lack of support, and self-doubt in technical skills. Based on this empirical work and analysis, we recommend that FMIS systems can best address the needs of small and medium farms by: 1) accounting for the farmers' different approaches to memorizing vs. storing data, 2) integrating rather than trying to replace existing practices, and 3) considering the economic and political motivations driving farm decision-making and practices.2024NFNatalie Friedman et al.Session 1e: Empowering Data WorkCSCW
Modeling Social Situation Awareness in Driving InteractionsThe design of self-driving vehicles requires an understanding of the social interactions between drivers in resolving vague encounters, such as at un-signalized intersections. In this paper, we make the case for social situation awareness as a model for understanding everyday driving interaction. Using a dual-participant VR driving simulator, we collected data from driving encounter scenarios to understand how (N=170) participant drivers behave with respect to one another. Using a social situation awareness questionnaire we developed, we assessed the participants' social awareness of other driver’s direction of approach to the intersection, and also logged signaling, speed and speed change, and heading of the vehicle. Drawing upon the statistically significant relationships in the variables in the study data, we propose a Social Situation Awareness model based on the approach, speed, change of speed, heading and explicit signaling from drivers.2024NKNavit Klein et al.Automated Driving Interface & Takeover DesignV2X (Vehicle-to-Everything) Communication DesignAutoUI
Behind the Scenes of CXR: Designing a Geo-Synchronized Communal eXtended Reality SystemWe have developed a Communal eXtended-Reality (CXR) system that enables groups of people in a shared moving vehicle to view a common geo-synchronized tour. This paper describes the geo-synchronized multi-user extended reality system we created to provide a situated and shared experience to promote community engagement. This paper describes (a) the technical implementation of the CXR system, which geo-locates and orients the view of the participant within the moving vehicle; (b) the immersive digital twin tour, critically aligned with the real-life location; (c) our fallback system, which allows people who feel disoriented or motion-sick to continue along with the content of the tour. We validated the sense of communality, comfort, and effectiveness of the system through in-ride observation and post-ride surveys. Our intent is to enable the development of similar systems to foster communal engagement in communities worldwide.2024SYSharon Yavo-Ayalon et al.V2X (Vehicle-to-Everything) Communication DesignSocial & Collaborative VRAR Navigation & Context AwarenessDIS
Multi-Modal eHMIs: The Relative Impact of Light and Sound in AV-Pedestrian InteractionExternal Human-Machine Interfaces (eHMIs) have been evaluated to facilitate interactions between Automated Vehicles (AVs) and pedestrians. Most eHMIs are, however, visual/ light-based solutions, and multi-modal eHMIs have received little attention to date. We ran an experimental video study (N = 29) to systematically understand the effect on pedestrian's willingness to cross the road and user preferences of a light-based eHMI (light bar on the bumper) and two sound-based eHMIs (bell sound and droning sound), and combinations thereof. We found no objective change in pedestrians' willingness to cross the road based on the nature of eHMI, although people expressed different subjective preferences for the different ways an eHMI may communicate, and sometimes even strong dislike for multi-modal eHMIs. This shows that the modality of the evaluated eHMI concepts had relatively little impact on their effectiveness. Consequently, this lays an important groundwork for accessibility considerations of future eHMIs, and points towards the insight that provisions can be made for taking user preferences into account without compromising effectiveness.2024DDDebargha Dey et al.Cornell TechExternal HMI (eHMI) — Communication with Pedestrians & CyclistsIn-Vehicle Haptic, Audio & Multimodal FeedbackCHI
Portobello: Extending Driving Simulation from the Lab to the RoadIn automotive user interface design, testing often starts with lab-based driving simulators and migrates toward on-road studies to mitigate risks. Mixed reality (XR) helps translate virtual study designs to the real road to increase ecological validity. However, researchers rarely run the same study in both in-lab and on-road simulators due to the challenges of replicating studies in both physical and virtual worlds. To provide a common infrastructure to port in-lab study designs on-road, we built a platform-portable infrastructure, Portobello, to enable us to run twinned physical-virtual studies. As a proof-of-concept, we extended the on-road simulator XR-OOM with Portobello. We ran a within-subjects, autonomous-vehicle crosswalk cooperation study (N=32) both in-lab and on-road to investigate study design portability and platform-driven influences on study outcomes. To our knowledge, this is the first system that enables the twinning of studies originally designed for in-lab simulators to be carried out in an on-road platform.2024FBFanjun Bu et al.Cornell TechAutomated Driving Interface & Takeover DesignV2X (Vehicle-to-Everything) Communication DesignMixed Reality WorkspacesCHI
Trash in Motion: Emergent Interactions with a Robotic TrashcanThe introduction of robots in public spaces raises many questions concerning emergent interactions with robots. In this paper, we use video analysis to study two robotic trashcans deployed in a busy city square. We focus on the movement-based practices that emerged between the robot, the robot operators, and the inhabitants of the square. These practices spanned ways of attracting the robot and disposing of trash, the robot 'asking' for trash, 'demonstrations' by those in the square, as well as passersby in the square navigating around and in coordination with the robots. In discussion, we document these 'spontaneous simple sequential systematics' - interactions that were systematic (they had an order), sequential (they had parts that happened one at a time), simple (in that they could be understood and copied by an observer) and spontaneous (they could be produced with no prompting or training). Building on this we discuss how we might think of robotic motion as a design space, along with HCI contributions to urban robotics.2024BBBarry Brown et al.Stockholm University, University of CopenhagenSocial Robot InteractionSmart Cities & Urban SensingCommunity Engagement & Civic TechnologyCHI
AdVANcing Design: Customizing Spaces for VanlifeThis study examines three modalities for designing live-in van interiors. Participants (N=18) situated within an empty van were asked to explore potential designs using physical cardboard prototyping, a commercial software application (Vanspace 3D) for planning van interiors, and an augmented reality application that we developed. Participants were asked to think aloud as they designed van interiors for fictive journeys using each modality. A qualitative evaluation was conducted to assess how participants' conceptualizations of space shifted across the use of each prototyping method. The results demonstrate that each design method influenced design outcomes due to the physicality of the task. This study highlights the importance of considering the role of physicality in the selection of prototyping modality for the design exploration process.2023SSSaki Suzuki et al.Shape-Changing Interfaces & Soft Robotic MaterialsShape-Changing Materials & 4D PrintingCustomizable & Personalized ObjectsAutoUI
XR-OOM: MiXed Reality driving simulation with real cars for research and designHigh-fidelity driving simulators can act as testbeds for designing in-vehicle interfaces or validating the safety of novel driver assistance features. In this system paper, we develop and validate the safety of a mixed reality driving simulator system that enables us to superimpose virtual objects and events into the view of participants engaging in real-world driving in unmodified vehicles. To this end, we have validated the mixed reality system for basic driver cockpit and low-speed driving tasks, comparing the use of the system with non-headset and with the headset driving conditions, to ensure that participants behave and perform similarly using this system as they would otherwise. This paper outlines the operational procedures and protocols for using such systems for cockpit tasks (like using the parking brake, reading the instrument panel, and turn signaling) as well as basic low-speed driving exercises (such as steering around corners, weaving around obstacles, and stopping at a fixed line) in ways that are safe, effective, and lead to accurate, repeatable data collection about behavioral responses in real-world driving tasks.2022DGDavid Goedicke et al.Cornell TechAutomated Driving Interface & Takeover DesignHead-Up Display (HUD) & Advanced Driver Assistance Systems (ADAS)Mixed Reality WorkspacesCHI
Unmaking as Agonism: Using Participatory Design with Youth to Surface Difference in an Intergenerational Urban ContextDesign has been used to contest existing socio-technical arrangements, provoke conversations around matters of concern, and operationalize radical theories such as agonism, which embraces difference and contention. However, the focus is usually on creating something new: a product, interface or artifact. In this paper, we investigate what happens when critical unmaking is deployed as a deliberate design strategy in an intergenerational, agonistic urban context. Specifically, we report on how youth in a six-week design internship used unmaking as a design move to subvert conventional narratives about their surrounding urban context. We analyze how this led to conflictual encounters at the local senior center, and compare it to the other, making-centric proposals which received favorable feedback but failed to raise the same important discussions. Through this ethnographic account, we argue that critical unmaking is important yet overlooked, and should be in the repertoire of design moves available for agonism and provocation.2022SSSamar Sabie et al.Cornell TechEmpowerment of Marginalized GroupsTechnology Ethics & Critical HCIParticipatory DesignCHI
Next Steps for Human-Computer IntegrationHuman-Computer Integration (HInt) is an emerging paradigm in which computational and human systems are closely interwoven. Integrating computers with the human body is not new. however, we believe that with rapid technological advancements, increasing real-world deployments, and growing ethical and societal implications, it is critical to identify an agenda for future research. We present a set of challenges for HInt research, formulated over the course of a five-day workshop consisting of 29 experts who have designed, deployed and studied HInt systems. This agenda aims to guide researchers in a structured way towards a more coordinated and conscientious future of human-computer integration.2020FMFlorian Floyd Mueller et al.Monash UniversityBrain-Computer Interface (BCI) & NeurofeedbackTechnology Ethics & Critical HCIUser Research Methods (Interviews, Surveys, Observation)CHI
Novel Human-Machine Interfaces for the Management of User-Vehicle Transitions in Automated Driving For automated vehicles operating at SAE Level 4 capability, control could feasibly be passed from machine to human and vice versa -regardless of whether minimal risk condition exists as a fallback solution. We propose two Human-Machine Interfaces (HMIs) to assist in the management of these transitions: 1) A ‘Responsibility Panel’ providing the necessary feedback for a user to understand who must undertake different driving related activities (look, brake, throttle, steer) and who might be liable if a fault arises (user or car company); 2) A ‘Readiness to Drive’ testing HMI that only allows a human to retake control when a certain level of competency is demonstrated. Future work should evaluate the effectiveness of our HMIs.2019GBGary Burnett et al.Automated Driving Interface & Takeover DesignAutoUI
Is Now A Good Time? An Empirical Study of Vehicle-Driver Communication TimingAdvances in automotive sensing systems and speech interfaces provide new opportunities for smarter driving assistants or infotainment systems. For both safety and consumer satisfaction reasons, any new system which interacts with drivers must do so at appropriate times. We asked 63 drivers, ''Is now a good time?'' to receive non-driving information during a 50-minute drive. We analyzed 2,734 responses and synchronized automotive and video data, and show that while the chances of choosing a good time can be determined with better success using easily accessible automotive data, certain nuances in the problem require a richer understanding of the driver and environment states in order to achieve higher performance. We illustrate several of these nuances with quantitative and qualitative analyses to contribute to the understanding of how to design a system that might simultaneously minimize the risk of interacting at a bad time while maximizing the window of allowable interruption.2019RSRob Semmens et al.Naval Postgraduate SchoolHead-Up Display (HUD) & Advanced Driver Assistance Systems (ADAS)Notification & Interruption ManagementCHI
How People Experience Autonomous Intersections: Taking a First-Person PerspectiveTop-down simulations of autonomous intersections neglect considerations for the human experience of being in cars driving through these autonomous intersections. To understand the impact that perspective has on perception of autonomous intersections, we conducted a driving simulator experiment and studied the experience in terms of perception, feelings, and pleasure. Based on this data, we discuss experiential factors of autonomous intersections that are perceived as beneficial or detrimental for the future driver. Furthermore, we present what the change of perspective implies for designing intersection models, future in-car interfaces and simulation techniques.2019SKSven Krome et al.Automated Driving Interface & Takeover DesignAutoUI
Face and Ecological Validity in Simulations: Lessons from Search-and-Rescue HRIIn fields where in situ performance cannot be measured, ecological validity is difficult to estimate. Drawing on theory from social psychology and virtual reality, we argue that face validity can be a useful proxy for ecological validity. We provide illustrative examples of this relationship from work in search-and-rescue HRI, and conclude with some practical guidelines for the construction of immersive simulations in general.2019LDLorin Dole et al.Cornell TechImmersion & Presence ResearchTeleoperation & TelepresenceField StudiesCHI