Vistoryteller: Designing Data Stories with LLM Agent-Based Generation and Interactive User ControlData stories that combine data, visualizations, and prose are widely used for communication, decision making, and persuasion, but producing them typically requires coordinated effort across specialized roles such as analysts, scripters, and designers, which is time consuming and difficult to manage. Existing AI-assisted methods generally treat storytelling as a single-agent task and offer only coarse, global controls, limiting an author’s ability to preserve and shape their communication intention over the course of a narrative. In this work, we present Vistoryteller, a multi-agent authoring system that models the division of labor found in human teams by assigning specialized large language model agents to complementary roles and orchestrating their interactions to generate cohesive, intention-aligned data stories. Vistoryteller supports fine-grained authorial control through two complementary mechanisms: a sketch-based tension-flow control for specifying how thematic emphasis and narrative tension should evolve, and a conversational interface for issuing localized directives to individual agents or to the team. We evaluate Vistoryteller with two controlled experiments and a qualitative user study. Results show that Vistoryteller generates narratives that align more closely with user intentions, preserve coherence across agent contributions, and surface diverse and expressive insights.2026YSYang Shi et al.College of Design and Innovation, Tongji UniversityGenerative AI (Text, Image, Music, Video)Data StorytellingCreative Collaboration & Feedback SystemsIUI
NetworkCanvas: Supporting Progressive Network Visualization Exploration via Adaptive RecommendationsNetwork visualization has become essential for understanding complex relationships across domains, yet network complexity creates an overwhelming exploration space where users frequently miss critical patterns. Existing tools often require predetermined analysis goals and manual workflow construction, limiting accessibility for non-experts. We present NetworkCanvas, a progressive network visualization system that guides users through personalized exploration via adaptive recommendations. Our approach combines a learning mechanism that adapts to user feedback, an analytic state graph preserving exploration provenance with branching paths, and a context-aware feedback interpreter that suggests analytical continuations based on selection patterns. Controlled studies demonstrate that NetworkCanvas users identified more noteworthy observations, reported higher confidence, and exhibited more systematic exploration compared to a baseline without recommendations. These results demonstrate that recommendation-guided exploration improves outcomes over unguided manual analysis; however, because our baseline lacked recommendation functionality entirely, the specific contribution of adaptive personalization versus static guidance remains an open question. Qualitative findings suggest that recommendations reduce analysis paralysis and support systematic exploration.2026WLWenchao Li et al.HUAWEI TECHNOLOGIES CO., LTD.Interactive Data VisualizationExploratory Search & Information SeekingKnowledge Graph & Semantic SearchCHI
Understanding the Effects of Interaction on Emotional Experiences in VRVirtual reality has been effectively used for eliciting emotions, yet most research focuses on the intensity of affective responses rather than on how interaction influences those experiences. To address this gap, we advance a validated VR emotion-elicitation dataset through two key extensions. First, we add a new high-arousal, high-valence scene and validate its effectiveness in a within-subject study (N=24). Second, we incorporate interactive elements into each scene, creating both interactive and non-interactive versions to examine the impact of interaction on emotional responses. We evaluate interaction through a multimodal approach combining subjective ratings and physiological signals to capture both conscious and unconscious affective responses. Our evaluation study (N=84) shows that interaction not only amplifies emotions but modulates them in context, supporting coping in negative scenes and enhancing enjoyment in positive scenes. These findings highlight the potential of scene-tailored interaction for different applications, where regulating emotions is as important as eliciting them.2026ZKZheyuan Kuang et al.The University of SydneyImmersion & Presence ResearchAffective Feedback & Emotion Regulation InterfacesSocial & Collaborative VRCHI
SemTabla: A Human-in-the-Loop Framework for Semantic Enrichment and Validation of Data TablesData tables are widely used to record critical information, enabling decision-makers to derive insights through table question answering (Table QA). However, the metadata from table schemas alone often fail to capture the underlying business semantics embedded in the tabular data, leading to reasoning errors. Existing automated approaches to semantic enrichment face challenges in insufficient data utilization, narrow feature coverage, and limited interpretability. To overcome these limitations, we propose SemTabla, an interactive system that employs a human-in-the-loop mechanism to extract comprehensive and interpretable semantics from tabular data. Our key contributions include: (1) a hierarchical framework for extracting semantic attributes; (2) a novel sampling method that identifies critical but rare row instances; and (3) an interactive interface that supports visualization, validation, and refinement of the extracted table semantics. A user study confirmed the system’s usability, and quantitative experiments demonstrate that the extracted semantics significantly enhance the reasoning capabilities of large language models.2026ZJZhuochen Jin et al.Huawei CloudExplainable AI (XAI)Interactive Data VisualizationUser Research Methods (Interviews, Surveys, Observation)CHI
InterQuest: A Mixed-Initiative Framework for Dynamic User Interest Modeling in Conversational SearchIn online information-seeking tasks (e.g., for products and restaurants), users seek information that aligns with their individual preferences to make informed decisions. However, existing systems often struggle to infer users' implicit interests—unstated yet essential preference factors that directly impact decision quality. Our formative study reveals that User-Centric Knowledge—cross-task persistent preference attributes of users (e.g., "user cares about functionality details for electronics")—serves as a key indicator for resolving users' implicit interests. However, constructing such knowledge from task-specific data alone is insufficient due to three types of uncertainties—cold-start limitation, content accuracy, and scope applicability—which require user-provided information for knowledge alignment. Based on these insights, we present InterQuest, an LLM-based conversational search agent that dynamically models user interests. InterQuest combines two strategies: (1) Dynamic User Knowledge Modeling, which infers and adjusts the content and scope of User-Centric Knowledge, and (2) Uncertainty-Driven Questioning, where InterQuest proactively asks questions to resolve knowledge uncertainties. A user study with 18 participants demonstrates that InterQuest outperforms the baselines in user interest inference, accuracy of user knowledge modeling, and the overall information-seeking experience. Additionally, our findings provide valuable design implications for improving mixed-initiative user modeling in future systems.2025YMYu Mei et al.Human-LLM CollaborationRecommender System UXAlgorithmic Fairness & BiasUIST
Navigating the Unknown: A Chat-Based Collaborative Interface for Personalized Exploratory TasksThe rise of large language models (LLMs) has revolutionized user interactions with knowledge-based systems, enabling chatbots to synthesize vast amounts of information and assist with complex, exploratory tasks. However, LLM-based chatbots often struggle to provide personalized support, particularly when users start with vague queries or lack sufficient contextual information. This paper introduces the Collaborative Assistant for Personalized Exploration (CARE), a system designed to enhance personalization in exploratory tasks by combining a multi-agent LLM framework with a structured user interface. CARE's interface consists of a Chat Panel, Solution Panel, and Needs Panel, enabling iterative query refinement and dynamic solution generation. The multi-agent framework collaborates to identify both explicit and implicit user needs, delivering tailored, actionable solutions. In a within-subject user study with 22 participants, CARE was consistently preferred over a baseline LLM chatbot, with users praising its ability to reduce cognitive load, inspire creativity, and provide more tailored solutions. Our findings highlight CARE's potential to transform LLM-based systems from passive information retrievers to proactive partners in personalized problem-solving and exploration. The code will be made available at https://aka.ms/chatbot-care.2025YPYingzhe Peng et al.Human-LLM CollaborationCrowdsourcing Task Design & Quality ControlIUI
AutoPBL: An LLM-powered Platform to Guide and Support Individual Learners Through Self Project-based LearningSelf project-based learning (SPBL) is a popular learning style where learners follow tutorials and build projects by themselves. SPBL combines project-based learning’s benefit of being engaging and effective with the flexibility of self-learning. However, insufficient guidance and support during SPBL may lead to unsatisfactory learning experiences and outcomes. While LLM chatbots (e.g., ChatGPT) could potentially serve as SPBL tutors, we have yet to see an SPBL platform with responsible and systematic LLM integration. To address this gap, we present AutoPBL, an interactive learning platform for SPBL learners. We examined human PBL tutors’ roles through formative interviews to inform our design. AutoPBL features an LLM-guided learning process with checkpoint questions and in-context Q&A. In a user study where 29 beginners learned machine learning through entry-level projects, we found that AutoPBL effectively improves learning outcomes and elicits better learning behavior and metacognition by clarifying current priorities and providing timely assistance.2025YZYihao Zhu et al.Tsinghua University, Department of Computer Science and TechnologyHuman-LLM CollaborationProgramming Education & Computational ThinkingIntelligent Tutoring Systems & Learning AnalyticsCHI
Characterizing Developers’ Linguistic Behaviors in Open Source Development across Their Social StatusesOpen Source Software (OSS) development has attracted numerous developers. As a typical complex sociotechnical system, an OSS project often forms a hierarchical social structure where a few developers are elite while the rest are non-elite. Differences in social status may result in distinct language use behaviors in interpersonal communication. Characterizing such behaviors is critical for supporting efficient and effective communication among developers with different social statuses. This study empirically compared elite and non-elite developers' language behaviors in their communication. We compiled a corpus of ~216,000 discourses collected from 20 large projects on GitHub. We investigated the linguistic differences in three aspects, namely, linguistic styles and characters, main concerns, and sentence patterns. Our findings reveal that elite and non-elite developers showed different linguistic patterns and had different concerns in their discourses. Their discourses also reflect the variation of the main focuses in the development process. Furthermore, elite and non-elite developers exhibited noticeable patterns in their linguistic behaviors in accordance with their roles and corresponding divisions of labor in the production process, no matter which semantic contexts. These findings provide implications for supporting communication that crosses social statuses in OSS development.2024YHYisi Han et al.Session 3b: Work, Non-Work, and Social TechnologiesCSCW
mP-Gait: Fine-grained Parkinson's Disease Gait Impairment Assessment with Robust Feature AnalysisZhang 等人提出 mP-Gait 系统，采用鲁棒特征分析实现帕金森病步态障碍的精细化自动评估，提升诊断客观性。2024WZWenhao Zhang et al.Human Pose & Activity RecognitionUbiComp
SonicVista: Towards Creating Awareness of Distant Scenes through SonificationGupta 等人开发 SonicVista 系统，通过声化技术将远程场景信息转换为声音，增强用户对远处环境的感知能力。2024CGChitralekha Gupta et al.Context-Aware ComputingUbiComp
LessonPlanner: Assisting Novice Teachers to Prepare Pedagogy-Driven Lesson Plans with Large Language ModelsPreparing a lesson plan, e.g., a detailed road map with strategies and materials for instructing a 90-minute class, is beneficial yet challenging for novice teachers. Large language models (LLMs) can ease this process by generating adaptive content for lesson plans, which would otherwise require teachers to create from scratch or search existing resources. In this work, we first conduct a formative study with six novice teachers to understand their needs for support of preparing lesson plans with LLMs. Then, we develop LessonPlanner that assists users to interactively construct lesson plans with adaptive LLM-generated content based on Gagne's nine events. Our within-subjects study (N=12) shows that compared to the baseline ChatGPT interface, LessonPlanner can significantly improve the quality of outcome lesson plans and ease users' workload in the preparation process. Our expert interviews (N=6) further demonstrate LessonPlanner's usefulness in suggesting effective teaching strategies and meaningful educational resources. We discuss concerns on and design considerations for supporting teaching activities with LLMs.2024HFHaoxiang Fan et al.Human-LLM CollaborationIntelligent Tutoring Systems & Learning AnalyticsUIST
mSilent: Towards General Corpus Silent Speech Recognition Using COTS mmWave Radar"Silent speech recognition (SSR) allows users to speak to the device without making a sound, avoiding being overheard or disturbing others. Compared to the video-based approach, wireless signal-based SSR can work when the user is wearing a mask and has fewer privacy concerns. However, previous wireless-based systems are still far from well-studied, e.g. they are only evaluated in corpus with highly limited size, making them only feasible for interaction with dozens of deterministic commands. In this paper, we present mSilent, a millimeter-wave (mmWave) based SSR system that can work in the general corpus containing thousands of daily conversation sentences. With the strong recognition capability, mSilent not only supports the more complex interaction with assistants, but also enables more general applications in daily life such as communication and input. To extract fine-grained articulatory features, we build a signal processing pipeline that uses a clustering-selection algorithm to separate articulatory gestures and generates a multi-scale detrended spectrogram (MSDS). To handle the complexity of the general corpus, we design an end-to-end deep neural network that consists of a multi-branch convolutional front-end and a Transformer-based sequence-to-sequence back-end. We collect a general corpus dataset of 1,000 daily conversation sentences that contains 21K samples of bi-modality data (mmWave and video). Our evaluation shows that mSilent achieves a 9.5% average word error rate (WER) at a distance of 1.5m, which is comparable to the performance of the state-of-the-art video-based approach. We also explore deploying mSilent in two typical scenarios of text entry and in-car assistant, and the less than 6% average WER demonstrates the potential of mSilent in general daily applications. https://dl.acm.org/doi/10.1145/3580838"2023SZShang Zeng et al.Voice User Interface (VUI) DesignIntelligent Voice Assistants (Alexa, Siri, etc.)Multilingual & Cross-Cultural Voice InteractionUbiComp
MagSound: Magnetic Field Assisted Wireless Earphone Tracking"Wireless earphones are pervasive acoustic sensing platforms that can be used for many applications such as motion tracking and handwriting input. However, wireless earphones suffer clock offset between the connected smart devices, which would accumulate error rapidly over time. Moreover, compared with smartphone and voice assistants, the acoustic signal transmitted by wireless earphone is much weaker due to the poor frequency response. In this paper, we propose MagSound, which uses the built-in magnets to improve the tracking and acoustic sensing performance of Commercial-Off-The-Shelf (COTS) earphones. Leveraging magnetic field strength, MagSound can predict the position of wireless earphones free from clock offset, which can be used to re-calibrate the acoustic tracking. Further, the fusion of the two modalities mitigates the accumulated clock offset and multipath effect. Besides, to increase the robustness to noise, MagSound employs finely designed Orthogonal Frequency-Division Multiplexing (OFDM) ranging signals. We implement a prototype of MagSound on COTS and perform experiments for tracking and handwriting input. Results demonstrate that MagSound maintains millimeter-level error in 2D tracking, and improves the handwriting recognition accuracy by 49.81%. We believe that MagSound can contribute to practical applications of wireless earphones-based sensing. https://dl.acm.org/doi/10.1145/3580889"2023LWLihao Wang et al.Hand Gesture RecognitionContext-Aware ComputingUbiquitous ComputingUbiComp
OralCam: Enabling Self-Examination and Awareness of Oral Health Using a Smartphone CameraDue to a lack of medical resources or oral health awareness, oral diseases are often left unexamined and untreated, affecting a large population worldwide. With the advent of low-cost, sensor-equipped smartphones, mobile apps offer a promising possibility for promoting oral health. However, to the best of our knowledge, no mobile health (mHealth) solutions can directly support a user to self-examine their oral health condition. This paper presents OralCam, the first interactive app that enables end-users' self-examination of five common oral conditions (diseases or early disease signals) by taking smartphone photos of one's oral cavity. OralCam allows a user to annotate additional information (e.g. living habits, pain, and bleeding) to augment the input image, and presents the output hierarchically, probabilistically and with visual explanations to help a laymen user understand examination results. Developed on our in-house dataset that consists of 3,182 oral photos annotated by dental experts, our deep learning based framework achieved an average detection sensitivity of 0.787 over five conditions with high localization accuracy. In a week-long in-the-wild user study (N=18), most participants had no trouble using OralCam and interpreting the examination results. Two expert interviews further validate the feasibility of OralCam for promoting users' awareness of oral health.2020YLYuan Liang et al.University of California, Los AngelesMental Health Apps & Online Support CommunitiesTelemedicine & Remote Patient MonitoringCHI