Preprints

Preprint

AURA: LLM Anonymization Against Agentic Re-Identification

Ziwen Li, Jianing Wen, and Tianshi Li

arXiv preprint May 2026

arXiv Code Website
Preprint

Disclose with Care: Designing Privacy Controls in Interview Chatbots

Ziwen Li, Ziang Xiao, and Tianshi Li

arXiv preprint Feb 2026

arXiv
Preprint

Agentic LLMs as Powerful Deanonymizers: Re-identification of Participants in the Anthropic Interviewer Dataset

Tianshi Li

arXiv preprint Jan 2026

📰 Media coverage: MIT Technology Review, POLITICO, The Straits Times, Northeastern Global News

arXiv
Preprint

Position: Privacy Is Not Just Memorization!

Niloofar Mireshghallah†, and Tianshi Li†

arXiv preprint Oct 2025

📰 Media coverage: Northeastern Global News, Help Net Security

arXiv

Publications

2026

COLM

CIDER: A Dataset of Contextual Disclosure Boundaries for Privacy Preference Alignment

Bingcan Guo, Eryue Xu, Jijie Zhou, Zhiping Zhang, and Tianshi Li

In COLM 2026 Oct 2026

Abs arXiv Dataset Code

Aligning large language models (LLMs) with human privacy preferences requires capturing individuals’ disclosure boundaries beyond general privacy norms. However, current models lack methods for eliciting such nuanced preferences and ground truth data to evaluate alignment in realistic settings. We introduce CIDER, a dataset of 14,850 human annotations from 169 users, forming 1,650 contextual disclosure boundary sets across 60 interpersonal communication scenarios where information sharing violates privacy norms. Each boundary represents a real user’s disclosure decisions over 9 sharing variants, given a communication role and AI-mediated condition. We formulate a prediction task in which models predict a user’s disclosure decision from historical boundaries, with varying levels of contextual information. Across eight open and proprietary models, personalization improves performance, with accuracy gains of up to 11.41% using six in-context examples. However, these gains arise from different mechanisms. Larger models such as GPT-5.4 (with medium reasoning effort) and Claude Sonnet 4.6 leverage semantic context to infer user-specific, context-dependent disclosure preferences for more accurate predictions, while smaller models tend to rely on structured heuristics based on disclosure granularity and identifiability and present trade-offs between false positives and false negatives. Our findings highlight the potential and limitations of current LLMs’ privacy modeling capability and position CIDER as a resource for advancing personalized privacy preference alignment.
COLM

Autonomy Reshapes How Personalization Affects Privacy Concerns and Trust in LLM Agents

Zhiping Zhang, Yi Evie Zhang, Freda Shi, and Tianshi Li

In COLM 2026 Oct 2026

Abs arXiv

LLM agents require personal information for personalization in order to effectively act on users’ behalf, but this raises privacy concerns that can discourage data sharing, limiting both the autonomy levels at which agents can operate and the effectiveness of personalization. Yet the expanded design space of agent autonomy also presents opportunities to shape these effects, which remain underexplored. We conducted a 3×3 between-subjects experiment (N=450) to study how agent autonomy level influences personalization’s effects on users’ privacy concerns, trust, and willingness to use, as well as the underlying psychological processes. We find that risk-contingent autonomy, where the agent delegates control to users upon detecting potential privacy leakage, through improving users’ perceived control, attenuates personalization’s adverse effects by reducing the increase in privacy concerns and the decrease in trust. Our results suggest that designing agent’s autonomy that supports human autonomy (both in terms of perceived control and oversight effectiveness) helps users benefit from personalization without being deterred by growing privacy concerns, contributing to the development of trustworthy LLM agents.
PETS

Precarious But Active: A Look At Privacy Behaviors in Chinese Transformative Fandom on a Censored and Surveilled Internet

Kelly Wang, Ruochen Liu, Ada Lerner, Abigail Marsh, and Tianshi Li

In Proceedings on Privacy Enhancing Technologies (PETS) 2026 Jul 2026

Abs PDF

Chinese transformative fandom has had to adapt to increasing censorship and surveillance on the Chinese internet in recent years, working around censorship on domestic platforms in order to continue participating in fandom. To investigate this phenomenon from a privacy perspective, we interviewed 10 overseas members of Chinese transformative fandom about their experiences with privacy and censorship, and we supplemented this with 153 social media comments from Weibo and Xiaohongshu (RedNote) on the same topic. We found that our data perceived the current state of Chinese online fandom as, at best, frustrating, and at worst unsafe. Fans could be discouraged as the platform prevented them from sharing their fanworks while within-fandom disagreements led some fans to silence or report each other. The censored state of Chinese platforms, however, could also make it difficult for the community to learn how to move to a blocked overseas platform. They responded to risks from both the state and their peers by leveraging precarious techniques of obscurity and anonymity, seeking strategies that would still allow engagement with fandom. We identify three key takeaways for privacy scholarship: the harms of censorship were felt at a community level, which created a tension with expected privacy solutions; faced with inevitable surveillance, fans nonetheless actively modeled threats as a community to inform their behaviors; and the sociotechnical environment of fans influenced how blocking and reporting other fans seemed necessary for curation, contributing to why they might expose each other to state-level harm.
DIS

PrivacyMotiv: Speculative Persona Journeys for Empathic and Motivating Privacy Reviews in UX Design

Zeya Chen, Jianing Wen, Yaxing Yao, Toby Jia-Jun Li, and Tianshi Li

In DIS 2026 Jun 2026

Abs arXiv Demo

UX professionals routinely conduct design reviews, yet privacy concerns are often overlooked—not only due to limited tools, but more fundamentally from low intrinsic motivation, driven by limited privacy knowledge, weak empathy for unexpectedly affected users, and low confidence in identifying harms. We present PrivacyMotiv, an LLM-powered system that supports privacy-oriented design diagnosis by generating speculative personas with UX user journeys centered on individuals vulnerable to privacy risks. Drawing on narrative strategies, the system constructs relatable and attention-drawing scenarios that show how ordinary design choices may cause unintended harms, expanding the scope of privacy reflection in UX. In a within-subjects study with professional UX practitioners (N=16), we compared participants’ self-proposed methods with PrivacyMotiv across two privacy audit tasks. Results show significant improvements in empathy, intrinsic motivation, and perceived usefulness. This work contributes empirical insight into motivational barriers in privacy-aware UX and a novel mechanism for structured privacy audits.
TAISAP

The Obvious Invisible Threat: LLM-Powered GUI Agents’ Vulnerability to Fine-Print Injections

Chaoran Chen, Zhiping Zhang, Bingcan Guo, Shang Ma, Ibrahim Khalilov, Simret A Gebreegziabher, Yanfang Ye, Ziang Xiao, and 3 more authors

In ACM Transactions on AI Security and Privacy (TAISAP) 2026 Apr 2026

📰 Media coverage: New Scientist

Abs arXiv

A Large Language Model (LLM) powered GUI agent is a specialized autonomous system that performs tasks on the user’s behalf according to high-level instructions. It does so by perceiving and interpreting the graphical user interfaces (GUIs) of relevant apps, often visually, inferring necessary sequences of actions, and then interacting with GUIs by executing the actions such as clicking, typing, and tapping. To complete real-world tasks, such as filling forms or booking services, GUI agents often need to process and act on sensitive user data. However, this autonomy introduces new privacy and security risks. Adversaries can inject malicious content into the GUIs that alters agent behaviors or induces unintended disclosures of private information. These attacks often exploit the discrepancy between visual saliency for agents and human users, or the agent’s limited ability to detect violations of contextual integrity in task automation. In this paper, we characterized six types of such attacks, and conducted an experimental study to test these attacks with six state-of-the-art GUI agents, 234 adversarial webpages, and 39 human participants. Our findings suggest that GUI agents are highly vulnerable, particularly to contextually embedded threats. Moreover, human users are also susceptible to many of these attacks, indicating that simple human oversight may not reliably prevent failures. This misalignment highlights the need for privacy-aware agent design. We propose practical defense strategies to inform the development of safer and more reliable GUI agents.
ICLR

Operationalizing Data Minimization for Privacy-Preserving LLM Prompting

Jijie Zhou, Niloofar Mireshghallah, and Tianshi Li

In ICLR 2026 Apr 2026

Abs arXiv Code Website

The rapid deployment of large language models (LLMs) in consumer applications has led to frequent exchanges of personal information. To obtain useful responses, users often share more than necessary, increasing privacy risks via memorization, context-based personalization, or security breaches. We present a framework to formally define and operationalize data minimization: for a given user prompt and response model, quantifying the least privacy-revealing disclosure that maintains utility, and propose a priority-queue tree search to locate this optimal point within a privacy-ordered transformation space. We evaluated the framework on four datasets spanning open-ended conversations (ShareGPT, WildChat) and knowledge-intensive tasks with single-ground-truth answers (CaseHOLD, MedQA), quantifying achievable data minimization with nine LLMs as the response model. Our results demonstrate that larger frontier LLMs can tolerate stronger data minimization while maintaining task quality than smaller open-source models (85.7% redaction for GPT-5 vs. 19.3% for Qwen2.5-0.5B). By comparing with our search-derived benchmarks, we find that LLMs struggle to predict optimal data minimization directly, showing a bias toward abstraction that leads to oversharing. This suggests not just a privacy gap, but a capability gap: models may lack awareness of what information they actually need to solve a task.
CHI

From Fragmentation to Integration: Exploring the Design Space of AI Agents for Human-as-the-Unit Privacy Management

Eryue Xu, and Tianshi Li

In CHI 2026 Apr 2026

Abs arXiv

Managing one’s digital footprint is overwhelming, as it spans multiple platforms and involves countless context-dependent decisions. Recent advances in agentic AI offer ways forward by enabling holistic, contextual privacy-enhancing solutions. Building on this potential, we adopted a “human-as-the-unit” perspective and investigated users’ cross-context privacy challenges through 12 semi-structured interviews. Results reveal that people rely on ad hoc manual strategies while lacking comprehensive privacy controls, highlighting nine privacy-management challenges across applications, temporal contexts, and relationships. To explore solutions, we generated nine AI agent concepts and evaluated them via a speed-dating survey with 116 US participants. The three highest-ranked concepts were all post-sharing management tools with half or full agent autonomy, with users expressing greater trust in AI accuracy than in their own efforts. Our findings highlight a promising design space where users see AI agents bridging the fragments in privacy management, particularly through automated, comprehensive post-sharing remediation of users’ digital footprints.
CHI

Dark Patterns Meet GUI Agents: LLM Agent Susceptibility to Manipulative Interfaces and the Role of Human Oversight

Jingyu Tang†, Chaoran Chen†, Jiawen Li, Zhiping Zhang, Bingcan Guo, Ibrahim Khalilov, Simret Araya Gebreegziabher, Bingsheng Yao, and 6 more authors

In CHI 2026 Apr 2026

📰 Media coverage: Northeastern Khoury News

Abs arXiv

The dark patterns, deceptive interface designs manipulating user behaviors, have been extensively studied for their effects on human decision-making and autonomy. Yet, with the rising prominence of LLM-powered GUI agents that automate tasks from high-level intents, understanding how dark patterns affect agents is increasingly important. We present a two-phase empirical study examining how agents, human participants, and human-AI teams respond to 16 types of dark patterns across diverse scenarios. Phase 1 highlights that agents often fail to recognize dark patterns, and even when aware, prioritize task completion over protective action. Phase 2 revealed divergent failure modes: humans succumb due to cognitive shortcuts and habitual compliance, while agents falter from procedural blind spots. Human oversight improved avoidance but introduced costs such as attentional tunneling and cognitive load. Our findings show neither humans nor agents are uniformly resilient, and collaboration introduces new vulnerabilities, suggesting design needs for transparency, adjustable autonomy, and oversight.
CHI

Exploring Collaboration Breakdowns Between Provider Teams and Patients in Post-Surgery Care

Bingsheng Yao†, Menglin Zhao†, Zhan Zhang, Pengqi Wang, Emma G Chester, Changchang Yin, Tianshi Li, Varun Mishra, and 6 more authors

In CHI 2026 Apr 2026

Abs arXiv

Post-surgery care involves ongoing collaboration between provider teams and patients, which starts from post-surgery hospitalization through home recovery after discharge. While prior HCI research has primarily examined patients’ challenges at home, less is known about how provider teams coordinate discharge preparation and care handoffs, and how breakdowns in communication and care pathways may affect patient recovery. To investigate this gap, we conducted semi-structured interviews with 13 healthcare providers and 4 patients in the context of gastrointestinal (GI) surgery. We found coordination boundaries between in- and out-patient teams, coupled with complex organizational structures within teams, impeded the invisible work of preparing patients’ home care plans and triaging patient information. For patients, these breakdowns resulted in inadequate preparation for home transition and fragmented self-collected data, both of which undermine timely clinical decision-making. Based on these findings, we outline design opportunities to formalize task ownership and handoffs, contextualize co-temporal signals, and align care plans with home resources.
TOSEM

“Should I Give Up Now?” Investigating LLM Pitfalls in Software Engineering

Jiessie Tie, Bingsheng Yao, Tianshi Li, Hongbo Fang, Syed Ishtiaque Ahmed, Dakuo Wang, and Shurui Zhou

In ACM Transactions on Software Engineering and Methodology Mar 2026

Abs PDF

Software engineers are increasingly incorporating AI assistants into their workflows to enhance productivity and alleviate cognitive load. However, experiences with large language models (LLMs) such as ChatGPT vary widely. While some engineers find them useful, others deem them counterproductive due to inaccuracies in their responses. Researchers have also observed that ChatGPT often provides incorrect information. Given these limitations, it is crucial to determine how to effectively integrate LLMs into software engineering (SE) workflow. Analyzing data from 26 participants in a complex web development task, we identified nine failure types categorized into incorrect or incomplete responses, cognitive overload, and context loss. Users attempted to mitigate these issues through scaffolding, prompt clarification, and debugging. However, 17 participants ultimately chose to abandon ChatGPT due to persistent failures. Our quantitative analysis revealed that unhelpful responses increased the likelihood of abandonment by a factor of 11, while each additional prompt reduced abandonment probability by 17%. This study advances the understanding of human-AI interaction in SE tasks and outlines directions for future research and tooling support.
NDSS

From Perception to Protection: A Developer-Centered Study of Security and Privacy Threats in Extended Reality (XR)

Kunlin Cai, Jinghuai Zhang, Ying Li, Zhiyuan Wang, Xun Chen, Tianshi Li, and Yuan Tian

In NDSS 2026 Feb 2026

arXiv

2025

CSCW

Secret Use of Large Language Models

Zhiping Zhang, Chenxinran Shen, Bingsheng Yao, Dakuo Wang, and Tianshi Li

In CSCW 2025 Oct 2025

Abs arXiv

The advancements of Large Language Models (LLMs) have decentralized the responsibility for the transparency of AI usage. Specifically, LLM users are now encouraged or required to disclose the use of LLM-generated content for varied types of real-world tasks. However, an emerging phenomenon, users’ secret use of LLMs, raises challenges in ensuring end users adhere to the transparency requirement. Our study used mixed-methods with an exploratory survey (125 real-world secret use cases reported) and a controlled experiment among 300 users to investigate the contexts and causes behind the secret use of LLMs. We found that such secretive behavior is often triggered by certain tasks, transcending demographic and personality differences among users. Task types were found to affect users’ intentions to use secretive behavior, primarily through influencing of perceived external judgment regarding LLM usage. Our results yield important insights for future work on designing interventions to encourage more transparent disclosure of LLM/AI use
CSCW

“I’m categorizing LLM as a productivity tool”: Examining ethics of LLM use in HCI research practices

Shivani Kapania, Ruiyi Wang, Toby Li, Tianshi Li, and Hong Shen

In CSCW 2025 Oct 2025

arXiv
UIST

Why am I seeing this: Democratizing End User Auditing for Online Content Recommendations

Chaoran Chen, Leyang Li, Luke Cao, Yanfang Ye, Tianshi Li, Yaxing Yao, and Toby Jia-Jun Li

In UIST 2025 Sep 2025

Abs arXiv

Personalized recommendation systems tailor content based on user attributes, which are either provided or inferred from private data. Research suggests that users often hypothesize about reasons behind contents they encounter (e.g., “I see this jewelry ad because I am a woman”), but they lack the means to confirm these hypotheses due to the opaqueness of these systems. This hinders informed decision-making about privacy and system use and contributes to the lack of algorithmic accountability. To address these challenges, we introduce a new interactive sandbox approach. This approach creates sets of synthetic user personas and corresponding personal data that embody realistic variations in personal attributes, allowing users to test their hypotheses by observing how a website’s algorithms respond to these personas. We tested the sandbox in the context of targeted advertisement. Our user study demonstrates its usability, usefulness, and effectiveness in empowering end-user auditing in a case study of targeting ads.
CHI

Rescriber: Smaller-LLM-Powered User-Led Data Minimization for LLM-Based Chatbots

Jijie Zhou, Eryue Xu, Yaoyao Wu, and Tianshi Li

In CHI 2025 Apr 2025

Abs arXiv PDF Code

The proliferation of LLM-based conversational agents has resulted in excessive disclosure of identifiable or sensitive information. However, existing technologies fail to offer perceptible control or account for users’ personal preferences about privacy-utility tradeoffs due to the lack of user involvement. To bridge this gap, we designed, built, and evaluated Rescriber, a browser extension that supports user-led data minimization in LLM-based conversational agents by helping users detect and sanitize personal information in their prompts. Our studies (N=12) showed that Rescriber helped users reduce unnecessary disclosure and addressed their privacy concerns. Users’ subjective perceptions of the system powered by Llama3-8B were on par with that by GPT-4o. The comprehensiveness and consistency of the detection and sanitization emerge as essential factors that affect users’ trust and perceived protection. Our findings confirm the viability of smaller-LLM-powered, user-facing, on-device privacy controls, presenting a promising approach to address the privacy and trust challenges of AI.
CHI

GenieWizard: Multimodal App Feature Discovery with Large Language Models

Jackie (Junrui) Yang, Yingtian Shi, Chris Gu, Zhang Zheng, Anisha Jain, Tianshi Li, Monica Lam, and James A. Landay

In CHI 2025 Apr 2025

Abs

Multimodal interactions are more flexible, efficient, and adaptable than graphical interactions, allowing users to execute commands beyond simply tapping GUI buttons. However, the flexibility of multimodal commands makes it hard for designers to prototype and provide design specs for developers. It’s also hard for developers to anticipate what actions users may want. We present GenieWizard, a tool to aid developers in discovering potential features to implement in multimodal interfaces. GenieWizard supports user-desired command discovery early in the implementation process, streamlining development. GenieWizard uses an LLM to generate potential user interactions and parse these interactions into a form that can be used to discover the missing features for developers. Our evaluations showed that GenieWizard can reliably simulate user interactions and identify missing features. In a study (N=12), we demonstrated that developers using GenieWizard can identify and implement 42% of the missing features of multimodal apps compared to only 10% without GenieWizard.
HAIPS

Privi: Assisting Users in Authoring Contextual Privacy Rules with an LLM Sandbox

Bingcan Guo, Zhiping Zhang, and Tianshi Li

In HAIPS ’25: Proceedings of the 2025 Workshop on Human-Centered AI Privacy and Security (co-located with CCS’25) Apr 2025

Abs PDF

Aligning language models (LM) with individual users’ latent preferences and internal values, such as privacy considerations, is crucial for enhancing output quality and preventing unwanted privacy leakage. Yet, existing methods struggle to capture individuals’ contextualized privacy preferences and formalize them in an extensible and generalizable way to guide model outputs. As an initial exploration, we present Privi, an interactive elicitation mechanism that generates synthetic communication scenarios, leverages users’ edits of candidate responses to infer privacy preferences, and formalizes them in an extensible privacy rule set. We conducted a within-subjects pilot study (N = 15) to evaluate Privi and the quality of the elicited privacy rules. Results show that responses generated under three conditions: pre-specified rules, elicited rules, no rules (model judgment), were comparable across three key evaluation dimensions: amount of privacy disclosure, perceived utility, and willingness to use. We further analyzed the synthetic scenarios and users’ editing behavior and identified future directions for improving Privi.
HAIPS

Speculating Unintended Creepiness: Exploring LLM-Powered Empathy Building for Privacy-Aware UX Design

Zeya Chen, Jianing Wen, Toby Jia-Jun Li, Yaxing Yao, and Tianshi Li

In HAIPS ’25: Proceedings of the 2025 Workshop on Human-Centered AI Privacy and Security (co-located with CCS’25) Apr 2025

Abs PDF

Despite increasing awareness of dark patterns and anti-patterns in UX design, privacy invasive design choices remain prevalent in real world systems. These choices often stem not from malicious intent, but from a lack of structured guidance and contextual understanding among designers. Designers face challenges not only in detecting deceptive interactions, but also in anticipating how certain features may cause harm. Contributing factors such as limited ability to recognize harm, lack of relatable design references, and challenges in connecting abstract privacy principles to concrete design scenarios, particularly when designing for non-dominant user groups. To address this, while currently implemented as a pipeline, PrivacyMotiv is envisioned as a future system that integrates user personas, journey maps, and design audits into a unified tool to help designers identify privacy harms and dark patterns. Grounded in motivation theory and contextual design thinking, our approach supports reasoning across multiple user-feature interactions situated in real world scenarios, with the goal of revealing hidden risks and inspiring designer empathy to promote privacy-aware design for everyone.
HAIPS

Beyond Permissions: Investigating Mobile Personalization with Simulated Personas

Ibrahim Khalilov, Chaoran Chen, Ziang Xiao, Tianshi Li, Toby Jia-Jun Li, and Yaxing Yao

In HAIPS ’25: Proceedings of the 2025 Workshop on Human-Centered AI Privacy and Security (co-located with CCS’25) Apr 2025

Abs PDF

Mobile applications increasingly rely on sensor data to infer user context and deliver personalized experiences. Yet, the mechanisms behind this personalization remain opaque to users and researchers alike. This paper presents a sandbox system that uses sensor spoofing and persona simulation to audit and visualize how mobile apps respond to inferred behaviors. Rather than treating spoofing as adversarial, we demonstrate its use as a tool for behavioral transparency and user empowerment. Our system injects multi-sensor profiles—generated from structured, lifestyle-based personas—into Android devices in real time, enabling users to observe app responses to contexts such as high activity, location shifts, or time-of-day changes. With automated screenshot capture and GPT-4 Vision-based UI summarization, our pipeline helps document subtle personalization cues. Preliminary findings show measurable app adaptations across fitness, e-commerce, and everyday service apps such as weather and navigation. We offer this toolkit as a foundation for privacy-enhancing technologies and user-facing transparency interventions.

2024

HCOMP

Investigating What Factors Influence Users’ Rating of Harmful Algorithmic Bias and Discrimination

Sara Kingsley†, Jiayin Zhi†, Wesley Hanwen Deng, Jaimie Lee, Sizhe Zhang, Motahhare Eslami*, Kenneth Holstein*, Jason I. Hong*, and 2 more authors

In HCOMP 2024 Oct 2024

Best Paper Award 🏆

PDF
NeurIPS

PrivacyLens: Evaluating Privacy Norm Awareness of Language Models in Action

Yijia Shao, Tianshi Li, Weiyan Shi, Yanchen Liu, and Diyi Yang

In NeurIPS Datasets and Benchmarks Sep 2024

arXiv Code Website
USENIX Security

A New Hope: Contextual Privacy Policies for Mobile Applications And an Approach Toward Automated Generation

Shidong Pan, Zhen Tao, Thong Hoang, Dawen Zhang, Tianshi Li, Zhenchang Xing, Xiwei Xu, Mark Staples, and 2 more authors

In USENIX Security 2024 Sep 2024

arXiv
CHI SIG

Human-Centered Privacy Research in the Age of Large Language Models

Tianshi Li, Sauvik Das, Hao-Ping (Hank) Lee, Dakuo Wang, Bingsheng Yao, and Zhiping Zhang

In CHI Conference on Human Factors in Computing Systems (CHI’24 Companion) Apr 2024

Abs arXiv

The emergence of large-language models (LLMs), and their increased use in user-facing systems, has led to substantial privacy concerns. To date, research on these privacy concerns has been model-centered: exploring how LLMs lead to privacy risks like memorization, or can be used to infer personal characteristics about people from their content. We argue that there is a need for more research focusing on the human aspect of these privacy issues: e.g., research on how design paradigms for LLMs affect users’ disclosure behaviors, users’ mental models and preferences for privacy controls, and the design of tools, systems, and artifacts that empower end-users to reclaim ownership over their personal data. To build usable, efficient, and privacy-friendly systems powered by these models with imperfect privacy properties, our goal is to initiate discussions to outline an agenda for conducting human-centered research on privacy issues in LLM-powered systems. This Special Interest Group (SIG) aims to bring together researchers with backgrounds in usable security and privacy, human-AI collaboration, NLP, or any other related domains to share their perspectives and experiences on this problem, to help our community establish a collective understanding of the challenges, research opportunities, research methods, and strategies to collaborate with researchers outside of HCI.
CHI

“It’s a Fair Game”, or Is It? Examining How Users Navigate Disclosure Risks and Benefits When Using LLM-Based Conversational Agents

Zhiping Zhang, Michelle Jia, Hao-Ping (Hank) Lee, Bingsheng Yao, Sauvik Das, Ada Lerner, Dakuo Wang, and Tianshi Li

In CHI Conference on Human Factors in Computing Systems Apr 2024

Abs arXiv PDF Talk Website

The widespread use of Large Language Model (LLM)-based conversational agents (CAs), especially in high-stakes domains, raises many privacy concerns. Building ethical LLM-based CAs that respect user privacy requires an in-depth understanding of the privacy risks that concern users the most. However, existing research, primarily model-centered, does not provide insight into users’ perspectives. To bridge this gap, we analyzed sensitive disclosures in real-world ChatGPT conversations and conducted semi-structured interviews with 19 LLM-based CA users. We found that users are constantly faced with trade-offs between privacy, utility, and convenience when using LLM-based CAs. However, users’ erroneous mental models and the dark patterns in system design limited their awareness and comprehension of the privacy risks. Additionally, the human-like interactions encouraged more sensitive disclosures, which complicated users’ ability to navigate the trade-offs. We discuss practical design guidelines and the needs for paradigmatic shifts to protect the privacy of LLM-based CA users.
CHI

ReactGenie: A Development Framework for Complex Multimodal Interactions Using Large Language Models

Jackie Junrui Yang, Yingtian Shi, Yuhan Zhang, Karina Li, Daniel Wan Rosli, Anisha Jain, Shuning Zhang, Tianshi Li, and 2 more authors

In CHI Conference on Human Factors in Computing Systems Apr 2024

Abs PDF

Multimodal interactions have been shown to be more flexible, efficient, and adaptable for diverse users and tasks than traditional graphical interfaces. However, existing multimodal development frameworks either do not handle the complexity and compositionality of multimodal commands well or require developers to write a substantial amount of code to support these multimodal interactions. In this paper, we present ReactGenie, a programming framework that uses a shared object-oriented state abstraction to support building complex multimodal mobile applications. Having different modalities share the same state abstraction allows developers using ReactGenie to seamlessly integrate and compose these modalities to deliver multimodal interaction.
IMWUT

Matcha: An IDE Plugin for Creating Accurate Privacy Nutrition Labels

Tianshi Li, Lorrie Faith Cranor, Yuvraj Agarwal, and Jason I Hong

Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. Apr 2024

Abs arXiv HTML Website

Apple and Google introduced their versions of privacy nutrition labels to the mobile app stores to better inform users of the apps’ data practices. However, these labels are self-reported by developers and have been found to contain many inaccuracies due to misunderstandings of the label taxonomy. In this work, we present Matcha, an IDE plugin that uses automated code analysis to help developers create accurate Google Play data safety labels. Developers can benefit from Matcha’s ability to detect user data accesses and transmissions while staying in control of the generated label by adding custom Java annotations and modifying an auto-generated XML specification. Our evaluation with 12 developers showed that Matcha helped our participants improved the accuracy of a label they created with Google’s official tool for a real-world app they developed. We found that participants preferred Matcha for its accuracy benefits. Drawing on Matcha, we discuss general design recommendations for developer tools used to create accurate standardized privacy notices.

2023

CSCW SIG

Shaping the Emerging Norms of Using Large Language Models in Social Computing Research

Hong Shen, Tianshi Li, Toby Jia-jun Li, Joon Sung Park, and Diyi Yang

In Computer Supported Cooperative Work and Social Computing (CSCW’23 Companion) Oct 2023

Abs HTML PDF

The emergence of Large Language Models (LLMs) has brought both excitement and concerns to social computing research. On the one hand, LLMs offer unprecedented capabilities in analyzing vast amounts of textual data and generating human-like responses, enabling researchers to delve into complex social phenomena. On the other hand, concerns are emerging regarding the validity, privacy, and ethics of the research when LLMs are involved. This SIG aims at offering an open space for social computing researchers who are interested in understanding the impacts of LLMs to discuss their current practices, perspectives, challenges when engaging with LLMs in their everyday work and collectively shaping the emerging norms of using LLMs in social computing research.
SOUPS Poster

Measuring the Effectiveness of Spinner-based Randomized-Response Differential Privacy Communication for Sensitive Data Sharing

Seoyoung Ko, Sriram Viswanathan, Alan Esquenazi, Swadhin Routray, Jatan Loya, Tianshi Li, and Lorrie Faith Cranor

In SOUPS 2023 Poster Aug 2023

HTML
SOUPS Poster

Analyzing Developers’ Perceptions, Attitudes and Challenges in Complying with Mobile App Store Privacy Requirements

Geetika Gopi, Peter Meyers, Thanyanun Charoensiritanasin, Jayson Jin, Ragashree Mysuru Chandrashekar, Lorrie Faith Cranor, and Tianshi Li

In SOUPS 2023 Poster Aug 2023

HTML
PhD Dissertation

Privacy Annotations: Designing Privacy Support for Developers

Tianshi Li

Aug 2023

Abs PDF

While data has driven many technological advancements, the ubiquitous collec- tion and sharing of data have caused a privacy trust crisis in our society. Developers play a crucial role in creating apps that respect user expectations and data usage norms, as they have a deep understanding of app behavior and can adjust the design accordingly. However, developers are not privacy experts. Developing a privacy- friendly app is often a challenging task due to their lack of 1) awareness of privacy issues, 2) knowledge of privacy best practices, and 3) time for handling privacy re- quirements. These problems have become more and more salient with the advent of a flurry of privacy requirements from platform providers (e.g., Google Play and Apple App Store) and laws (e.g., GDPR, CCPA), creating urgent needs for effective, opportune, and usable privacy support for developers. Hence, I propose Privacy Support for Developers as a new area of interest at the intersection of privacy, HCI, and software engineering research. The first challenge is that although there has been some research on developers’ challenges for handling privacy requirements, they tend to be more descriptive than prescriptive. Therefore, our community still lacks a clear direction of how to solve the problems. To fill in this gap, I first synthesize developers’ needs for designing privacy-enhancing devel- oper support based on my work and past literature to provide a roadmap for future explorations into this problem. Informed by the identified needs, I present my work that pioneers a novel type of developer tooling: Privacy-Enhancing Integrated Development Environment (IDE) Plugins. I propose privacy annotation, a type of structured metadata that embeds pri- vacy information such as data use purposes directly in code. Based on this concept, I designed, implemented, and evaluated three plugins for Android Studio, the official IDE for Android development, to increase developers’ awareness and knowledge of privacy best practices and to reduce the work required for complying with privacy requirements. With one set of annotations, my tools offer privacy support in multiple aspects, including 1) detection of sensitive API calls and third-party SDKs to support accurate understanding, documentation, and disclosure of data practices, 2) just-in- time reminders and lightweight code repair features (quick-fixes) to help develop- ers conform to best practices, and 3) annotation-based declarative programming to generate in-app privacy notices and privacy nutrition labels required by app stores. My studies demonstrated that my tools effectively improved developers’ awareness and adoption of privacy best practices, reduced the workload for completing privacy compliance tasks, and enhanced the accuracy of the generated privacy notices.

2022

PETS

Charting app developers’ journey through privacy regulation features in ad networks

Mohammad Tahaei, Kopo M Ramokapane, Tianshi Li, Jason I Hong, and Awais Rashid

Proc. Priv. Enhancing Technol. Jul 2022

Abs HTML PDF

Mobile apps enable ad networks to collect and track users. App developers are given “configurations” on these platforms to limit data collection and adhere to privacy regulations; however, the prevalence of apps that violate privacy regulations because of third parties, including ad networks, begs the question of how developers work through these configurations and how easy they are to utilize. We study privacy regulations-related interfaces on three widely used ad networks using two empirical studies, a systematic review and think-aloud sessions with eleven developers, to shed light on how ad networks present privacy regulations and how usable the provided configurations are for developers. We find that information about privacy regulations is scattered in several pages, buried under multiple layers, and uses terms and language developers do not understand. While ad networks put the burden of complying with the regulations on developers, our participants, on the other hand, see ad networks responsible for ensuring compliance with regulations. To assist developers in building privacy regulations-compliant apps, we suggest dedicating a section to privacy, offering easily accessible configurations (both in graphical and code level), building testing systems for privacy regulations, and creating multimedia materials such as videos to promote privacy values in the ad networks’ documentation.
TOCHI

C-PAK: Correcting and completing variable-length Prefix-based Abbreviated Keystrokes

Tianshi Li, Philip Quinn, and Shumin Zhai

ACM Trans. Comput. Hum. Interact. Jul 2022

Abs HTML PDF

Improving keystroke savings is a long-term goal of text input research. We present a study into the design space of an abbreviated style of text input called C-PAK (Correcting and completing variable-length Prefix-based Abbreviated Keystrokes) for text entry on mobile devices. Given a variable length and potentially inaccurate input string (e.g. “li g t m”), C-PAK aims to expand it into a complete phrase (e.g. “looks good to me”). We develop a C-PAK prototype keyboard, PhraseWriter , based on a current state-of-the-art mobile keyboard consisting of 1.3 million n-grams and 164,000 words. Using computational simulations on a large dataset of realistic input text, we found that, in comparison to conventional single-word suggestions, PhraseWriter improves the maximum keystroke savings rate by 6.7% (from \(46.3% \) to \(49.4% \) ), reduces the word error rate by 14.7%, and is particularly advantageous for common phrases. We conducted a lab study of novice user behavior and performance which found that users could quickly utilize the C-PAK style abbreviations implemented in PhraseWriter, achieving a higher keystroke savings rate than forward suggestions (25% vs. 16%). Furthermore, they intuitively and successfully abbreviated more with common phrases. However, users had a lower overall text entry rate due to their limited experience with the system (28.5 words per minute vs. 37.7). We outline future technical directions to improve C-PAK over the PhraseWriter baseline, and further opportunities to study the perceptual, cognitive, and physical action trade-offs that underlie the learning curve of C-PAK systems.
CHI

Understanding Challenges for Developers to Create Accurate Privacy Nutrition Labels

Tianshi Li, Kayla Reiman, Yuvraj Agarwal, Lorrie Faith Cranor, and Jason I Hong

In CHI Conference on Human Factors in Computing Systems Apr 2022

Best Paper Honorable Mention Award 🏅

📰 Media coverage: The Washington Post, Consumer Reports

Abs HTML PDF

Apple announced the introduction of app privacy details to their App Store in December 2020, marking the first ever real-world, large-scale deployment of the privacy nutrition label concept, which had been introduced by researchers over a decade earlier. The Apple labels are created by app developers, who self-report their app’s data practices. In this paper, we present the first study examining the usability and understandability of Apple’s privacy nutrition label creation process from the developer’s perspective. By observing and interviewing 12 iOS app developers about how they created the privacy label for a real-world app that they developed, we identified common challenges for correctly and efficiently creating privacy labels. We discuss design implications both for improving Apple’s privacy label design and for future deployment of other standardized privacy notices.
PETS

Understanding privacy-related advice on Stack Overflow

Mohammad Tahaei, Tianshi Li, and Kami Vaniea

Proc. Priv. Enhancing Technol. Apr 2022

Abs HTML PDF

Abstract Privacy tasks can be challenging for developers, resulting in privacy frameworks and guidelines from the research community which are designed to assist developers in considering privacy features and applying privacy enhancing technologies in early stages of software development. However, how developers engage with privacy design strategies is not yet well understood. In this work, we look at the types of privacy-related advice developers give each other and how that advice maps to Hoepman’s privacy design strategies. We qualitatively analyzed 119 privacy-related accepted answers on Stack Overflow from the past five years and extracted 148 pieces of advice from these answers. We find that the advice is mostly around compliance with regulations and ensuring confidentiality with a focus on the inform, hide, control, and minimize of the Hoepman’s privacy design strategies. Other strategies, abstract, separate, enforce, and demonstrate, are rarely advised. Answers often include links to official documentation and online articles, highlighting the value of both official documentation and other informal materials such as blog posts. We make recommendations for promoting the under-stated strategies through tools, and detail the importance of providing better developer support to handle third-party data practices.
CHI LBW

Understanding iOS privacy nutrition labels: An exploratory large-scale analysis of app store data

Yucheng Li, Deyuan Chen, Tianshi Li, Yuvraj Agarwal, Lorrie Faith Cranor, and Jason I Hong

In CHI Conference on Human Factors in Computing Systems Extended Abstracts Apr 2022

HTML PDF
TOCHI

Alert now or never: Understanding and predicting notification preferences of smartphone users

Tianshi Li, Julia Katherine Haines, Miguel Flores Ruiz Eguino, Jason I Hong, and Jeffrey Nichols

ACM Trans. Comput. Hum. Interact. Feb 2022

Abs HTML PDF

Notifications are an indispensable feature of mobile devices, but their delivery can interrupt and distract users. Prior work has examined interventions, such as deferring notification delivery to opportune moments, but has not systematically studied how users might prefer an intelligent system to manage their notifications. Hence, we directly probed Android smartphone users’ notification preferences via a one-week experience-sampling study ( N = 35). We found that users prefer mitigating undesired interruptions by suppressing alerts over deferring them and referred to notification content factors more frequently than contextual factors for explaining their preferences. Then we demonstrated the challenges and potentials of leveraging user actions to help predict notification preferences. Specifically, we showed that a model personalized using user actions achieved a performance gain of 39% than a generic model. This improvement is similar to the 42% performance gain using labels solicited from the user while using observable user actions causes no extra disruption.

2021

IMWUT

Honeysuckle: Annotation-Guided Code Generation of In-App Privacy Notices

Tianshi Li, Elijah B Neundorfer, Yuvraj Agarwal, and Jason I Hong

Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. Sep 2021

Abs HTML PDF Talk Demo

In-app privacy notices can help smartphone users make informed privacy decisions. However, they are rarely used in real-world apps, since developers often lack the knowledge, time, and resources to design and implement them well. We present Honeysuckle, a programming tool that helps Android developers build in-app privacy notices using an annotation-based code generation approach facilitated by an IDE plugin, a build system plugin, and a library. We conducted a within-subjects study with 12 Android developers to evaluate Honeysuckle. Each participant was asked to implement privacy notices for two popular open-source apps using the Honeysuckle library as a baseline as well as the annotation-based approach. Our results show that the annotation-based approach helps developers accomplish the task faster with significantly lower cognitive load. Developers preferred the annotation-based approach over the library approach because it was much easier to learn and use and allowed developers to achieve various types of privacy notices using a unified code format, which can enhance code readability and benefit team collaboration.
PMC

What makes people install a COVID-19 contact-tracing app? Understanding the influence of app design and individual difference on contact-tracing app adoption intention

Tianshi Li, Camille Cobb, Jackie (junrui) Yang, Sagar Baviskar, Yuvraj Agarwal, Beibei Li, Lujo Bauer, and Jason I Hong

Pervasive Mob. Comput. Aug 2021

Best Research Paper 2019-2021 Award 🏆

Abs HTML PDF

Smartphone-based contact-tracing apps are a promising solution to help scale up the conventional contact-tracing process. However, low adoption rates have become a major issue that prevents these apps from achieving their full potential. In this paper, we present a national-scale survey experiment (N=1963) in the U.S. to investigate the effects of app design choices and individual differences on COVID-19 contact-tracing app adoption intentions. We found that individual differences such as prosocialness, COVID-19 risk perceptions, general privacy concerns, technology readiness, and demographic factors played a more important role than app design choices such as decentralized design vs. centralized design, location use, app providers, and the presentation of security risks. Certain app designs could exacerbate the different preferences in different sub-populations which may lead to an inequality of acceptance to certain app design choices (e.g., developed by state health authorities vs. a large tech company) among different groups of people (e.g., people living in rural areas vs. people living in urban areas). Our mediation analysis showed that one’s perception of the public health benefits offered by the app and the adoption willingness of other people had a larger effect in explaining the observed effects of app design choices and individual differences than one’s perception of the app’s security and privacy risks. With these findings, we discuss practical implications on the design, marketing, and deployment of COVID-19 contact-tracing apps in the U.S.
arXiv

The design of the user interfaces for Privacy Enhancements for Android

Jason I Hong, Yuvraj Agarwal, Matt Fredrikson, Mike Czapik, Shawn Hanna, Swarup Sahoo, Judy Chun, Won-Woo Chung, and 17 more authors

Apr 2021

Abs arXiv

We present the design and design rationale for the user interfaces for Privacy Enhancements for Android (PE for Android). These UIs are built around two core ideas, namely that developers should explicitly declare the purpose of why sensitive data is being used, and these permission-purpose pairs should be split by first party and third party uses. We also present a taxonomy of purposes and ways of how these ideas can be deployed in the existing Android ecosystem.
CSCW

How Developers Talk About Personal Data and What It Means for User Privacy: A Case Study of a Developer Forum on Reddit

Tianshi Li, Elizabeth Louie, Laura Dabbish, and Jason I Hong

Proc. ACM Hum. Comput. Interact. Jan 2021

Abs HTML PDF Talk

While online developer forums are major resources of knowledge for application developers, their roles in promoting better privacy practices remain underexplored. In this paper, we conducted a qualitative analysis of a sample of 207 threads (4772 unique posts) mentioning different forms of personal data from the /r/androiddev forum on Reddit. We started with bottom-up open coding on the sampled posts to develop a typology of discussions about personal data use and conducted follow-up analyses to understand what types of posts elicited in-depth discussions on privacy issues or mentioned risky data practices. Our results show that Android developers rarely discussed privacy concerns when talking about a specific app design or implementation problem, but often had active discussions around privacy when stimulated by certain external events representing new privacy-enhancing restrictions from the Android operating system, app store policies, or privacy laws. Developers often felt these restrictions could cause considerable cost yet fail to generate any compelling benefit for themselves. Given these results, we present a set of suggestions for Android OS and the app store to design more effective methods to enhance privacy, and for developer forums(e.g., /r/androiddev) to encourage more in-depth privacy discussions and nudge developers to think more about privacy.

2020

arXiv

Decentralized is not risk-free: Understanding public perceptions of privacy-utility trade-offs in COVID-19 contact-tracing apps

Tianshi Li, Jackie, Yang, Cori Faklaris, Jennifer King, Yuvraj Agarwal, Laura Dabbish, and Jason I Hong

May 2020

Abs arXiv

Contact-tracing apps have potential benefits in helping health authorities to act swiftly to halt the spread of COVID-19. However, their effectiveness is heavily dependent on their installation rate, which may be influenced by people’s perceptions of the utility of these apps and any potential privacy risks due to the collection and releasing of sensitive user data (e.g., user identity and location). In this paper, we present a survey study that examined people’s willingness to install six different contact-tracing apps after informing them of the risks and benefits of each design option (with a U.S.-only sample on Amazon Mechanical Turk, N=208). The six app designs covered two major design dimensions (centralized vs decentralized, basic contact tracing vs. also providing hotspot information), grounded in our analysis of existing contact-tracing app proposals. Contrary to assumptions of some prior work, we found that the majority of people in our sample preferred to install apps that use a centralized server for contact tracing, as they are more willing to allow a centralized authority to access the identity of app users rather than allowing tech-savvy users to infer the identity of diagnosed users. We also found that the majority of our sample preferred to install apps that share diagnosed users’ recent locations in public places to show hotspots of infection. Our results suggest that apps using a centralized architecture with strong security protection to do basic contact tracing and providing users with other useful information such as hotspots of infection in public places may achieve a high adoption rate in the U.S.
CHI EA

Analyzing the Role of General App Creators in Protecting the Privacy of Vulnerable Populations

Tianshi Li, and Jason I. Hong

Apr 2020

2019

SIGMETRICS

Demystifying Complex Workload–DRAM Interactions: An Experimental Study

Saugata Ghose, Tianshi Li, Nastaran Hajinazar, Damla Senol Cali, and Onur Mutlu

Proc. ACM meas. anal. comput. syst. Dec 2019

HTML PDF

2018

IMWUT

Coconut: An IDE Plugin for Developing Privacy-Friendly Apps

Tianshi Li, Yuvraj Agarwal, and Jason I Hong

Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. Dec 2018

Abs HTML PDF Demo Website

Although app developers are responsible for protecting users’ privacy, this task can be very challenging. In this paper, we present Coconut, an Android Studio plugin that helps developers handle privacy requirements by engaging developers to think about privacy during the development process and providing real-time feedback on potential privacy issues. We start by presenting new findings based on a series of semi-structured interviews with Android developers, probing into the difficulties with privacy that developers face when building apps. Based on these findings, we implemented a proof-of-concept prototype of Coconut and evaluated it in a controlled lab study with 18 Android developers (including eight professional developers). Our study results suggest that apps developed with Coconut handled privacy concerns better, and the developers that used Coconut had a better understanding of their code’s behavior and wrote a better privacy policy for their app. We also found that requiring developers to do a small amount of annotating work regarding their apps’ personal data practices during the development process may result in a significant improvement in app privacy.

2017

arXiv

Using ECC DRAM to adaptively increase memory capacity

Yixin Luo, Saugata Ghose, Tianshi Li, Sriram Govindan, Bikash Sharma, Bryan Kelly, Amirali Boroumand, and Onur Mutlu

Jun 2017

Abs arXiv

Modern DRAM modules are often equipped with hardware error correction capabilities, especially for DRAM deployed in large-scale data centers, as process technology scaling has increased the susceptibility of these devices to errors. To provide fast error detection and correction, error-correcting codes (ECC) are placed on an additional DRAM chip in a DRAM module. This additional chip expands the raw capacity of a DRAM module by 12.5%, but the applications are unable to use any of this extra capacity, as it is used exclusively to provide reliability for all data. In reality, there are a number of applications that do not need such strong reliability for all their data regions (e.g., some user batch jobs executing on a public cloud), and can instead benefit from using additional DRAM capacity to store extra data. Our goal in this work is to provide the additional capacity within an ECC DRAM module to applications when they do not need the high reliability of error correction. In this paper, we propose Capacity- and Reliability-Adaptive Memory (CREAM), a hardware mechanism that adapts error correcting DRAM modules to offer multiple levels of error protection, and provides the capacity saved from using weaker protection to applications. For regions of memory that do not require strong error correction, we either provide no ECC protection or provide error detection using multibit parity. We evaluate several layouts for arranging the data within ECC DRAM in these reduced-protection modes, taking into account the various trade-offs exposed from exploiting the extra chip. Our experiments show that the increased capacity provided by CREAM improves performance by 23.0% for a memory caching workload, and by 37.3% for a commercial web search workload executing production query traces. In addition, CREAM can increase bank-level parallelism within DRAM, offering further performance improvements.

2016

SIGMETRICS

Understanding Latency Variation in Modern DRAM Chips: Experimental Characterization, Analysis, and Optimization.

Kevin K Chang, Abhijith Kashyap, Hasan Hassan, Saugata Ghose, Kevin Hsieh, Donghyuk Lee, Tianshi Li, Gennady Pekhimenko, and 2 more authors

In Proceedings of the 2016 ACM SIGMETRICS International Conference on Measurement and Modeling of Computer Science Jun 2016

HTML PDF