CINXE.COM

<!DOCTYPE html> <html lang="en"> <head> <meta charset="utf-8"> <meta name="viewport" content="width=device-width,initial-scale=1.0, shrink-to-fit=no"> <link rel="apple-touch-icon" sizes="180x180" href="/images/favicon/apple-touch-icon.png"> <link rel="icon" type="image/png" sizes="32x32" href="/images/favicon/favicon-32x32.png"> <link rel="icon" type="image/png" sizes="16x16" href="/images/favicon/favicon-16x16.png"> <link rel="mask-icon" href="/images/favicon/safari-pinned-tab.svg" color="#00695c"> <link rel="shortcut icon" href="/images/favicon/favicon.ico"> <meta name="msapplication-TileColor" content="#da532c"> <meta name="msapplication-config" content="/images/favicon/browserconfig.xml"> <meta name="theme-color" content="#ffffff"> <meta property="og:url" content="https://pair.withgoogle.com" /> <meta property="og:type" content="website" /> <meta property="og:title" content="People + AI Research" /> <meta property="og:description" content="People + AI Research (PAIR) is a multidisciplinary team at Google that explores the human side of AI." /> <meta property="og:image" content="https://pair.withgoogle.com/images/pair.jpg" /> <meta name=”robots” content=”index, follow”> <title>Our Research | PAIR</title> <link rel="stylesheet" href="https://fonts.googleapis.com/icon?family=Material+Icons|Material+Icons+Outlined"> <link rel="stylesheet" href="https://fonts.googleapis.com/css?family=Roboto:400,700,900"> <link rel="stylesheet" href="https://fonts.googleapis.com/css?family=Open+Sans:300" rel="stylesheet"> <link rel="stylesheet" href="/css/material.min.css"> <link rel="stylesheet" href="/css/pair-main.css"> </head> <script async src="https://www.googletagmanager.com/gtag/js?id=UA-138505774-1"></script> <script> window.dataLayer = window.dataLayer || []; function gtag(){dataLayer.push(arguments);} gtag('js', new Date()); gtag('config', 'UA-138505774-1'); </script> <body> <div class="mdl-layout mdl-js-layout mdl-layout--overlay-drawer-button"> <main class="mdl-layout__content"> <nav id="navigation" class="mdl-grid" aria-label="Main site navigation"> <div class="mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone one-col-spacer"></div> <div class=" mdl-cell mdl-cell--3-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="nav-logo-small"> <a href="/"><img src="/images/pair-logo.svg" alt="PAIR Logo"/></a> </div> </div> <div class="nav-links mdl-cell mdl-cell--8-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <a class="nav-guidebook" href="/guidebook/">Guidebook</a> <a class="nav-explorables" href="/explorables/">Explorables</a> <a class="nav-tools" href="/tools/">Tools</a> <a class="nav-research" href="/research/">Research</a> <a class="nav-events" href="/events/">Events</a> <a href="https://medium.com/people-ai-research" target="_blank"><img src="/images/PAIR/logo-medium.png"/></a> </div> </nav> <section id="sub-content"> <div class="single-page-content"> <div class="mdl-grid sub-top"> <div class="mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone one-col-spacer"></div> <div class="sub-top-centered mdl-cell--6-col mdl-cell--6-col-tablet mdl-cell--4-col-phone"> <div class="sub-top-floater"> <div class="display2"> Our Research </div> <p>We conduct human-AI interaction research across the disciplines of computer science, HCI and design. We focus on explainability, interpretability, fairness, data visualization, human-centered AI, and ML for scientific discovery.</p> </div> </div> <div class="sub-top-centered mdl-cell--5-col mdl-cell--4-col-tablet mdl-cell--4-col-phone hideme840"> <div class="sub-top-floater"> <div class="sub-illustration-overlay"><img src='/images/PAIR/pair-sub-research.png'/></div> </div> </div> </div> <div class="full-white-container"> <div class="sub-top-image"><img class="sub-flat-border-top" src="/images/PAIR/inside-flat-bg.png"/></div> <div class="mdl-grid sub-body"> <div class="mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone one-col-spacer"></div> <div class="mdl-cell--10-col mdl-cell--6-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid"> <p style="width: 100%; text-align:right;"> <br/> <a class="research-sorting-link" href="#papers">Papers</a> | <a class="research-sorting-link" href="#interactive-articles-blogposts-and-websites">Interactive articles</a> </p> <h3 id="papers">Papers</h3> <h4 id="2024">2024</h4> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://arxiv.org/abs/2310.12963" target="_blank">AutoMix: Automatically Mixing Language Models</a> <div class="pair-research-details"> Pranjal Aggarwal, Aman Madaan, Ankit Anand, Srividya Pranavi Potharaju, Swaroop Mishra, Pei Zhou, Aditya Gupta, Dheeraj Rajagopal, Karthik Kappaganthu, Yiming Yang, Shyam Upadhyay, Manaal Faruqui, Mausam - NeurIPS 2024 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">ML</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://arxiv.org/abs/2406.12094" target="_blank">Who’s asking? User personas and the mechanics of latent misalignment</a> <div class="pair-research-details"> Asma Ghandeharioun*, Ann Yuan*, Marius Guerard, Emily Reif, Michael A. Lepori, Lucas Dixon - NeurIPS 2024 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">LLM, Interpretability</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://attrib-workshop.cc/" target="_blank">ATTRIB 2024: 2nd Workshop on Attributing Model Behavior at Scale</a> <div class="pair-research-details"> Tolga Bolukbasi, Logan Engstrom, Andrew Ilyas, Sadhika Malladi, Elisa Nguyen, Sung Min Park - NeurIPS 2024 Workshop </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">ML, LLM, Interpretability</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://arxiv.org/abs/2311.09799" target="_blank">How Far Can We Extract Diverse Perspectives from Large Language Models?</a> <div class="pair-research-details"> Shirley Anugrah Hayati, Minhwa Lee, Dheeraj Rajagopal, Dongyeop Kang - EMNLP 2024 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">LLM, ML Fairness</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://arxiv.org/abs/2410.17413" target="_blank">Scalable Influence and Fact Tracing for Large Language Model Pretraining</a> <div class="pair-research-details"> Tyler A. Chang, Dheeraj Rajagopal, Tolga Bolukbasi, Lucas Dixon, Ian Tenney - arXiv 2024-10-22 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">LLM, Interpretability, Data-Centric AI</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://doi.org/10.1109/TVCG.2024.3456354" target="_blank">LLM Comparator: Interactive Analysis of Side-by-Side Evaluation of Large Language Models</a> <div class="pair-research-details"> Minsuk Kahng, Ian Tenney, Mahima Pushkarna, Michael Xieyang Liu, James Wexler, Emily Reif, Krystal Kallarackal, Minsuk Chang, Michael Terry, Lucas Dixon - IEEE VIS 2024 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">LLM, Visualization, Interpretability</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://pair.withgoogle.com/explorables/patchscopes/" target="_blank">Can Large Language Models Explain Their Internal Mechanisms?</a> <div class="pair-research-details"> Nada Hussein*, Asma Ghandeharioun*, Ryan Mullins, Emily Reif, Jimbo Wilson, Nithum Thain, Lucas Dixon - IEEE VIS 2024, VISxAI </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">LLM, Interpretability, Visualization</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://arxiv.org/abs/2404.01361" target="_blank">LLM Attributor: Interactive Visual Attribution for LLM Generation</a> <div class="pair-research-details"> Seongmin Lee, Zijie J. Wang, Aishwarya Chakravarthy, Alec Helbling, ShengYun Peng, Mansi Phute, Duen Horng Chau, Minsuk Kahng - IEEE VIS 2024 Poster </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">LLM, Interpretability, Visualization</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://arxiv.org/abs/2410.07304" target="_blank">The Moral Turing Test: Evaluating Human-LLM Alignment in Moral Decision-Making</a> <div class="pair-research-details"> Basile Garcia, Crystal Qian, Stefano Palminteri - arXiv 2024-10-09 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">ML, LLM</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://colmweb.org/AcceptedPapers.html" target="_blank">Automata-based constraints for language model decoding</a> <div class="pair-research-details"> Terry Koo, Frederick Liu, Luheng He - COLM 2024 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">ML, LLM</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://arxiv.org/pdf/2405.03806" target="_blank">In Situ AI Prototyping: Infusing Multimodal Prompts into Mobile Settings with MobileMaker</a> <div class="pair-research-details"> Michael Xieyang Liu*, Savvas Petridis*, Alex Fiannaca, Vivian Tsai, Michael Terry, Carrie Cai - VL/HCC 2024 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">HCI, LLM</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://aclanthology.org/2024.acl-short.52/" target="_blank">ConstitutionalExperts: Training a Mixture of Principle-based Prompts</a> <div class="pair-research-details"> Savvas Petridis*, Ben Wedin*, Ann Yuan*, James Wexler, Nithum Thain - ACL 2024 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">ML, LLM</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://arxiv.org/abs/2408.00118" target="_blank">Gemma 2: Improving Open Language Models at a Practical Size</a> <div class="pair-research-details"> Gemma Team (including Lucas Dixon, Minsuk Kahng, Ryan Mullins) - arXiv 2024-07-31 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">LLM</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://escholarship.org/uc/item/8kz5787g#main" target="_blank">Can Generative Multimodal Models Count to Ten?</a> <div class="pair-research-details"> Sunayana Rane, Alexander Ku, Jason Baldridge, Ian Tenney, Tom Griffiths, Been Kim - CogSci 2024 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">Text2Image, Interpretability</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://icml.cc/virtual/2024/poster/33565" target="_blank">A Mechanistic Understanding of Alignment Algorithms: A Case Study on DPO and Toxicity</a> <div class="pair-research-details"> Andrew Lee, Xiaoyan Bai, Itamar Pres, Martin Wattenberg, Jonathan K. Kummerfeld, Rada Mihalcea - ICML 2024 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">LLM, ML, Interpretability</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://arxiv.org/abs/2401.06102" target="_blank">Patchscopes: A Unifying Framework for Inspecting Hidden Representations of Language Models</a> <div class="pair-research-details"> Asma Ghandeharioun, Avi Caciularu, Adam Pearce, Lucas Dixon, Mor Geva - ICML 2024 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">LLM, ML, Interpretability</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://arxiv.org/abs/2312.03656" target="_blank">Interpretability Illusions in the Generalization of Simplified Models</a> <div class="pair-research-details"> Dan Friedman, Andrew Lampinen, Lucas Dixon, Danqi Chen, Asma Ghandeharioun - ICML 2024 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">LLM, ML, Interpretability</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://arxiv.org/abs/2402.02992" target="_blank">Decoding-time Realignment of Language Models</a> <div class="pair-research-details"> Tianlin Liu, Shangmin Guo, Leonardo Bianco, Daniele Calandriello, Quentin Berthet, Felipe Llinares, Jessica Hoffmann, Lucas Dixon, Michal Valko, Mathieu Blondel - ICML 2024 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">ML, LLM</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://arxiv.org/abs/2407.14662" target="_blank">Relational Composition in Neural Networks: A Survey and Call to Action</a> <div class="pair-research-details"> Martin Wattenberg, Fernanda B. Viégas - ICML 2024 Mechanistic Interpretability Workshop </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">ML, Interpretability</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://dl.acm.org/doi/abs/10.1145/3643834.3661613" target="_blank">PromptInfuser: How Tightly Coupling AI and UI Design Impacts Designers' Workflows</a> <div class="pair-research-details"> Savvas Petridis, Michael Terry, Carrie J. Cai - DIS 2024 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">HCI, LLM</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://cacm.acm.org/research/lampost-ai-writing-assistance-for-adults-with-dyslexia-using-large-language-models/" target="_blank">LaMPost: AI Writing Assistance for Adults with Dyslexia Using Large Language Models</a> <div class="pair-research-details"> Steven M. Goodman, Erin Buehler, Patrick Clary, Andy Coenen, Aaron Donsbach, Tiffanie N. Horne, Michal Lahav, Robert MacDonald, Rain Breaw Michaels, Ajit Narayanan, Mahima Pushkarna, Joel Riley, Alex Santana, Lei Shi, Rachel Sweeney, Phil Weaver, Ann Yuan, and Meredith Ringel Morris - CACM 2024-06-06 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">LLM, HCI</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://arxiv.org/abs/2403.12075" target="_blank">Adversarial Nibbler: An Open Red-Teaming Method for Identifying Diverse Harms in Text-to-Image Generation</a> <div class="pair-research-details"> Jessica Quaye, Alicia Parrish, Oana Inel, Charvi Rastogi, Hannah Rose Kirk, Minsuk Kahng, Erin van Liemt, Max Bartolo, Jess Tsang, Justin White, Nathan Clement, Rafael Mosquera, Juan Ciro, Vijay Janapa Reddi, Lora Aroyo - FAccT 2024 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">Safety, HCI</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://arxiv.org/abs/2403.08904" target="_blank">Detecting Hallucination and Coverage Errors in Retrieval Augmented Generation for Controversial Topics</a> <div class="pair-research-details"> Tyler Chang, Jessica Hoffmann, Erin van Liemt, Nithum Thain, Kathy Meier-Hellstern, Lucas Dixon, Katrin Tomanek - LREC-COLING 2024 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">LLM, ML, Safety</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://arxiv.org/abs/2402.15350" target="_blank">Farsight: Fostering Responsible AI Awareness During AI Application Prototyping</a> <div class="pair-research-details"> Jay Wang, Chinmay Kulkarni, Lauren Wilcox, Michael Terry, Michael Madaio - CHI 2024 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">HCI, LLM, Safety</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://dl.acm.org/doi/10.1145/3613904.3642325" target="_blank">From Provenance to Aberrations: Image Creator and Screen Reader User Perspectives on Alt Text for AI-Generated Images</a> <div class="pair-research-details"> Maitraye Das, Alexander J. Fiannaca, Meredith Ringel Morris, Shaun K. Kane, Cynthia L. Bennett - CHI 2024 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">HCI</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://arxiv.org/abs/2402.14880" target="_blank">Automatic Histograms: Leveraging Language Models for Text Dataset Exploration</a> <div class="pair-research-details"> Emily Reif, Crystal Qian, James Wexler, Minsuk Kahng - CHI 2024 EA </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">LLM, Visualization</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://arxiv.org/abs/2402.10524" target="_blank">LLM Comparator: Visual Analytics for Side-by-Side of Evaluation of Large Language Models</a> <div class="pair-research-details"> Minsuk Kahng, Ian Tenney, Mahima Pushkarna, Michael Xieyang Liu, James Wexler, Emily Reif, Krystal Kallarackal, Minsuk Chang, Michael Terry, Lucas Dixon - CHI 2024 EA </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">LLM, Visualization, Interpretability, HCI</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://arxiv.org/abs/2402.16611" target="_blank">Understanding the Dataset Practitioners Behind Large Language Model Development</a> <div class="pair-research-details"> Crystal Qian, Emily Reif, Minsuk Kahng - CHI 2024 EA </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">LLM, HCI</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://arxiv.org/abs/2404.07362" target="_blank">”We Need Structured Output”: Towards User-centered Constraints on Large Language Model Output</a> <div class="pair-research-details"> Michael Xieyang Liu, Frederick Liu, Alexander J. Fiannaca, Terry Koo, Lucas Dixon, Michael Terry, Carrie J. Cai - CHI 2024 EA </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">LLM, HCI, User control</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://dl.acm.org/doi/10.1145/3613905.3650948" target="_blank">Designing for Human-Agent Alignment: Understanding what humans want from their agents</a> <div class="pair-research-details"> Tesh Goyal, Minsuk Chang, Michael Terry - CHI 2024 EA </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">HCI, LLM</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://arxiv.org/abs/2405.06079" target="_blank">Believing Anthropomorphism: Examining the Role of Anthropomorphic Cues on Trust in Large Language Models</a> <div class="pair-research-details"> Michelle Cohn, Mahima Pushkarna, Gbolahan O. Olanubi, Joseph M. Moran, Daniel Padgett, Zion Mengesha, Courtney Heldreth - CHI 2024 EA </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">HCI, LLM</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://sites.google.com/corp/view/chi2024-sensemaking-workshop/" target="_blank">Sensemaking: What is it today?</a> <div class="pair-research-details"> Daniel M. Russell, Laura Koesten, Aniket Kittur, Nitesh Goyal, Pengyi Zhang, Michael Xieyang Liu - CHI 2024 Workshop </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">HCI</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://arxiv.org/abs/2403.08904" target="_blank">Detecting Hallucination and Coverage Errors in Retrieval Augmented Generation for Controversial Topics</a> <div class="pair-research-details"> Tyler A. Chang, Katrin Tomanek, Jessica Hoffmann, Nithum Thain, Erin MacMurray van Liemt, Kathleen Meier-Hellstern, Lucas Dixon - LREC-COLING 2024-05-01 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">LLM, hallucination, agile classifiers, low-data regime</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://arxiv.org/abs/2404.07498" target="_blank">Interactive Prompt Debugging with Sequence Salience</a> <div class="pair-research-details"> Ian Tenney, Ryan Mullins, Bin Du, Shree Pandya, Minsuk Kahng, Lucas Dixon - arXiv 2024-04-11 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">LLM, Interpretability, Visualization</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://dl.acm.org/doi/10.1145/3640543.3645198%20" target="_blank">Take It, Leave It, or Fix It: Measuring Productivity and Trust in Human-AI Collaboration</a> <div class="pair-research-details"> Crystal Qian, James Wexler - IUI 2024 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">ML, HCI, LLM, Future of Work</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://dl.acm.org/doi/abs/10.1145/3640543.3645144" target="_blank">ConstitutionMaker: Interactively Critiquing Large Language Models by Converting Feedback into Principles</a> <div class="pair-research-details"> Savvas Petridis, Benjamin D Wedin, James Wexler, Mahima Pushkarna, Aaron Donsbach, Nitesh Goyal, Carrie J Cai, Michael Terry - IUI 2024 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">LLM, HCI</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://arxiv.org/abs/2403.10704" target="_blank">PERL: Parameter Efficient Reinforcement Learning from Human Feedback</a> <div class="pair-research-details"> Hakim Sidahmed, Samrat Phatale, Alex Hutcheson, Zhuonan Lin, Zhang Chen, Zac Yu, Jarvis Jin, Roman Komarytsia, Christiane Ahlheim, Yonghao Zhu, Simral Chaudhary, Bowen Li, Saravanan Ganesh, Bill Byrne, Jessica Hoffmann, Hassan Mansoor, Wei Li, Abhinav Rastogi, Lucas Dixon - arXiv 2024-03-15 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">LLM, low-data regime</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://aclanthology.org/2024.acl-short.52/" target="_blank">ConstitutionalExperts: Training a Mixture of Principle-based Prompts</a> <div class="pair-research-details"> Savvas Petridis, Ben Wedin, Ann Yuan, James Wexler, Nithum Thain - ACL 2024 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">LLM, agile classifiers, Interpretability</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://arxiv.org/abs/2305.13169" target="_blank">A Pretrainer's Guide to Training Data: Measuring the Effects of Data Age, Domain Coverage, Quality, & Toxicity</a> <div class="pair-research-details"> Shayne Longpre, Gregory Yauney, Emily Reif, Katherine Lee, Adam Roberts, Barret Zoph, Denny Zhou, Jason Wei, Kevin Robinson, David Mimno, Daphne Ippolito - NAACL 2024 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840"></div> </div> </div> <h4 id="2023">2023</h4> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://ieeexplore.ieee.org/abstract/document/10360888" target="_blank">Visualizing Linguistic Diversity of Text Datasets Synthesized by Large Language Models</a> <div class="pair-research-details"> Emily Reif, Minsuk Kahng, Savvas Petridis - IEEE VIS 2023 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">HCI, VIS</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://dl.acm.org/doi/abs/10.1145/3604915.3608845" target="_blank">Large language models are competitive near cold-start recommenders for language-and item-based preferences</a> <div class="pair-research-details"> Scott Sanner, Krisztian Balog, Filip Radlinski, Ben Wedin, Lucas Dixon - RecSys 2023 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">LLM, Recommender</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://pair.withgoogle.com/explorables/grokking/" target="_blank">Do Machine Learning Models Memorize or Generalize?</a> <div class="pair-research-details"> Adam Pearce, Asma Ghandeharioun, Nada Hussein, Nithum Thain, Martin Wattenberg, Lucas Dixon - IEEE VIS 2023, VISxAI </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840"></div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://dl.acm.org/doi/abs/10.1145/3563657.3596046" target="_blank">From Discovery to Adoption: Understanding the ML Practitioners’ Interpretability Journey</a> <div class="pair-research-details"> Narges Ashtari, Ryan Mullins, Crystal Qian, James Wexler, Ian Tenney, Mahima Pushkarna - DIS 2023 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">HCI, Interpretability</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://arxiv.org/abs/2305.15498" target="_blank">Large Language Models for User Interest Journeys</a> <div class="pair-research-details"> Konstantina Christakopoulou, Alberto Lalama, Cj Adams, Iris Qu, Yifat Amir, Samer Chucri, Pierce Vollucci, Fabio Soldo, Dina Bseiso, Sarah Scodel, Lucas Dixon, Ed H. Chi, Minmin Chen - arXiv 2023-05-24 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">LLM, HCI</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://arxiv.org/abs/2305.15498" target="_blank">Large Language Models for User Interest Journeys</a> <div class="pair-research-details"> Konstantina Christakopoulou, Alberto Lalama, Cj Adams, Iris Qu, Yifat Amir, Samer Chucri, Pierce Vollucci, Fabio Soldo, Dina Bseiso, Sarah Scodel, Lucas Dixon, Ed H. Chi, Minmin Chen - arXiv 2023-05-24 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">LLM, HCI</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://arxiv.org/abs/2301.12243" target="_blank">Investigating How Practitioners Use Human-AI Guidelines: A Case Study on the People + AI Guidebook</a> <div class="pair-research-details"> Nur Yildirim, Mahima Pushkarna, Nitesh Goyal, Martin Wattenberg, Fernanda Viégas - CHI 2023 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">HCI, AI, UX</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://savvaspetridis.github.io/papers/promptinfuser.pdf" target="_blank">PromptInfuser: Bringing User Interface Mock-ups to Life with Large Language Models</a> <div class="pair-research-details"> Savvas Petridis, Michael Terry, Carrie Cai - CHI 2023 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">LLM, HCI</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://research.google/pubs/pub52262/" target="_blank">Programming without a Programming Language: Challenges and Opportunities for Designing Developer Tools for Prompt Programming</a> <div class="pair-research-details"> Alex Fiannaca, Chinmay Kulkarni, Carrie Cai, Michael Terry - CHI 2023 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">LLM, HCI</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://arxiv.org/abs/2303.12647" target="_blank">A Word is Worth a Thousand Pictures: Prompts as AI Design Material</a> <div class="pair-research-details"> Chinmay Kulkarni, Stefania Druga, Minsuk Chang, Alex Fiannaca, Carrie Cai, Michael Terry - arXiv 2023-03-22 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">HCI, Text2Image</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://arxiv.org/abs/2303.12253" target="_blank">The Prompt Artists</a> <div class="pair-research-details"> Minsuk Chang, Stefania Druga, Alex Fiannaca, Pedro Vergani, Chinmay Kulkarni, Carrie Cai, Michael Terry - arXiv, C&C 2023-03-22 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">HCI, Text2Image, Generative Art</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://ieeexplore.ieee.org/document/10078352" target="_blank">DRIP: Domain Refinement Iteration with Polytopes for Backward Reachability Analysis of Neural Feedback Loops</a> <div class="pair-research-details"> Michael Everett; Rudy Bunel; Shayegan Omidshafiei - IEEE L-CSS 2023-03-22 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">ML, Safety</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://arxiv.org/abs/2302.06541" target="_blank">Towards Agile Text Classifiers for Everyone</a> <div class="pair-research-details"> Maximilian Mozes, Jessica Hoffmann, Katrin Tomanek, Muhamed Kouate, Nithum Thain, Ann Yuan, Tolga Bolukbasi, Lucas Dixon - arXiv 2023-03-14 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">ML, Safety, LLM</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://arxiv.org/abs/2302.06598" target="_blank">Gradient-Based Automated Iterative Recovery for Parameter-Efficient Tuning</a> <div class="pair-research-details"> Maximilian Mozes, Tolga Bolukbasi, Ann Yuan, Frederick Liu, Nithum Thain, Lucas Dixon - arXiv 2023-03-14 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">ML, Interpretability</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://arxiv.org/abs/2303.08114" target="_blank">Simfluence: Modeling the Influence of Individual Training Examples by Simulating Training Runs</a> <div class="pair-research-details"> Kelvin Guu, Albert Webson, Ellie Pavlick, Lucas Dixon, Ian Tenney, Tolga Bolukbasi - arXiv 2023-03-14 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">LLM, Interpretability</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://pair.withgoogle.com/explorables/uncertainty-ood/" target="_blank">From Confidently Incorrect Models to Humble Ensembles</a> <div class="pair-research-details"> Nithum Thain, Adam Pearce, Jasper Snoek and Balaji Lakshminarayanan - 2023-03-01 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">ML</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://arxiv.org/abs/2302.00412" target="_blank">KNNs of Semantic Encodings for Rating Prediction</a> <div class="pair-research-details"> Léo Laugier, Thomas Bonald, Lucas Dixon, Raghuram Vadapalli - arXiv 2023-02-01 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">Recommender, NLP</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://arxiv.org/abs/2301.04518" target="_blank">Large Scale Qualitative Evaluation of Generative Image Model Outputs</a> <div class="pair-research-details"> Yannick Assogba, Adam Pearce, Madison Elliott - arXiv 2023-01-11 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">Interpretability, Visualization</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://arxiv.org/abs/2301.04213" target="_blank">Does Localization Inform Editing? Surprising Differences in Causality-Based Localization vs. Knowledge Editing in Language Models</a> <div class="pair-research-details"> Peter Hase, Mohit Bansal, Been Kim, Asma Ghandeharioun - arXiv 2023-01-10 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">LLM, Interpretability</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://hdl.handle.net/10125/102701" target="_blank">Synthetic APIs: Enabling Language Models to Act as Interlocutors Between Natural Language and Code</a> <div class="pair-research-details"> Ryan Mullins, Michael Terry - HICSS 2023 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">LLM, Program Synthesis, HCI</div> </div> </div> <h4 id="2022">2022</h4> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://arxiv.org/abs/2211.16385" target="_blank">Multi-Agent Reinforcement Learning for Microprocessor Design Space Exploration</a> <div class="pair-research-details"> Srivatsan Krishnan, Natasha Jaques, Shayegan Omidshafiei, Dan Zhang, Izzeddin Gur, Vijay Janapa Reddi, Aleksandra Faust - NeurIPS ML4Sys Workshop 2022 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">ML, RL</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://proceedings.neurips.cc/paper_files/paper/2022/hash/1663fba7b56da1e96bed6e30546a07b0-Abstract-Conference.html" target="_blank">Beyond Rewards: a Hierarchical Perspective on Offline Multiagent Behavioral Analysis</a> <div class="pair-research-details"> Shayegan Omidshafiei, Andrei Kapishnikov, Yannick Assogba, Lucas Dixon, Been Kim - NeurIPS 2022 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">ML, RL, Interpretability</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://pair.withgoogle.com/explorables/saliency/" target="_blank">Searching for Unintended Biases With Saliency</a> <div class="pair-research-details"> Astrid Bertrand, Adam Pearce and Nithum Thain - 2022-12-01 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">ML, ML Fairness, Interpretability</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://openreview.net/forum?id=zt5JpGQ8WhH" target="_blank">Concept-based Understanding of Emergent Multi-Agent Behavior</a> <div class="pair-research-details"> Niko Grupen, Natasha Jaques, Been Kim, Shayegan Omidshafiei - NeurIPS DeepRL Workshop 2022 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">ML, RL</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://www.pnas.org/doi/10.1073/pnas.2206625119" target="_blank">Acquisition of chess knowledge in AlphaZero</a> <div class="pair-research-details"> Thomas McGrath,Andrei Kapishnikov, Nenad Tomaševa, Adam Pearce, Martin Wattenberg, Demis Hassabis, Been Kim, Ulrich Paquet, and Vladimir Kramnik - PNAS 2022-11-14 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">Interpretability</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://pair.withgoogle.com/explorables/federated-learning/" target="_blank">How Federated Learning Protects Privacy</a> <div class="pair-research-details"> Nicole Mitchell and Adam Pearce - 2022-11-01 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">ML, Privacy</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://arxiv.org/abs/2205.00501" target="_blank">Is Your Toxicity My Toxicity? Exploring the Impact of Rater Identity on Toxicity Annotation</a> <div class="pair-research-details"> Tesh Goyal, Ian Kivlichan, Rachel Rosen, Lucy Vasserman - CSCW 2022 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">HCI, ML Fairness, Toxicity, Data-Centric AI</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://doi.org/10.5210/fm.v27i5.12285" target="_blank">Toxicity detection sensitive to conversational context</a> <div class="pair-research-details"> - First Monday 2022-09-05 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">NLP, Toxicity</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://dl.acm.org/doi/pdf/10.1145/3477495.3531873" target="_blank">On natural language user profiles for transparent and scrutable recommendation</a> <div class="pair-research-details"> Filip Radlinski, Krisztian Balog, Fernando Diaz, Lucas Dixon, Ben Wedin - ACM SIGIR 2022 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">LLM, Recommender</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://aclanthology.org/2022.findings-emnlp.180/" target="_blank">Towards Tracing Knowledge in Language Models Back to the Training Data</a> <div class="pair-research-details"> Ekin Akyurek, Tolga Bolukbasi, Frederick Liu, Binbin Xiong, Ian Tenney, Jacob Andreas, Kelvin Guu - Findings of EMNLP 2022 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">LLM, Interpretability</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://research.google/pubs/pub51151/" target="_blank">A Systematic Review and Thematic Analysis of Community-Collaborative Approaches to Computing Research</a> <div class="pair-research-details"> Ned Cooper Tiffanie Horne Gillian Hayes Courtney Heldreth Michal Lahav Jess Scon Holbrook Lauren Wilcox - CHI 2022 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">HCI</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://arxiv.org/pdf/2202.11168.pdf" target="_blank">"You have to prove the threat is real": Understanding the needs of Female Journalists and Activists to Document and Report Online Harassment. </a> <div class="pair-research-details"> Tesh Goyal, Leslie Park, Lucy Vasserman - CHI 2022 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">HCI, Toxicity</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="http://arxiv.org/abs/2203.10748" target="_blank">Whose AI Dream? In search of the aspiration in data annotation</a> <div class="pair-research-details"> Ding Wang, Shantanu Prabhat, Nithya Sambasivan - CHI 2022 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">HCI, Data-Centric AI, Future of Work</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://dl.acm.org/doi/10.1145/3491102.3517533" target="_blank">“Because AI is 100% right and safe”: User Vulnerabilities and Sources of AI Authority in India</a> <div class="pair-research-details"> Shivani Kapania, Oliver Siy, Gabe Clapper, Azhagu SP, Nithya Sambasivan - CHI 2022 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">HCI, ML Fairness</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://dl.acm.org/doi/abs/10.1145/3491102.3517578" target="_blank">Deskilling of Domain Expertise in AI Development</a> <div class="pair-research-details"> Nithya Sambasivan, Rajesh Veeraraghavan - CHI 2022 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">HCI, Future of Work</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://dl.acm.org/doi/10.1145/3491102.3501868" target="_blank">When is ML Data Good? Datafication in Public Health in India</a> <div class="pair-research-details"> Divy Thakkar*, Azra Ismail*, Alex Hanna, Pratyush Kumar, Nithya Sambasivan, Neha Kumar - CHI 2022 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">HCI, Data-Centric AI</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://dl.acm.org/doi/10.1145/3491101.3519729" target="_blank">PromptChainer: Chaining Large Language Model Prompts through Visual Programming</a> <div class="pair-research-details"> Tongshuang Wu, Ellen Jiang, Aaron Donsbach, Jeff Gray, Alejandra Molina, Michael Terry, Carrie J Cai - CHI 2022 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">HCI, LLM</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://dl.acm.org/doi/abs/10.1145/3491102.3517582" target="_blank">AI Chains: Transparent and Controllable Human-AI Interaction by Chaining Large Language Model Prompts</a> <div class="pair-research-details"> Tongshuang Wu, Michael Terry, Carrie J Cai - CHI 2022 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">HCI, LLM</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://dl.acm.org/doi/abs/10.1145/3491102.3501870" target="_blank">Discovering the Syntax and Strategies of Natural Language Programming with Generative Language Models</a> <div class="pair-research-details"> Ellen Jiang, Edwin Toh, Alejandra Molina, Kristen Olson, Claire Kayacik, Aaron Donsbach, Carrie J Cai, Michael Terry - CHI 2022 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">HCI, LLM</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://dl.acm.org/doi/abs/10.1145/3491101.3503564" target="_blank">Prompt-based Prototyping with Large Language Models</a> <div class="pair-research-details"> Ellen Jiang, Kristen Olson, Edwin Toh, Alejandra Molina, Aaron Donsbach, Michael Terry, Carrie J Cai - CHI 2022 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">HCI, LLM</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://arxiv.org/abs/2105.15164" target="_blank">DISSECT: Disentangled Simultaneous Explanations via Concept Traversals</a> <div class="pair-research-details"> Asma Ghandeharioun, Been Kim, Chun-Liang Li, Brendan Jou, Brian Eoff, Rosalind W. Picard - ICLR 2022 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">Interpretability</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://dl.acm.org/doi/10.1145/3531146.3533231" target="_blank">Data Cards: Purposeful and Transparent Dataset Documentation for Responsible AI</a> <div class="pair-research-details"> Mahima Pushkarna, Andrew Zaldivar, Oddur Kjartansson - FAccT 2022 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">HCI, ML Fairness, Data-Centric AI</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://pair.withgoogle.com/explorables/uncertainty-calibration/" target="_blank">Are Model Predictions Probabilities</a> <div class="pair-research-details"> Nithum Thain, Adam Pearce, Jasper Snoek & Mahima Pushkarna - 2022-03-01 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">ML</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://arxiv.org/pdf/2107.07430" target="_blank">Wordcraft: Story Writing With Large Language Models</a> <div class="pair-research-details"> Ann Yuan, Andy Coenen, Emily Reif, Daphne Ippolito - IUI 2022 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">HCI, LLM</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://arxiv.org/abs/2201.11196" target="_blank">IMACS: Image Model Attribution Comparison Summaries</a> <div class="pair-research-details"> Eldon Schoop, Ben Wedin, Andrei Kapishnikov, Tolga Bolukbasi, Michael Terry - arXiv 2022 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">HCI, Interpretability, Visualization</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://arxiv.org/abs/2111.06467" target="_blank">SynthBio: A Case Study in Faster Curation of Text Datasets</a> <div class="pair-research-details"> Ann Yuan, Daphne Ippolito, Vitaly Nikolaev, Chris Callison-Burch, Andy Coenen, Sebastian Gehrmann - NeurIPS Datasets and Benchmarks 2021 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">Data-Centric AI</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://pair.withgoogle.com/explorables/dataset-worldviews/" target="_blank">Datasets Have Worldviews</a> <div class="pair-research-details"> Dylan Baker - 2022-01-01 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">Data-Centric AI, ML Fairness</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://pair.withgoogle.com/explorables/private-and-fair/" target="_blank">Can a Model Be Differentially Private and Fair?</a> <div class="pair-research-details"> Adam Pearce - 2022-01-01 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">ML, Privacy, ML Fairness</div> </div> </div> <h4 id="2021">2021</h4> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://dl.acm.org/doi/10.1145/3506698" target="_blank">Designerly Tele-Experiences: a New Approach to Remote Yet Still Situated Co-Design</a> <div class="pair-research-details"> Ferran Altarriba Bertran, Alexandra Pometko, Muskan Gupta, Lauren Wilcox, Reeta Banerjee, Katherine Isbister - TOCHI 2021-12-13 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">HCI</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://ml4health.github.io/2021/poster_B4.html" target="_blank">Specialized Healthsheet for Healthcare Datasets</a> <div class="pair-research-details"> Negar Rostamzadeh, Subhrajit Roy, Diana Mincu, Andrew Smart, Lauren Wilcox, Mahima Pushkarna, Razvan Amironesei, Jessica Schrouff, Madeleine Elish, Nyalleng Moorosi, Berk Ustun, Noah Broesti, Katherine Heller: - ML4Health 2021 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">ML Dev</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://arxiv.org/abs/2111.10223" target="_blank">Toxicity Detection can be Sensitive to the Conversational Context</a> <div class="pair-research-details"> Alexandros Xenos, John Pavlopoulos, Ion Androutsopoulos, Lucas Dixon, Jeffrey Sorensen, Leo Laugier - arxiv 2021-11-19 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">ML, NLP, Toxicity</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://arxiv.org/abs/2109.03910" target="_blank">A Recipe For Arbitrary Text Style Transfer with Large Language Models</a> <div class="pair-research-details"> Emily Reif, Daphne Ippolito, Ann Yuan, Andy Coenen, Chris Callison-Burch, Jason Wei - ICLR 2022 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">LLM</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://dl.acm.org/doi/abs/10.1145/3474349.3480209" target="_blank">GenLine and GenForm: Two Tools for Interacting with Generative Language Models in a Code Editor</a> <div class="pair-research-details"> Ellen Jiang, Edwin Toh, Alejandra Molina, Aaron Donsbach, Carrie Cai, Michael Terry - UIST 2021 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">HCI</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://arxiv.org/abs/2109.09358" target="_blank">Augmenting the User-Item Graph with Textual Similarity Models</a> <div class="pair-research-details"> Federico López, Martin Scholz, Jessica Yung, Marie Pellat, Michael Strube, Lucas Dixon - arXiv 2021-09-20 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">Recommender</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://distill.pub/2021/gnn-intro/" target="_blank">A Gentle Introduction to Graph Neural Networks</a> <div class="pair-research-details"> Benjamin Sanchez-Lengeling, Emily Reif, Adam Pearce, Alexander B. Wiltschko - Distll 2021-09-02 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">Explorable</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://arxiv.org/abs/2108.07732" target="_blank">Program Synthesis with Large Language Models</a> <div class="pair-research-details"> Jacob Austin, Augustus Odena, Maxwell Nye, Maarten Bosma, Henryk Michalewski, David Dohan, Ellen Jiang, Carrie Cai, Michael Terry, Quoc Le, Charles Sutton - arXiv 2021-08-16 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">Program Synthesis, LLM</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://pair.withgoogle.com/explorables/fill-in-the-blank/" target="_blank">What Have Language Models Learned?</a> <div class="pair-research-details"> Adam Pearce - IEEE VIS 2021, VISxAI </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">LLM, Visualization</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://pair-code.github.io/saliency/#guided-ig" target="_blank">Guided Integrated Gradients: An Adaptive Path Method for Removing Noise</a> <div class="pair-research-details"> Andrei Kapishnikov, Subhashini Venugopalan, Besim Avci, Ben Wedin, Michael Terry, Tolga Bolukbasi - CVPR 2021 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">Interpretability</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://dl.acm.org/doi/abs/10.1145/3411764.3445413" target="_blank">Isolation in Coordination: Challenges of Caregivers in the USA.</a> <div class="pair-research-details"> Mark Schurgin, Mark Schlager, Laura Vardoulakis, Laura Pina, Lauren Wilcox - CHI 2021 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">HCI</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://dl.acm.org/doi/fullHtml/10.1145/3411764.3445192" target="_blank">Breakdowns and Breakthroughs: Observing Musicians' Responses to the COVID-19 Pandemic</a> <div class="pair-research-details"> Carrie Cai, Michelle Carney, Nida Zada, Michael Terry - CHI 2021 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">HCI</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://dl.acm.org/doi/fullHtml/10.1145/3411764.3445219" target="_blank">AI as Social Glue: Uncovering the Roles of Deep Generative AI during Social Music Composition</a> <div class="pair-research-details"> Mia Suh, Emily Youngbloom, Michael Terry, Carrie Cai - CHI 2021 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">HCI</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://dl.acm.org/doi/10.1145/3411763.3443435" target="_blank">Onboarding Materials as Cross-functional Boundary Objects for Developing AI Assistants</a> <div class="pair-research-details"> Cai, Winter, Steiner, Wilcox, Terry - CHI 2021 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">HCI</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://dl.acm.org/doi/10.1145/3411764.3445518" target="_blank">"Everyone wants to do the model work, not the data work": Data Cascades in High-Stakes AI</a> <div class="pair-research-details"> Nithya Sambasivan, Shivani Kapania, Hannah Highfill, Diana Akrong, Praveen Paritosh, Lora Aroyo - CHI 2021 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">HCI, Data-Centric AI</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://dl.acm.org/doi/abs/10.1145/3411764.3445432" target="_blank">“Brilliant AI Doctor” in Rural Clinics: Challenges in AI-Powered Clinical Decision Support System Deployment</a> <div class="pair-research-details"> Dakuo Wang, Liuping Wang, Zhan Zhang, Ding Wang, Haiyi Zhu, Yvonne Gao, Xiangmin Fan, Feng Tian - CHI 2021 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">HCI</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://arxiv.org/abs/2104.07143" target="_blank">An Interpretability Illusion for BERT</a> <div class="pair-research-details"> Tolga Bolukbasi, Adam Pearce, Ann Yuan, Andy Coenen, Emily Reif, Fernanda Viégas, Martin Wattenberg - arXiv 2021-04-14 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">Interpretability</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://arxiv.org/abs/2102.05456" target="_blank">Civil rephrases of toxic texts with self-supervised transformers</a> <div class="pair-research-details"> L Laugier, I Pavlopoulos, J Sorensen, L Dixon - EACL 2021 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">Toxicity, NLP</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://arxiv.org/abs/2101.09995" target="_blank">Re-imagining Algorithmic Fairness in India and Beyond</a> <div class="pair-research-details"> Nithya Sambasivan, Erin Arnesen, Ben Hutchinson, Tulsee Doshi, Vinodkumar Prabhakaran - FAacT 2021 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">HCI, ML Fairness</div> </div> </div> <h4 id="2020">2020</h4> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://nips.cc/virtual/2020/public/poster_417fbbf2e9d5a28a855a11894b2e795a.html" target="_blank">Evaluating Attribution for Graph Neural Networks</a> <div class="pair-research-details"> Sanchez-Lengeling, Wei, Lee, Reif, Wang, Qian, McCloskey, Colwell, Wiltschko - NeurIPS 2020 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">Interpretability</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://arxiv.org/abs/2008.05122" target="_blank">The Language Interpretability Tool: Interactive Exploration, Explanation, and Counterfactual Analysis of NLP models</a> <div class="pair-research-details"> Tenney, Wexler, Bastings, Bolukbasi, Coenen, Gehrmann, Jiang, Pushkarna, Radebaugh, Reif, Yuan - EMNLP 2020 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">Interpretability, Visualization</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://arxiv.org/abs/2012.03659" target="_blank">Non-portability of Algorithmic Fairness in India</a> <div class="pair-research-details"> Sambasivan, Arnesen, Hutchinson, Prabhakaran - NeurIPS 2020 workshop </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">HCI, ML Fairness</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://www.aclweb.org/anthology/2020.alw-1.15.pdf" target="_blank">Six Attributes of Unhealthy Conversation</a> <div class="pair-research-details"> I Price, J Gifford-Moore, J Flemming, S Musker, M Roichman, G Sylvain, L Dixon - WOAH 2020 workshop paper </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">NLP</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://research.google/pubs/pub49764/" target="_blank">Artificial intelligence assistance improves Gleason grading of prostate needle core biopsies</a> <div class="pair-research-details"> David F. Steiner, Kunal Nagpal, Rory Sayres, Davis J. Foote, Benjamin D. Wedin, Adam Pearce, Carrie J. Cai, Samantha R. Winter, Matthew Symonds, Liron Yatziv, Andrei Kapishnikov, Trissia Brown, Isabelle Flament-Auvigne, Fraser Tan, Martin C. Stumpe, Pan-Pan Jiang, Yun Liu, Po-Hsuan Cameron Chen, Greg S. Corrado, Michael Terry, Craig H. Mermel - JAMA Open Network 2020 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">HCI</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://jamanetwork.com/journals/jamanetworkopen/fullarticle/2772831" target="_blank">Evaluation of the Use of Combined Artificial Intelligence and Pathologist Assessment to Review and Grade Prostate Biopsies.</a> <div class="pair-research-details"> Steiner, Nagpal, Sayres, Foote, Wedin, Pearce, Cai, Winter, Symonds, Yatziv, Kapishnikov, Brown, Flament, Tan, Stumpe, Jiang, Liu, Chen, Corrado, Terry, Mermel - JAMA Open Network 2020 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">HCI</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://arxiv.org/abs/2011.05429" target="_blank">Debugging Tests for Model Explanations</a> <div class="pair-research-details"> Julius Adebayo, Michael Muelly, Ilaria Liccardi, Been Kim - NeurIPS / WHI 2020 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">Interpretability</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://pair.withgoogle.com/explorables/anonymization/" target="_blank">How randomized response can help collect sensitive information responsibly</a> <div class="pair-research-details"> Adam Pearce and Ellen Jiang - 2020-09-01 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">ML, Privacy, Visualization</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://interactions.acm.org/archive/view/november-december-2020/beyond-the-portal" target="_blank">Beyond the Portal: Re-imagining the Post-pandemic Future of Work</a> <div class="pair-research-details"> Thakkar, Kumar, Sambasivan - ACM interactions 27(6) 2020 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">HCI</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://arxiv.org/abs/2010.05388" target="_blank">AI Song Contest: Human-AI Co-Creation in Songwriting</a> <div class="pair-research-details"> Cheng-Zhi Anna Huang, Hendrik Vincent Koops, Ed Newton-Rex, Monica Dinculescu, Carrie J. Cai - ISMIR 2020 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">HCI</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://www.microsoft.com/en-us/research/publication/on-the-making-and-breaking-of-social-music-improvisation-during-the-covid-19-pandemic/" target="_blank">On the Making and Breaking of Social Music Improvisation during the COVID-19 Pandemic</a> <div class="pair-research-details"> Carrie J. Cai, Michael Terry - New Future of Work Symposium 2020 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">HCI</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://arxiv.org/abs/2007.04612" target="_blank">Concept Bottleneck Models</a> <div class="pair-research-details"> Pang Wei Koh, Thao Nguyen, Yew Siang Tang, Stephen Mussmann, Emma Pierson, Been Kim, Percy Liang - ICML 2020 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">Interpretability</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://arxiv.org/abs/1903.01069" target="_blank">Neural Networks Trained on Natural Scenes Exhibit Gestalt Closure</a> <div class="pair-research-details"> Kim, Reif, Wattenberg, Bengio, Mozer - Computational Brain & Behavior 2020-06-29 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">Interpretability</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://arxiv.org/abs/2006.00998" target="_blank">Toxicity Detection: Does Context Really Matter?</a> <div class="pair-research-details"> Pavlopoulos, Sorensen, Dixon, Thain, Androutsopoulos - ACL 2020 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">Toxicity</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://pair.withgoogle.com/explorables/hidden-bias/" target="_blank">Hidden Bias</a> <div class="pair-research-details"> Adam Pearce - 2020-05-01 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">ML Fairness, Visualization</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://pair.withgoogle.com/explorables/measuring-fairness/" target="_blank">Measuring Fairness</a> <div class="pair-research-details"> Adam Pearce - 2020-05-01 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">ML Fairness, Visualization</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://dl.acm.org/doi/abs/10.1145/3313831.3376674" target="_blank">Towards an AI-powered Future that Works for Vocational Workers</a> <div class="pair-research-details"> Thakkar, Kumar, Sambasivan - CHI 2020 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">HCI</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://arxiv.org/abs/2004.05476" target="_blank">Classifying Constructive Comments</a> <div class="pair-research-details"> Kolhatkar, Thain, Sorensen, Dixon, Taboada - Special Issue Journal, WWW, ALW, 2020 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">NLP</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://arxiv.org/abs/1910.07969" target="_blank">On Completeness-aware Concept-Based Explanations in Deep Neural Networks</a> <div class="pair-research-details"> Chih-Kuan Yeh, Been Kim, Sercan O. Arik, Chun-Liang Li, Tomas Pfister, Pradeep Ravikumar - NeurIPS 2020 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">Interpretability</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="" target="_blank">Expert Discussions Improve Comprehension of Difficult Cases in Medical Image Assessment</a> <div class="pair-research-details"> Mike Shaekerman, Carrie Cai, Abigail Huang, Rory Sayres - CHI 2020 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">HCI</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="http://youralien.github.io/files/cococo_chi2020_copy.pdf" target="_blank">Novice-AI Music Co-Creation via AI-Steering Tools for Deep Generative Nets</a> <div class="pair-research-details"> Ryan Louie, Andy Coenen, Anna Huang, Michael Terry, Carrie Cai - CHI 2020 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">HCI</div> </div> </div> <h4 id="2019">2019</h4> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://research.google/pubs/pub48985/" target="_blank">AI in Nigeria</a> <div class="pair-research-details"> Courtney Heldreth, Fernanda Viégas, Titi Akinsanmi, Diana Akrong - 2019-12-01 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">HCI</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://arxiv.org/abs/1806.10758" target="_blank">A Benchmark for Interpretability Methods in Deep Neural Networks</a> <div class="pair-research-details"> Sara Hooker, Dumitru Erhan, Pieter-Jan Kindermans, Been Kim - NeurIPS 2019 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">Interpretability</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://dl.acm.org/doi/10.1145/3359206" target="_blank">"Hello AI": Uncovering the Onboarding Needs of Medical Practitioners for Human-AI Collaborative Decision-Making</a> <div class="pair-research-details"> Carrie J. Cai, Samantha Winter, David Steiner, Lauren Wilcox, Michael Terry - CSCW 2019-11-01 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">HCI</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://ojs.aaai.org/index.php/HCOMP/article/view/5280" target="_blank">Human Evaluation of Models Built for Interpretability</a> <div class="pair-research-details"> Isaac Lage, Emily Chen, Jeffrey He, Menaka Narayanan, Been Kim, Samuel Gershman and Finale Doshi-Velez - HCOMP 2019 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">Interpretability</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://drive.google.com/file/d/1O7GIx-Y-0Q_1j4mcnTiQGdD_Pz0aDgVu/view" target="_blank">Software Developers Learning Machine Learning: Motivations, Hurdles, and Desires</a> <div class="pair-research-details"> Carrie J. Cai, Philip J. Guo - VLHCC 2019 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">ML Dev</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://pair-code.github.io/saliency/#xrai" target="_blank">XRAI: Better Attributions Through Regions</a> <div class="pair-research-details"> Andrei Kapishnikov, Tolga Bolukbasi, Fernanda Viégas, Michael Terry - ICCV 2019 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">Interpretability, Visualization</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://www.aclweb.org/anthology/W19-3810/" target="_blank">Debiasing Embeddings for Reduced Gender Bias in Text Classification</a> <div class="pair-research-details"> Flavien Prost, Nithum Thain, Tolga Bolukbasi - ACL 2019-08-02 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">Interpretability</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://arxiv.org/abs/1907.09701" target="_blank">BIM: Towards Quantitative Evaluation of Interpretability Methods with Ground Truth</a> <div class="pair-research-details"> Mengjiao Yang, Been Kim - arXiv 2019-07-23 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">Interpretability</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://arxiv.org/abs/1907.06637" target="_blank">The Bach Doodle: Approachable music composition with machine learning at scale</a> <div class="pair-research-details"> Cheng-Zhi Anna Huang, Curtis Hawthorne, Adam Roberts, Monica Dinculescu, James Wexler, Leon Hong, Jacob Howcroft - arXiv 2019-07-14 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">HCI</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://arxiv.org/abs/1907.04135" target="_blank">The What-If Tool: Interactive Probing of Machine Learning Models</a> <div class="pair-research-details"> James Wexler, Mahima Pushkarna, Tolga Bolukbasi, Martin Wattenberg, Fernanda Viégas, Jimbo Wilson - VIS 2019 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">Interpretability</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://www.nature.com/articles/s41746-019-0131-z" target="_blank">Similar Image Search for Histopathology: SMILY</a> <div class="pair-research-details"> Narayan Hegde, Jason D. Hipp, Yun Liu, Michael Emmert-Buck, Emily Reif, Daniel Smilkov, Michael Terry, Carrie J. Cai, Mahul B. Amin, Craig H. Mermel, Phil Q. Nelson, Lily H. Peng, Greg S. Corrado, Martin C. Stumpe - Nature 2019-06-21 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">HCI</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://arxiv.org/abs/1906.02715" target="_blank">Visualizing and Measuring the Geometry of BERT</a> <div class="pair-research-details"> Andy Coenen, Emily Reif, Ann Yuan, Been Kim, Adam Pearce, Fernanda Viégas, Martin Wattenberg - NeurIPS 2019-06-06 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">Interpretability, Visualization</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://arxiv.org/abs/1903.01069" target="_blank">Do Neural Networks Show Gestalt Phenomena? An Exploration of the Law of Closure</a> <div class="pair-research-details"> Been Kim, Emily Reif, Martin Wattenberg, Samy Bengio - arXiv 2019-03-04 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">Interpretability</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://dl.acm.org/doi/10.1145/3301275.3302289" target="_blank">The Effects of Example-based Explanations in a Machine Learning Interface</a> <div class="pair-research-details"> Carrie J Cai, Jonas Jongejan, Jess Holbrook - IUI 2019 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">HCI</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://arxiv.org/abs/1902.02960" target="_blank">Human-Centered Tools for Coping with Imperfect Algorithms during Medical Decision-Making</a> <div class="pair-research-details"> Carrie J. Cai, Emily Reif, Narayan Hegde, Jason Hipp, Been Kim, Daniel Smilkov, Martin Wattenberg, Fernanda Viégas, Greg S. Corrado, Martin C. Stumpe, Michael Terry - CHI 2019 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">HCI</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://arxiv.org/abs/1902.03129" target="_blank">Towards Automatic Concept-based Explanations</a> <div class="pair-research-details"> Amirata Ghorbani, James Wexler, James Zou, Been Kim - NeurIPS 2019-02-07 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">Interpretability</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://arxiv.org/abs/1901.05350" target="_blank">Tensorflow.js: Machine learning for the web and beyond</a> <div class="pair-research-details"> Daniel Smilkov, Nikhil Thorat, Yannick Assogba, Ann Yuan, Nick Kreeger, Ping Yu, Kangyi Zhang, Shanqing Cai, Eric Nielsen, David Soergel, Stan Bileschi, Michael Terry, Charles Nicholson, Sandeep N. Gupta, Sarah Sirajuddin, D. Sculley, Rajat Monga, Greg Corrado, Fernanda B. Viégas, Martin Wattenberg - SysML 2019-01-16 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">ML Dev</div> </div> </div> <h4 id="2018">2018</h4> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://arxiv.org/abs/1810.10118" target="_blank">Interpreting Black Box Predictions using Fisher Kernels</a> <div class="pair-research-details"> Rajiv Khanna, Been Kim, Joydeep Ghosh, Oluwasanmi Koyejo - AISTATS 2018-10-23 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">Interpretability</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://arxiv.org/abs/1810.05798" target="_blank">ClinicalVis: Supporting Clinical Task-Focused Design Evaluation</a> <div class="pair-research-details"> Marzyeh Ghassemi, Mahima Pushkarna, James Wexler, Jesse Johnson, Paul Varghese - arXiv 2018-10-13 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">Visualization</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://arxiv.org/abs/1810.03292" target="_blank">Sanity Checks for Saliency Maps</a> <div class="pair-research-details"> Julius Adebayo, Justin Gilmer, Michael Muelly, Ian Goodfellow, Moritz Hardt, Been Kim - NeurIPS 2018-10-08 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">Interpretability</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://arxiv.org/abs/1809.01587" target="_blank">GAN Lab: Understanding Complex Deep Generative Models using Interactive Visual Experimentation</a> <div class="pair-research-details"> Minsuk Kahng, Nikhil Thorat, Duen Horng Chau, Fernanda Viégas, Martin Wattenberg - VIS 2018 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">ML Dev, Visualization, Explorable</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://www.nature.com/articles/s41586-018-0438-y" target="_blank">Deep learning of aftershock patterns following large earthquakes</a> <div class="pair-research-details"> Phoebe M. R. DeVries, Fernanda Viégas, Martin Wattenberg & Brendan J. Meade - Nature 2018-08-29 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">Other</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://arxiv.org/abs/1805.11783" target="_blank">To Trust Or Not To Trust A Classifier</a> <div class="pair-research-details"> Heinrich Jiang, Been Kim, Melody Y. Guan, Maya Gupta - NeurIPS 2018-05-30 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">Interpretability</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://arxiv.org/abs/1805.11571" target="_blank">Human-in-the-Loop Interpretability Prior</a> <div class="pair-research-details"> Isaac Lage, Andrew Slavin Ross, Been Kim, Samuel J. Gershman, Finale Doshi-Velez - NeurIPS 2018-05-29 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">HCI</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://www.nature.com/articles/s41746-018-0029-1" target="_blank">Scalable and accurate deep learning with electronic health records</a> <div class="pair-research-details"> Alvin Rajkomar, Eyal Oren, Kai Chen, Andrew M. Dai, Nissan Hajaj, Michaela Hardt, Peter J. Liu, Xiaobing Liu, Jake Marcus, Mimi Sun, Patrik Sundberg, Hector Yee, Kun Zhang, Yi Zhang, Gerardo Flores, Gavin E. Duggan, Jamie Irvine, Quoc Le, Kurt Litsch, Alexander Mossin, Justin Tansuwan, De Wang, James Wexler, Jimbo Wilson, Dana Ludwig, Samuel L. Volchenboum, Katherine Chou, Michael Pearson, Srinivasan Madabushi, Nigam H. Shah, Atul J. Butte, Michael D. Howell, Claire Cui, Greg S. Corrado & Jeffrey Dean - Nature 2018-05-08 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">HCI</div> </div> </div> <h4 id="2017">2017</h4> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://arxiv.org/abs/1711.11279" target="_blank">Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)</a> <div class="pair-research-details"> Been Kim, Martin Wattenberg, Justin Gilmer, Carrie Cai, James Wexler, Fernanda Viégas, Rory Sayres - ICML 2018 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">Interpretability</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://arxiv.org/abs/1708.03788" target="_blank">Direct-Manipulation Visualization of Deep Networks</a> <div class="pair-research-details"> Daniel Smilkov, Shan Carter, D. Sculley, Fernanda B. Viégas, Martin Wattenberg - arXiv 2017-08-12 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">ML Dev, Visualization, Explorable</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://arxiv.org/abs/1706.03825" target="_blank">SmoothGrad: removing noise by adding noise</a> <div class="pair-research-details"> Daniel Smilkov, Nikhil Thorat, Been Kim, Fernanda Viégas, Martin Wattenberg - arXiv 2017-06-12 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">Interpretability</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://idl.cs.washington.edu/files/2018-TensorFlowGraph-VAST.pdf" target="_blank">Visualizing Dataflow Graphs of Deep Learning Models in TensorFlow</a> <div class="pair-research-details"> Kanit Wongsuphasawat, Daniel Smilkov, James Wexler, Jimbo Wilson, Dandelion Mane, Doug Fritz, Dilip Krishnan, Fernanda B. Viégas, and Martin Wattenberg - VIS 2017 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">ML Dev, Visualization</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="http://summit.fas.harvard.edu/files/meade/files/m17.pdf" target="_blank">What Is Better Than Coulomb Failure Stress? A Ranking of Scalar Static Stress Triggering Mechanisms from 105 Mainshock-Aftershock Pairs</a> <div class="pair-research-details"> Brendan J. Meade, Phoebe M. R. DeVries, Jeremy Faller, Fernanda Viégas, Martin Wattenberg - Geophysical Research Letters 2017-01-01 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">Other</div> </div> </div> <h4 id="2016">2016</h4> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://doi.org/10.1785/0220160169" target="_blank">Interactive Visualization of Spatially Amplified GNSS Time-Series Position Fields</a> <div class="pair-research-details"> Brendan J. Meade, William T. Freeman, James Wilson, Fernanda Viégas, and Martin Wattenberg - 2016-11-23 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">Other, Visualization</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://arxiv.org/abs/1611.05469" target="_blank">Embedding Projector: Interactive Visualization and Interpretation of Embeddings</a> <div class="pair-research-details"> Daniel Smilkov, Nikhil Thorat, Charles Nicholson, Emily Reif, Fernanda B. Viégas, Martin Wattenberg - NeurIPS 2016-11-16 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">ML Dev, Visualization, Interpretability</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://arxiv.org/abs/1603.04467" target="_blank">TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems</a> <div class="pair-research-details"> Martín Abadi, Ashish Agarwal, Paul Barham, Eugene Brevdo, Zhifeng Chen, Craig Citro, Greg S. Corrado, Andy Davis, Jeffrey Dean, Matthieu Devin, Sanjay Ghemawat, Ian Goodfellow, Andrew Harp, Geoffrey Irving, Michael Isard, Yangqing Jia, Rafal Jozefowicz, Lukasz Kaiser, Manjunath Kudlur, Josh Levenberg, Dan Mane, Rajat Monga, Sherry Moore, Derek Murray, Chris Olah, Mike Schuster, Jonathon Shlens, Benoit Steiner, Ilya Sutskever, Kunal Talwar, Paul Tucker, Vincent Vanhoucke, Vijay Vasudevan, Fernanda Viégas, Oriol Vinyals, Pete Warden, Martin Wattenberg, Martin Wicke, Yuan Yu, Xiaoqiang Zheng - arXiv 2016-03-16 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">ML Dev</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://research.google/pubs/pub45733/" target="_blank">Google's Multilingual Neural Machine Translation System: Enabling Zero-Shot Translation</a> <div class="pair-research-details"> Melvin Johnson, Mike Schuster, Quoc V. Le, Maxim Krikun, Yonghui Wu, Zhifeng Chen, Nikhil Thorat, Fernanda Viégas, Martin Wattenberg, Greg Corrado, Macduff Hughes, Jeffrey Dean - ACL 2016-01-01 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">Interpretability</div> </div> </div> <h4 id="2014">2014</h4> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://research.google/pubs/pub42901/" target="_blank">Visualizing Statistical Mix Effects and Simpson's Paradox</a> <div class="pair-research-details"> Zan Armstrong, Martin Wattenberg - InfoVis 2014 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">HCI, Visualization</div> </div> </div> <h4 id="2013">2013</h4> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://dl.acm.org/doi/10.1145/2488388.2488504" target="_blank">Google+Ripples: a native visualization of information flow</a> <div class="pair-research-details"> Fernanda Viégas, Martin Wattenberg, Jack Hebert, Geoffrey Borggaard, Alison Cichowlas, Jonathan Feinberg, Jon Orwant, Christopher Wren - WWW 2013-05-01 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">Visualization</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://research.google/pubs/pub41159/" target="_blank">Ad Click Prediction: a View from the Trenches</a> <div class="pair-research-details"> H. Brendan McMahan, Gary Holt, D. Sculley, Michael Young, Dietmar Ebner, Julian Grady, Lan Nie, Todd Phillips, Eugene Davydov, Daniel Golovin, Sharat Chikkerur, Dan Liu, Martin Wattenberg, Arnar Mar Hrafnkelsson, Tom Boulos, Jeremy Kubica - KDD 2013 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">HCI</div> </div> </div> <h4 id="2012">2012</h4> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://dl.acm.org/doi/10.1145/2157136.2157322" target="_blank">Through the looking glass: talking about the world with visualization</a> <div class="pair-research-details"> Fernanda Viégas & Martin Wattenberg - ACM 2012-02-01 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">Visualization</div> </div> </div> <h4 id="2011">2011</h4> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="http://hint.fm/projects/luscious/" target="_blank">Luscious</a> <div class="pair-research-details"> Fernanda Viégas & Martin Wattenberg - 2011-07-01 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">Visualization</div> </div> </div> <h4 id="2010">2010</h4> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://www.oreilly.com/library/view/beautiful-visualization/9781449379889/ch11.html" target="_blank">Beautiful History: Visualizing Wikipedia (Chapter 11)</a> <div class="pair-research-details"> Fernanda Viégas & Martin Wattenberg - O'Reilly 2010-07-01 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">Visualization</div> </div> </div> <h3 id="interactive-articles-blogposts-and-websites">Interactive Articles, Blogposts and Websites</h3> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://pair.withgoogle.com/explorables/patchscopes/" target="_blank">Can Large Language Models Explain Their Internal Mechanisms?</a> <div class="pair-research-details"> Nada Hussein*, Asma Ghandeharioun*, Ryan Mullins, Emily Reif, Jimbo Wilson, Nithum Thain, Lucas Dixon - IEEE VIS 2024, VISxAI </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">LLM, Interpretability, Visualization</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://pair.withgoogle.com/explorables/grokking/" target="_blank">Do Machine Learning Models Memorize or Generalize?</a> <div class="pair-research-details"> Adam Pearce, Asma Ghandeharioun, Nada Hussein, Nithum Thain, Martin Wattenberg, Lucas Dixon - IEEE VIS 2023, VISxAI </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840"></div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://pair.withgoogle.com/explorables/uncertainty-ood/" target="_blank">From Confidently Incorrect Models to Humble Ensembles</a> <div class="pair-research-details"> Nithum Thain, Adam Pearce, Jasper Snoek and Balaji Lakshminarayanan - 2023-03-01 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">ML</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://pair.withgoogle.com/explorables/saliency/" target="_blank">Searching for Unintended Biases With Saliency</a> <div class="pair-research-details"> Astrid Bertrand, Adam Pearce and Nithum Thain - 2022-12-01 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">ML, ML Fairness, Interpretability</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://pair.withgoogle.com/explorables/federated-learning/" target="_blank">How Federated Learning Protects Privacy</a> <div class="pair-research-details"> Nicole Mitchell and Adam Pearce - 2022-11-01 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">ML, Privacy</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://pair.withgoogle.com/explorables/uncertainty-calibration/" target="_blank">Are Model Predictions Probabilities</a> <div class="pair-research-details"> Nithum Thain, Adam Pearce, Jasper Snoek & Mahima Pushkarna - 2022-03-01 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">ML</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://pair.withgoogle.com/explorables/dataset-worldviews/" target="_blank">Datasets Have Worldviews</a> <div class="pair-research-details"> Dylan Baker - 2022-01-01 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">Data-Centric AI, ML Fairness</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://pair.withgoogle.com/explorables/private-and-fair/" target="_blank">Can a Model Be Differentially Private and Fair?</a> <div class="pair-research-details"> Adam Pearce - 2022-01-01 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">ML, Privacy, ML Fairness</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://pair.withgoogle.com/explorables/fill-in-the-blank/" target="_blank">What Have Language Models Learned?</a> <div class="pair-research-details"> Adam Pearce - IEEE VIS 2021, VISxAI </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">LLM, Visualization</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://pair.withgoogle.com/explorables/measuring-diversity/" target="_blank">Measuring Diversity</a> <div class="pair-research-details"> Adam Pearce, Dylan Baker, Ellen Jiang, Meg Mitchell* and Timnit Gebru* - 2021-03-01 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">ML Fairness</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://pair.withgoogle.com/explorables/data-leak/" target="_blank">Why Some Models Leak Data</a> <div class="pair-research-details"> Adam Pearce and Ellen Jiang - 2020-12-01 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">ML, Visualization, Privacy</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://pair.withgoogle.com/explorables/anonymization/" target="_blank">How randomized response can help collect sensitive information responsibly</a> <div class="pair-research-details"> Adam Pearce and Ellen Jiang - 2020-09-01 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">ML, Privacy, Visualization</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://pair-code.github.io/covid19_symptom_dataset/map/" target="_blank">Explore COVID-19 Symptoms Search Trends</a> <div class="pair-research-details"> Adam Pearce - 2020-09-20 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">Visualization</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://pair.withgoogle.com/explorables/hidden-bias/" target="_blank">Hidden Bias</a> <div class="pair-research-details"> Adam Pearce - 2020-05-01 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">ML Fairness, Visualization</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://pair.withgoogle.com/explorables/measuring-fairness/" target="_blank">Measuring Fairness</a> <div class="pair-research-details"> Adam Pearce - 2020-05-01 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">ML Fairness, Visualization</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://pair-code.github.io/interpretability/uncertainty-over-space/" target="_blank">Communicating Model Uncertainty Over Space</a> <div class="pair-research-details"> Adam Pearce - 2020-02-19 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">HCI, Visualization, Explorable</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://pair-code.github.io/interpretability/text-dream/blogpost/" target="_blank">What does BERT dream of? Deep Dreaming with BERT.</a> <div class="pair-research-details"> Alex Bäuerle, James Wexler - 2020-01-14 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">Interpretability, Visualization</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://pair-code.github.io/understanding-umap/" target="_blank">Understanding UMAP</a> <div class="pair-research-details"> Andy Coenen, Adam Pearce - 2019-11-01 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">ML Dev, Visualization, Explorable, Interpretability</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://pair-code.github.io/interpretability/context-atlas/blogpost/" target="_blank">Language, Context, and Geometry in Neural Networks, Part II</a> <div class="pair-research-details"> Andy Coenen, Emily Reif, Ann Yuan, Been Kim, Adam Pearce, Fernanda Viégas, Martin Wattenberg - 2019-09-06 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">Interpretability, Visualization</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://pair-code.github.io/interpretability/bert-tree/" target="_blank">Language, Context, and Geometry in Neural Networks, Part I</a> <div class="pair-research-details"> Andy Coenen, Emily Reif, Ann Yuan, Been Kim, Adam Pearce, Fernanda Viégas, Martin Wattenberg - 2019-08-09 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">Interpretability</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://www.google.com/doodles/celebrating-johann-sebastian-bach" target="_blank">The Bach Doodle: Celebrating Johann Sebastian Bach</a> <div class="pair-research-details"> Cheng-Zhi Anna Huang, Curtis Hawthorne, Adam Roberts, Monica Dinculescu, James Wexler, Leon Hong, Jacob Howcroft - 2019-07-14 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">HCI</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="https://distill.pub/2016/misread-tsne/" target="_blank">How to Use t-SNE Effectively</a> <div class="pair-research-details"> Martin Wattenberg, Fernanda Viégas, Ian Johnson - Distill 2016-10-13 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">ML Dev, Visualization, Explorable</div> </div> </div> <div class="mdl-cell mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid mdl-research-grid"> <div class="pair-research-name mdl-cell mdl-cell--9-col mdl-cell--6-col-tablet mdl-cell--3-col-phone"> <a href="http://hint.fm/projects/luscious/" target="_blank">Luscious</a> <div class="pair-research-details"> Fernanda Viégas & Martin Wattenberg - 2011-07-01 </div> </div> <div class="mdl-cell mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone hideme840"></div> <div class="pair-research-type mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--1-col-phone hideme840">Visualization</div> </div> </div> <p><br/><br/></p> </div> </div> </div> <div class="sub-bot-image"><img class="sub-flat-border-top" src="/images/PAIR/inside-flat-bot-bg.png"/></div> </div> </div> </section> <style> body { background-color: #f0f6f5; } </style> </section> <section id="footer-content"> <div class="single-page-content"> <div class="mdl-grid pair-footer-container"> <div class="mdl-cell--1-col mdl-cell--4-col-tablet mdl-cell--1-col-phone one-col-spacer"></div> <div class="footer-left mdl-cell mdl-cell--7-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="google-logo hideme840"><img alt="Google Logo" src="/images/google-logo.svg"/></div> <a href="https://research.google/" target="_blank">Google Research</a> <a href="https://design.google/library/ai/" target="_blank">Google Design</a> </div> <div class="footer-right mdl-cell mdl-cell--4-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <a href="mailto:peopleai@google.com" target="_blank"><img src="/images/PAIR/mail.png"/></a> <a href="https://github.com/pair-code" target="_blank"><img src="/images/PAIR/github.png"/></a> <a href="https://medium.com/people-ai-research" target="_blank"><img src="/images/PAIR/medium.png"/></a> <a href="https://www.youtube.com/channel/UCnnns-uu4yy9BXfYSPIX5AA" target="_blank"><img src="/images/PAIR/youtube.png"/></a> </div> <div class="google-hidden-logo showme840"><img alt="Google Logo" src="/images/google-logo.svg"/></div> </div> </section> </main> </div> </div> <script src="/js/material.min.js"></script> <script src="/js/smoothscroll.min.js"></script> <script type="text/javascript" src="https://www.gstatic.com/feedback/js/help/prod/service/lazy.min.js"></script> <script src="/js/pair-page.js"></script> <script data-autoload-cookie-consent-bar="true" src="https://www.gstatic.com/brandstudio/kato/cookie_choice_component/cookie_consent_bar.v2.js"></script> </body> </html>