CINXE.COM

<!DOCTYPE html> <html lang="en"> <head> <meta charset="utf-8"> <meta name="viewport" content="width=device-width,initial-scale=1.0, shrink-to-fit=no"> <link rel="apple-touch-icon" sizes="180x180" href="/images/favicon/apple-touch-icon.png"> <link rel="icon" type="image/png" sizes="32x32" href="/images/favicon/favicon-32x32.png"> <link rel="icon" type="image/png" sizes="16x16" href="/images/favicon/favicon-16x16.png"> <link rel="mask-icon" href="/images/favicon/safari-pinned-tab.svg" color="#00695c"> <link rel="shortcut icon" href="/images/favicon/favicon.ico"> <meta name="msapplication-TileColor" content="#da532c"> <meta name="msapplication-config" content="/images/favicon/browserconfig.xml"> <meta name="theme-color" content="#ffffff"> <meta property="og:url" content="https://pair.withgoogle.com" /> <meta property="og:type" content="website" /> <meta property="og:title" content="People + AI Research" /> <meta property="og:description" content="People + AI Research (PAIR) is a multidisciplinary team at Google that explores the human side of AI." /> <meta property="og:image" content="https://pair.withgoogle.com/images/pair.jpg" /> <meta name=”robots” content=”index, follow”> <title>AI Explorables | PAIR</title> <link rel="stylesheet" href="https://fonts.googleapis.com/icon?family=Material+Icons|Material+Icons+Outlined"> <link rel="stylesheet" href="https://fonts.googleapis.com/css?family=Roboto:400,700,900"> <link rel="stylesheet" href="https://fonts.googleapis.com/css?family=Open+Sans:300" rel="stylesheet"> <link rel="stylesheet" href="/css/material.min.css"> <link rel="stylesheet" href="/css/pair-main.css"> </head> <script async src="https://www.googletagmanager.com/gtag/js?id=UA-138505774-1"></script> <script> window.dataLayer = window.dataLayer || []; function gtag(){dataLayer.push(arguments);} gtag('js', new Date()); gtag('config', 'UA-138505774-1'); </script> <body> <div class="mdl-layout mdl-js-layout mdl-layout--overlay-drawer-button"> <main class="mdl-layout__content"> <nav id="navigation" class="mdl-grid" aria-label="Main site navigation"> <div class="mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone one-col-spacer"></div> <div class=" mdl-cell mdl-cell--3-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="nav-logo-small"> <a href="/"><img src="/images/pair-logo.svg" alt="PAIR Logo"/></a> </div> </div> <div class="nav-links mdl-cell mdl-cell--8-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <a class="nav-guidebook" href="/guidebook/">Guidebook</a> <a class="nav-explorables" href="/explorables/">Explorables</a> <a class="nav-tools" href="/tools/">Tools</a> <a class="nav-research" href="/research/">Research</a> <a class="nav-events" href="/events/">Events</a> <a href="https://medium.com/people-ai-research" target="_blank"><img src="/images/PAIR/logo-medium.png"/></a> </div> </nav> <section id="sub-content"> <div class="single-page-content"> <div class="mdl-grid sub-top"> <div class="mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone one-col-spacer"></div> <div class="sub-top-centered mdl-cell--6-col mdl-cell--6-col-tablet mdl-cell--4-col-phone"> <div class="sub-top-floater"> <div class="display2"> AI Explorables </div> <h4 id="big-ideas-in-machine-learning-simply-explained">Big ideas in machine learning, simply explained</h4> <p>The rapidly increasing usage of machine learning raises complicated questions: How can we tell if models are fair? Why do models make the predictions that they do? What are the privacy implications of feeding enormous amounts of data into models?</p> <p>This ongoing series of interactive essays will walk you through these important concepts.</p> </div> </div> <div class="sub-top-centered mdl-cell--5-col mdl-cell--4-col-tablet mdl-cell--4-col-phone hideme840"> <div class="sub-top-floater"> <div class="sub-illustration-overlay"><img src='/images/PAIR/pair-sub-explorables.png'/></div> </div> </div> </div> <div class="full-white-container"> <div class="sub-top-image"><img class="sub-flat-border-top" src="/images/PAIR/inside-flat-bg.png"/></div> <div class="mdl-grid sub-body"> <div class="mdl-cell--1-col mdl-cell--1-col-tablet mdl-cell--1-col-phone one-col-spacer"></div> <div class="mdl-cell--10-col mdl-cell--6-col-tablet mdl-cell--4-col-phone"> <div class="mdl-grid"> <div class='pair-third-card pair-card mdl-cell mdl-cell--4-col mdl-cell--8-col-tablet mdl-cell--4-col-phone explorable-card'> <div class="pair-card-image pair-third-card-image"> <a href="/explorables/patchscopes/" target="_blank"><img src="https://pair.withgoogle.com/explorables/images/patching-up-close.png"/></a> </div> <div class="pair-card-type"></div> <div class="pair-card-title pair-third-card-title"><h3><a href="/explorables/patchscopes/" target="_blank">Can Large Language Models Explain Their Internal Mechanisms?</a></h3></div> <div class="pair-card-content pair-third-card-content"> <p>An introduction to Patchscopes, an inspection framework for explaining the hidden representations of LLMs, with LLMs.</p> </div> <div class="pair-card-cta"> <a href="/explorables/patchscopes/" target="_blank"><img src="/images/PAIR/arrow-link-out.png" class="arrow-out"/>Start exploring</a> </div> </div> <div class='pair-third-card pair-card mdl-cell mdl-cell--4-col mdl-cell--8-col-tablet mdl-cell--4-col-phone explorable-card'> <div class="pair-card-image pair-third-card-image"> <a href="/explorables/grokking/" target="_blank"><img src="https://pair.withgoogle.com/explorables/images/grokking-abstract.png"/></a> </div> <div class="pair-card-type"></div> <div class="pair-card-title pair-third-card-title"><h3><a href="/explorables/grokking/" target="_blank">Do Machine Learning Models Memorize or Generalize?</a></h3></div> <div class="pair-card-content pair-third-card-content"> <p>An introduction to grokking and mechanistic interpretability.</p> </div> <div class="pair-card-cta"> <a href="/explorables/grokking/" target="_blank"><img src="/images/PAIR/arrow-link-out.png" class="arrow-out"/>Start exploring</a> </div> </div> <div class='pair-third-card pair-card mdl-cell mdl-cell--4-col mdl-cell--8-col-tablet mdl-cell--4-col-phone explorable-card'> <div class="pair-card-image pair-third-card-image"> <a href="/explorables/fill-in-the-blank/" target="_blank"><img src="https://pair.withgoogle.com/explorables/images/fill-in-the-blank-abstract.png"/></a> </div> <div class="pair-card-type"></div> <div class="pair-card-title pair-third-card-title"><h3><a href="/explorables/fill-in-the-blank/" target="_blank">What Have Language Models Learned?</a></h3></div> <div class="pair-card-content pair-third-card-content"> <p>By asking language models to fill in the blank, we can probe their understanding of the world.</p> </div> <div class="pair-card-cta"> <a href="/explorables/fill-in-the-blank/" target="_blank"><img src="/images/PAIR/arrow-link-out.png" class="arrow-out"/>Start exploring</a> </div> </div> <div class='pair-third-card pair-card mdl-cell mdl-cell--4-col mdl-cell--8-col-tablet mdl-cell--4-col-phone explorable-card'> <div class="pair-card-image pair-third-card-image"> <a href="/explorables/uncertainty-ood/" target="_blank"><img src="https://pair.withgoogle.com/explorables/images/uncertainty-ood-abstract.png"/></a> </div> <div class="pair-card-type"></div> <div class="pair-card-title pair-third-card-title"><h3><a href="/explorables/uncertainty-ood/" target="_blank">Confidently Incorrect Models to Humble Ensembles</a></h3></div> <div class="pair-card-content pair-third-card-content"> <p>ML models sometimes make confidently incorrect predictions when they encounter out of distribution data. Ensembles of models can make better predictions by averaging away mistakes.</p> </div> <div class="pair-card-cta"> <a href="/explorables/uncertainty-ood/" target="_blank"><img src="/images/PAIR/arrow-link-out.png" class="arrow-out"/>Start exploring</a> </div> </div> <div class='pair-third-card pair-card mdl-cell mdl-cell--4-col mdl-cell--8-col-tablet mdl-cell--4-col-phone explorable-card'> <div class="pair-card-image pair-third-card-image"> <a href="/explorables/anonymization/" target="_blank"><img src="/images/PAIR/explorable-anonymization.png"/></a> </div> <div class="pair-card-type"></div> <div class="pair-card-title pair-third-card-title"><h3><a href="/explorables/anonymization/" target="_blank">Collecting Sensitive Information </a></h3></div> <div class="pair-card-content pair-third-card-content"> <p>The availability of giant datasets and faster computers is making it harder to collect and study private information without inadvertently violating people’s privacy.</p> </div> <div class="pair-card-cta"> <a href="/explorables/anonymization/" target="_blank"><img src="/images/PAIR/arrow-link-out.png" class="arrow-out"/>Start exploring</a> </div> </div> <div class='pair-third-card pair-card mdl-cell mdl-cell--4-col mdl-cell--8-col-tablet mdl-cell--4-col-phone explorable-card'> <div class="pair-card-image pair-third-card-image"> <a href="/explorables/data-leak/" target="_blank"><img src="https://pair.withgoogle.com/explorables/images/model-inversion.png"/></a> </div> <div class="pair-card-type"></div> <div class="pair-card-title pair-third-card-title"><h3><a href="/explorables/data-leak/" target="_blank">Why Some Models Leak Data</a></h3></div> <div class="pair-card-content pair-third-card-content"> <p>Machine learning models use large amounts of data, some of which can be sensitive. If they’re not trained correctly, sometimes that data is inadvertently revealed.</p> </div> <div class="pair-card-cta"> <a href="/explorables/data-leak/" target="_blank"><img src="/images/PAIR/arrow-link-out.png" class="arrow-out"/>Start exploring</a> </div> </div> <div class='pair-third-card pair-card mdl-cell mdl-cell--4-col mdl-cell--8-col-tablet mdl-cell--4-col-phone explorable-card'> <div class="pair-card-image pair-third-card-image"> <a href="/explorables/private-and-fair/" target="_blank"><img src="https://pair.withgoogle.com/explorables/images/private-and-fair-abstract.png"/></a> </div> <div class="pair-card-type"></div> <div class="pair-card-title pair-third-card-title"><h3><a href="/explorables/private-and-fair/" target="_blank">Can a Model Be Differentially Private and Fair?</a></h3></div> <div class="pair-card-content pair-third-card-content"> <p>Training models with differential privacy stops models from inadvertently leaking sensitive data, but there’s an unexpected side-effect: reduced accuracy on underrepresented subgroups.</p> </div> <div class="pair-card-cta"> <a href="/explorables/private-and-fair/" target="_blank"><img src="/images/PAIR/arrow-link-out.png" class="arrow-out"/>Start exploring</a> </div> </div> <div class='pair-third-card pair-card mdl-cell mdl-cell--4-col mdl-cell--8-col-tablet mdl-cell--4-col-phone explorable-card'> <div class="pair-card-image pair-third-card-image"> <a href="/explorables/federated-learning/" target="_blank"><img src="https://pair.withgoogle.com/explorables/images/federated-learning-abstract.png"/></a> </div> <div class="pair-card-type"></div> <div class="pair-card-title pair-third-card-title"><h3><a href="/explorables/federated-learning/" target="_blank">Federated Learning</a></h3></div> <div class="pair-card-content pair-third-card-content"> <p>Most machine learning models are trained by collecting vast amounts of data on a central server. Federated learning makes it possible to train models without any user’s raw data leaving their device.</p> </div> <div class="pair-card-cta"> <a href="/explorables/federated-learning/" target="_blank"><img src="/images/PAIR/arrow-link-out.png" class="arrow-out"/>Start exploring</a> </div> </div> <div class='pair-third-card pair-card mdl-cell mdl-cell--4-col mdl-cell--8-col-tablet mdl-cell--4-col-phone explorable-card'> <div class="pair-card-image pair-third-card-image"> <a href="/explorables/uncertainty-calibration/" target="_blank"><img src="https://pair.withgoogle.com/explorables/images/uncertainty-calibration-abstract.png"/></a> </div> <div class="pair-card-type"></div> <div class="pair-card-title pair-third-card-title"><h3><a href="/explorables/uncertainty-calibration/" target="_blank">Are Model Predictions Probabilities?</a></h3></div> <div class="pair-card-content pair-third-card-content"> <p>Machine learning models express their uncertainty as model scores, but through calibration we can transform these scores into probabilities for more effective decision making.</p> </div> <div class="pair-card-cta"> <a href="/explorables/uncertainty-calibration/" target="_blank"><img src="/images/PAIR/arrow-link-out.png" class="arrow-out"/>Start exploring</a> </div> </div> <div class='pair-third-card pair-card mdl-cell mdl-cell--4-col mdl-cell--8-col-tablet mdl-cell--4-col-phone explorable-card'> <div class="pair-card-image pair-third-card-image"> <a href="/explorables/saliency/" target="_blank"><img src="https://pair.withgoogle.com/explorables/images/saliency-abstract.png"/></a> </div> <div class="pair-card-type"></div> <div class="pair-card-title pair-third-card-title"><h3><a href="/explorables/saliency/" target="_blank">Searching for Unintended Biases With Saliency</a></h3></div> <div class="pair-card-content pair-third-card-content"> <p>Machine learning models sometimes learn from spurious correlations in training data. Trying to understand <em>how</em> models make predictions gives us a shot at spotting flawed models.</p> </div> <div class="pair-card-cta"> <a href="/explorables/saliency/" target="_blank"><img src="/images/PAIR/arrow-link-out.png" class="arrow-out"/>Start exploring</a> </div> </div> <div class='pair-third-card pair-card mdl-cell mdl-cell--4-col mdl-cell--8-col-tablet mdl-cell--4-col-phone explorable-card'> <div class="pair-card-image pair-third-card-image"> <a href="/explorables/dataset-worldviews/" target="_blank"><img src="https://pair.withgoogle.com/explorables/images/dataset-worldviews-shareimg.png"/></a> </div> <div class="pair-card-type"></div> <div class="pair-card-title pair-third-card-title"><h3><a href="/explorables/dataset-worldviews/" target="_blank">Datasets Have Worldviews</a></h3></div> <div class="pair-card-content pair-third-card-content"> <p>Every dataset communicates a different perspective. When you shift your perspective, your conclusions can shift, too.</p> </div> <div class="pair-card-cta"> <a href="/explorables/dataset-worldviews/" target="_blank"><img src="/images/PAIR/arrow-link-out.png" class="arrow-out"/>Start exploring</a> </div> </div> <div class='pair-third-card pair-card mdl-cell mdl-cell--4-col mdl-cell--8-col-tablet mdl-cell--4-col-phone explorable-card'> <div class="pair-card-image pair-third-card-image"> <a href="/explorables/measuring-diversity/" target="_blank"><img src="https://pair.withgoogle.com/explorables/images/measuring-diversity.png"/></a> </div> <div class="pair-card-type"></div> <div class="pair-card-title pair-third-card-title"><h3><a href="/explorables/measuring-diversity/" target="_blank">Measuring Diversity</a></h3></div> <div class="pair-card-content pair-third-card-content"> <p>Search results that reflect historic inequities can amplify stereotypes and perpetuate under-representation. Carefully measuring diversity in data sets can help.</p> </div> <div class="pair-card-cta"> <a href="/explorables/measuring-diversity/" target="_blank"><img src="/images/PAIR/arrow-link-out.png" class="arrow-out"/>Start exploring</a> </div> </div> <div class='pair-third-card pair-card mdl-cell mdl-cell--4-col mdl-cell--8-col-tablet mdl-cell--4-col-phone explorable-card'> <div class="pair-card-image pair-third-card-image"> <a href="/explorables/measuring-fairness/" target="_blank"><img src="/images/PAIR/explorable-fairness.png"/></a> </div> <div class="pair-card-type"></div> <div class="pair-card-title pair-third-card-title"><h3><a href="/explorables/measuring-fairness/" target="_blank">Measuring Fairness</a></h3></div> <div class="pair-card-content pair-third-card-content"> <p>There are multiple ways to measure accuracy. No matter how we build our model, accuracy across these measures will vary when applied to different groups of people.</p> </div> <div class="pair-card-cta"> <a href="/explorables/measuring-fairness/" target="_blank"><img src="/images/PAIR/arrow-link-out.png" class="arrow-out"/>Start exploring</a> </div> </div> <div class='pair-third-card pair-card mdl-cell mdl-cell--4-col mdl-cell--8-col-tablet mdl-cell--4-col-phone explorable-card'> <div class="pair-card-image pair-third-card-image"> <a href="/explorables/hidden-bias/" target="_blank"><img src="/images/PAIR/explorable-bias.png"/></a> </div> <div class="pair-card-type"></div> <div class="pair-card-title pair-third-card-title"><h3><a href="/explorables/hidden-bias/" target="_blank">Hidden Bias</a></h3></div> <div class="pair-card-content pair-third-card-content"> <p>Models trained on real-world data can encode real-world bias. Hiding information about protected classes doesn’t always fix things — sometimes it can even hurt.</p> </div> <div class="pair-card-cta"> <a href="/explorables/hidden-bias/" target="_blank"><img src="/images/PAIR/arrow-link-out.png" class="arrow-out"/>Start exploring</a> </div> </div> </div> </div> </div> <div class="sub-bot-image"><img class="sub-flat-border-top" src="/images/PAIR/inside-flat-bot-bg.png"/></div> </div> </div> </section> <style> body { background-color: #f0f6f5; } </style> </section> <section id="footer-content"> <div class="single-page-content"> <div class="mdl-grid pair-footer-container"> <div class="mdl-cell--1-col mdl-cell--4-col-tablet mdl-cell--1-col-phone one-col-spacer"></div> <div class="footer-left mdl-cell mdl-cell--7-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="google-logo hideme840"><img alt="Google Logo" src="/images/google-logo.svg"/></div> <a href="https://research.google/" target="_blank">Google Research</a> <a href="https://design.google/library/ai/" target="_blank">Google Design</a> </div> <div class="footer-right mdl-cell mdl-cell--4-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <a href="mailto:peopleai@google.com" target="_blank"><img src="/images/PAIR/mail.png"/></a> <a href="https://github.com/pair-code" target="_blank"><img src="/images/PAIR/github.png"/></a> <a href="https://medium.com/people-ai-research" target="_blank"><img src="/images/PAIR/medium.png"/></a> <a href="https://www.youtube.com/channel/UCnnns-uu4yy9BXfYSPIX5AA" target="_blank"><img src="/images/PAIR/youtube.png"/></a> </div> <div class="google-hidden-logo showme840"><img alt="Google Logo" src="/images/google-logo.svg"/></div> </div> </section> </main> </div> </div> <script src="/js/material.min.js"></script> <script src="/js/smoothscroll.min.js"></script> <script type="text/javascript" src="https://www.gstatic.com/feedback/js/help/prod/service/lazy.min.js"></script> <script src="/js/pair-page.js"></script> <script data-autoload-cookie-consent-bar="true" src="https://www.gstatic.com/brandstudio/kato/cookie_choice_component/cookie_consent_bar.v2.js"></script> </body> </html>