CINXE.COM

<!DOCTYPE html> <html> <head> <meta name="viewport" content="width=device-width, initial-scale=1.0">   <script async src="https://www.googletagmanager.com/gtag/js?id=UA-46457317-11"></script> <script> window.dataLayer = window.dataLayer || []; function gtag(){dataLayer.push(arguments);} gtag('js', new Date()); gtag('config', 'UA-46457317-11'); </script> <title>WIT - Getting Started</title> <link rel="stylesheet" href="https://fonts.googleapis.com/icon?family=Material+Icons"> <link rel="stylesheet" href="/what-if-tool/assets/css/material.min.css"> <link rel="stylesheet" href="/what-if-tool/assets/css/material-components-web.min.css"> <link href="https://fonts.googleapis.com/css?family=Open+Sans:400,400italic,500,500italic,700,700italic" rel="stylesheet" type="text/css"> <link href="https://fonts.googleapis.com/css?family=Manrope:400,400italic,500,500italic,700,700italic" rel="stylesheet" type="text/css">  <link href='/what-if-tool/assets/css/new.css' rel='stylesheet' type='text/css'> <link rel="icon" href="/what-if-tool/assets/images/favicon.png" type="image/png"/> <style> .sub-title, .sub-copy { color: #FFFFFF; } </style> </head> <body> <div class="mdl-layout mdl-layout--no-desktop-drawer-button mdl-js-layout mdl-layout--fixed-header"> <header class="mdl-layout__header"> <div class="mdl-layout__header-row">  <span class="mdl-layout__title"><a href="/what-if-tool/">What-If Tool</a></span>  <div class="mdl-layout-spacer"></div>  <nav class="mdl-navigation mdl-layout--large-screen-only"> <a class="mdl-navigation__link" href="/what-if-tool/get-started/">GET STARTED</a> <a class="mdl-navigation__link" href="/what-if-tool/learn/">TUTORIALS</a> <a class="mdl-navigation__link" href="/what-if-tool/explore/">DEMOS</a> <a class="mdl-navigation__link" href="/what-if-tool/faqs/">FAQs</a> <a class="mdl-navigation__link" href="https://groups.google.com/forum/#!forum/what-if-tool" target="-_blank">GET INVOLVED<img class="header-arrow" src="/what-if-tool/assets/images/arrow-link-out.png"/></a> <a class="mdl-navigation__link" href="https://github.com/pair-code/what-if-tool" target="-_blank">GITHUB<img class="header-arrow" src="/what-if-tool/assets/images/arrow-link-out.png"/></a> </nav> </div> </header> <div class="mdl-layout__drawer"> <span class="mdl-layout__title"><a href="/what-if-tool/">What-If Tool</a></span> <nav class="mdl-navigation"> <a class="mdl-navigation__link" href="/what-if-tool/get-started/">GET STARTED</a> <a class="mdl-navigation__link" href="/what-if-tool/learn/">TUTORIALS</a> <a class="mdl-navigation__link" href="/what-if-tool/explore/">DEMOS</a> <a class="mdl-navigation__link" href="/what-if-tool/faqs/">FAQs</a> <a class="mdl-navigation__link" href="https://groups.google.com/forum/#!forum/what-if-tool" target="_blank">GET INVOLVED<img class="header-arrow" src="/what-if-tool/assets/images/arrow-link-out.png"/></a> <a class="mdl-navigation__link" href="https://github.com/pair-code/what-if-tool" target="-_blank">GITHUB<img class="header-arrow" src="/what-if-tool/assets/images/arrow-link-out.png"/></a>  </nav> </div> <main class="mdl-layout__content hero-banner"> <div class="sub-hero-container" style="background-image:url( /what-if-tool/assets/images/banner-getting-started.png );"> <div class="hero-container mdl-grid"> <div class="mdl-cell--8-col mdl-cell--6-col-tablet mdl-cell--4-col-phone"> <div class="sub-title">pip install witwidget</div> <div class="sub-copy">Use the What If Tool directly in your Notebooks, in TensorBoard, and with Cloud AI models.</div> </div> </div> </div> <div class="sub-page-container mdl-grid"> <div class="mdl-cell--12-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <div class="mdl-cell--8-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <p><a name="notebooks"></a></p> <h2>Model understanding in Notebooks</h2> <p>The What-If Tool is available as an extension in Jupyter, Colaboratory, and Cloud AI Platform notebooks. Use the What-If Tool to analyze classification or regression models on datapoints as inputs directly from within the notebook.</p> <p>A custom prediction function can be used to load any model, and provide additional customizations to the What-If Tool, including feature attribution methods like SHAP, Integrated Gradients, or SmoothGrad.</p> <div class="link-out"><a href="/what-if-tool/learn/tutorials/notebooks/">See setup guide</a><img class="external-arrow" src="/what-if-tool/assets/images/arrow-link-out.png"/></div> <div class="spacer" style="height:30px;"></div> <div class="section-action">Explore</div> <h3>Notebooks</h3> <div class="mdl-grid no-padding"> <div class="demo-card mdl-cell mdl-cell--6-col mdl-cell--4-col-tablet mdl-cell--4-col-phone"> <div class="demo-card-title"><a href="https://colab.research.google.com/github/pair-code/what-if-tool/blob/master/WIT_Model_Comparison.ipynb" target="_blank">Compare income classification on UCI census data</a></div> <div class="demo-card-tags"> <span class="demo-tag"> binary classification </span> <span class="demo-tag"> model comparison </span> </div> <div class="demo-card-data-source-title">DATA SOURCE</div> <div class="demo-card-data-source"> <a href="http://archive.ics.uci.edu/ml/datasets/Census+Income" target="_blank">UCI Census Income Dataset</a> </div> <div class="demo-card-copy">Compare two binary classification models that predict whether a person earns more than $50k a year, based on their census information. Examine how different features affect each models' prediction, in relation to each other.</div> <div class="demo-card-cta-button"><a href="https://colab.research.google.com/github/pair-code/what-if-tool/blob/master/WIT_Model_Comparison.ipynb"></a></div> </div> <div class="demo-card mdl-cell mdl-cell--6-col mdl-cell--4-col-tablet mdl-cell--4-col-phone"> <div class="demo-card-title"><a href="https://colab.research.google.com/github/pair-code/what-if-tool/blob/master/WIT_Toxicity_Text_Model_Comparison.ipynb" target="_blank">Text toxicity classifiers</a></div> <div class="demo-card-tags"> <span class="demo-tag"> binary classification </span> <span class="demo-tag"> model comparison </span> <span class="demo-tag"> keras model </span> <span class="demo-tag"> custom distance </span> </div> <div class="demo-card-data-source-title">DATA SOURCE</div> <div class="demo-card-data-source"> <a href="https://figshare.com/articles/Wikipedia_Talk_Labels_Toxicity/4563973" target="_blank">Wikpedia Comments Dataset</a> </div> <div class="demo-card-copy">Use the What-If Tool to compare two pre-trained models from ConversationAI that determine sentence toxicity, one of which was trained on a more balanced dataset. Examine their performance side-by-side on the Wikipedia Comments dataset. These are keras models which do not use TensorFlow examples as an input format.</div> <div class="demo-card-cta-button"><a href="https://colab.research.google.com/github/pair-code/what-if-tool/blob/master/WIT_Toxicity_Text_Model_Comparison.ipynb"></a></div> </div> </div> <div class="spacer" style="height:50px;"></div> <p><a name="cloud-ai"></a></p> <h2>Easily explore Cloud AI model results</h2> <p>The What-If Tool can be easily configured to analyze AI Platform Prediction-hosted classification or regression models.</p> <p>Use the What-If Tool to display and investigate attribution values for individual input features in relation to model predictions.</p> <div class="link-out"><a href="https://cloud.google.com/ai-platform/prediction/docs/using-what-if-tool" target="_blank">Visit Cloud AI</a><img class="external-arrow" src="/what-if-tool/assets/images/arrow-link-out.png"/></div> <div class="spacer" style="height:30px;"></div> <div class="section-action">Explore</div> <h3>Notebooks</h3> <div class="mdl-grid no-padding"> <div class="demo-card mdl-cell mdl-cell--6-col mdl-cell--4-col-tablet mdl-cell--4-col-phone"> <div class="demo-card-title"><a href="https://colab.research.google.com/github/pair-code/what-if-tool/blob/master/xgboost_caip.ipynb" target="_blank">Mortgage classification with AI Platform</a></div> <div class="demo-card-tags"> <span class="demo-tag"> binary classification </span> <span class="demo-tag"> cloud ai platform </span> </div> <div class="demo-card-data-source-title">DATA SOURCE</div> <div class="demo-card-data-source"> <a href="https://www.ffiec.gov/hmda/hmdaflat.htm" target="_blank">Home Mortgage Disclosure Act Dataset</a> </div> <div class="demo-card-copy">Explore a mortgage classification model that has been deployed on Cloud AI Platform. This model was created with the XGBoost platform and not TensorFlow.</div> <div class="demo-card-cta-button"><a href="https://colab.research.google.com/github/pair-code/what-if-tool/blob/master/xgboost_caip.ipynb"></a></div> </div> <div class="demo-card mdl-cell mdl-cell--6-col mdl-cell--4-col-tablet mdl-cell--4-col-phone"> <div class="demo-card-title"><a href="https://colab.sandbox.google.com/github/pair-code/what-if-tool/blob/master/keras_sklearn_compare_caip_e2e.ipynb" target="_blank">Training and comparing wine quality models with AI Platform</a></div> <div class="demo-card-tags"> <span class="demo-tag"> regression </span> <span class="demo-tag"> model comparison </span> <span class="demo-tag"> cloud ai platform </span> <span class="demo-tag"> keras model </span> <span class="demo-tag"> scikit-learn model </span> </div> <div class="demo-card-data-source-title">DATA SOURCE</div> <div class="demo-card-data-source"> <a href="https://archive.ics.uci.edu/ml/datasets/wine+quality" target="_blank">UCI Wine Quality Dataset</a> </div> <div class="demo-card-copy">Train both a scikit-learn and keras model to predict wine quality and deploy them to Cloud AI Platform. Then use the What-If Tool to compare the two models. This demo requires a Google Cloud Platform account.</div> <div class="demo-card-cta-button"><a href="https://colab.sandbox.google.com/github/pair-code/what-if-tool/blob/master/keras_sklearn_compare_caip_e2e.ipynb"></a></div> </div> </div> <div class="spacer" style="height:50px;"></div> <p><a name="tensorboard"></a></p> <h2>Use the What-If Tool in TensorBoard</h2> <p>Quickly explore your models in the What-If Tool from directly within TensorBoard, by providing the What-If Tool with a model server host and port, and a dataset for the model to perform predictions on.</p> <p>The What-If Tool accepts a variety of data types. Upload data as tf.Examples, tf.SequenceExamples, or even a CSV file.</p> <div class="link-out"><a href="/what-if-tool/learn/tutorials/tensorboard/">See setup guide</a><img class="external-arrow" src="/what-if-tool/assets/images/arrow-link-out.png"/></div> <div class="spacer" style="height:30px;"></div> <div class="section-action">Explore</div> <h3>Web demos</h3> <div class="mdl-grid no-padding"> <div class="demo-card mdl-cell mdl-cell--6-col mdl-cell--4-col-tablet mdl-cell--4-col-phone"> <div class="demo-card-title"><a href="/what-if-tool/demos/uci.html" target="_blank">Compare income classification on UCI census data</a></div> <div class="demo-card-tags"> <span class="demo-tag"> binary classification </span> <span class="demo-tag"> model comparison </span> </div> <div class="demo-card-data-source-title">DATA SOURCE</div> <div class="demo-card-data-source"> <a href="http://archive.ics.uci.edu/ml/datasets/Census+Income" target="_blank">UCI Census Income Dataset</a> </div> <div class="demo-card-copy">Compare two binary classification models that predict whether a person earns more than $50k a year, based on their census information. Examine how different features affect each models' prediction, in relation to each other.</div> <div class="demo-card-cta-button"><a href="/what-if-tool/demos/uci.html"></a></div> </div> <div class="demo-card mdl-cell mdl-cell--6-col mdl-cell--4-col-tablet mdl-cell--4-col-phone"> <div class="demo-card-title"><a href="/what-if-tool/demos/image.html" target="_blank">Explore celebrity face image smile classification</a></div> <div class="demo-card-tags"> <span class="demo-tag"> binary classification </span> <span class="demo-tag"> image recognition </span> </div> <div class="demo-card-data-source-title">DATA SOURCE</div> <div class="demo-card-data-source"> <a href="http://mmlab.ie.cuhk.edu.hk/projects/CelebA.html" target="_blank">CelebA Dataset</a> </div> <div class="demo-card-copy">Predict whether an image contains a smiling face using this binary classification model on the CelebA dataset. Can you identify which group was missing from the training data, resulting in a biased model?</div> <div class="demo-card-cta-button"><a href="/what-if-tool/demos/image.html"></a></div> </div> </div> </div> </div> <div class="fixed-sub-navigation hide-me"> <a href="#notebooks">With notebooks</a><a href="#cloud-ai">With Google Cloud</a><a href="#tensorboard">On Tensorboard</a> </div> </div> <div class="footer-container mdl-grid"> <div class="mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--4-col-phone"><a href="https://pair.withgoogle.com/" target="_blank"><img src="/what-if-tool/assets/images/pair-logo.svg"/></a></div> <div class="mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--4-col-phone"><a href="https://research.google/teams/brain/pair/" target="_blank">Google Research</a></div> <div class="mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--4-col-phone"><a href="https://groups.google.com/forum/#!forum/what-if-tool" target="_blank">Get Involved</a></div> <div class="mdl-cell mdl-cell--2-col mdl-cell--2-col-tablet mdl-cell--4-col-phone"><a href="https://github.com/pair-code" target="_blank">Github</a></div> <div class="footer-icons mdl-cell mdl-cell--4-col mdl-cell--8-col-tablet mdl-cell--4-col-phone"> <a href="mailto:peopleai@google.com" target="_blank"><img src="/what-if-tool/assets/images/mail.png"/></a> <a href="https://medium.com/people-ai-research" target="_blank"><img src="/what-if-tool/assets/images/medium.png"/></a> <a href="https://www.youtube.com/channel/UCnnns-uu4yy9BXfYSPIX5AA" target="_blank"><img src="/what-if-tool/assets/images/youtube.png"/></a> </div> </div> </main> </div> </body> <script defer src="/what-if-tool/assets/js/material.min.js"></script> <script defer src="/what-if-tool/assets/js/material-components-web.min.js"></script> </html>