Margaret Mitchell

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd"> <html> <head> <title>Margaret Mitchell</title> <LINK REL=StyleSheet HREF="css/style.css" TYPE="text/css" MEDIA=screen> <meta name="description" content="AI Researcher. Ethics in Artificial Intelligence, Natural Language Processing, Computer Vision" /> <meta name="keywords" content="AI Researcher, A.I. Researcher, AI, A.I., Natural Language Processing, NLP, Computer Vision, Machine Learning, Artificial Intelligence, Google, Microsoft, Microsoft Research, MSR, Hugging Face" /> <meta name="robots" content="index,follow" /> <link rel="stylesheet" href="css/scroll_style.css">   <link rel="stylesheet" href="js/jquery.mCustomScrollbar.css"> <script src="js/jquery.min.js"></script> <script> $(function(){ $("#sidebar").load("sidebar.html", function() {$('#home').addClass("active");}); }); $(function(){ $("#header").load("header.html"); }); </script> </head> <body> <div id="page"> <div id="header"> </div> <div id="mainarea"> <div id="sidebar"> </div> </div> <div id="contentarea"> <div id="whoami"> <br/><br/> <center> Jump to...<br/> <h1><a href="#Updates">Updates</a> · <a href="#About">About Me</a></h1></center>    <hr style="border: 1px solid #FFFFFF; width:50%;" /> <h3><a name="Publications" href="https://scholar.google.com/citations?hl=en&user=5na92fcAAAAJ&view_op=list_works&sortby=pubdate">Peer-reviewed publications by year</a> or by <a href="https://scholar.google.com/citations?hl=en&user=5na92fcAAAAJ&view_op=list_works">citation count</a></h3> <div>Welcome to my website! I'm Margaret (Meg) Mitchell, an interdisciplinary computer scientist & researcher working in the tech industry. My specializations are in AI ethics and natural language processing (NLP), and my work focuses on machine learning (ML), ML data processing & governance, ML documentation, AI system evaluation, AI development processes, and societal impact. My projects often bring in some of my additional background in computer vision, linguistics, psychology, and augmentative & alternative communication (AAC). I am currently Chief Ethics Scientist and researcher at AI startup Hugging Face. I do not update this website often; it may be somewhat out of date. </div> <BR/> <h3>Some Recent Videos</h3> <div> <a href="https://www.youtube.com/watch?v=idbqipz8oJQ" target="_blank">Bloomberg Technology Interview: Google's Gemini Issues are Fixable, February 2024</a> <br/> <iframe width="560" height="315" src="https://www.youtube.com/embed/idbqipz8oJQ?si=7HI2FQmAj0ddgwBt" title="Bloomberg Technology: Google's Gemini Issues are Fixable, February 2024" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" referrerpolicy="strict-origin-when-cross-origin" allowfullscreen></iframe> </div> <br/> <div> <a href="https://www.youtube.com/embed/xddQq3opSzU?si=0LGDMk4bMy7WpCZt&start=352" target="_blank">Science Gallery London Keynote & Panel: Building Better AI in the Open, Summer 2023</a> <iframe width="560" height="315" src="https://www.youtube.com/embed/xddQq3opSzU?si=0LGDMk4bMy7WpCZt&start=352" title="Openness and Governance" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" referrerpolicy="strict-origin-when-cross-origin" allowfullscreen></iframe> </div> <br/> <div> <a href="https://www.youtube.com/embed/urBrxQ4e8WY?si=sG8FiPbutMa9DRbU" target="_blank">PyCon Keynote: Python, Data, and Bias, Spring 2023</a> <br/> <iframe width="560" height="315" src="https://www.youtube.com/embed/urBrxQ4e8WY?si=sG8FiPbutMa9DRbU" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" referrerpolicy="strict-origin-when-cross-origin" allowfullscreen></iframe> </div> <br/> <div> <a href="https://youtu.be/T2e6Y37EAGo?t=9" target="_blank">Keynote talk at the Stanford Center for Research on Foundation Models Workshop, Summer 2021</a> <br/> <iframe width="560" height="315" src="https://www.youtube.com/embed/T2e6Y37EAGo?start=9" title="Cementing a Foundation of Inequity in AI" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe> </div> <br/>  <div id="fb-root"></div> <script async defer src="https://connect.facebook.net/en_US/sdk.js#xfbml=1&version=v3.2"></script> <a href="https://www.facebook.com/wired/videos/1295687017451479/?t=22", target="_blank">With WIRED and Ash Carter, Former U.S. Secretary of Defense, 2020</a> <br/>  <div class="fb-video" data-href="https://www.facebook.com/wired/videos/1295687017451479/?t=22" data-width="560" data-show-text="false"> <div class="fb-xfbml-parse-ignore"> <blockquote cite="https://www.facebook.com/wired/videos/1295687017451479/?t=22" class="fb-xfbml-parse-ignore"> </blockquote> </div> </div> <br/> <br/> <div> <a href="https://www.youtube.com/embed/5pMQGT3O4CI?si=Cnb7Ff4vi0emcqLd" target="_blank">Google I/O Presentation: Writing the Playbook for Fair & Ethical Artificial Intelligence & Machine Learning, 2019</a> <iframe width="560" height="315" src="https://www.youtube.com/embed/5pMQGT3O4CI?si=Cnb7Ff4vi0emcqLd" title="Google I/O: Writing the Playbook for Fair & Ethical Artificial Intelligence & Machine Learning, 2019" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" referrerpolicy="strict-origin-when-cross-origin" allowfullscreen></iframe> </div> <br/> <h2><a name="Updates">Sporadically Updated Timeline</a></h2> <div id="content-1" class="content"> <table cellpadding="10" border="0"> <tr><td valign="top">2021:</td><td valign="top">I have incorporated into my own Ethical AI LLC. Drop me a line if you want to work with me!</td></tr> <tr><td valign="top">2019:</td><td valign="top">I'm updating a lot less now. =) Working on AI Ethics and not enough "free" time (like everyone in this field...).</td></tr> <tr><td valign="top">Late October 2018:</td><td valign="top"><a href="https://arxiv.org/abs/1810.03993">Model Cards for model reporting</a> accepted to <a href="https://fatconference.org">FAT*</a>. Model cards are short reports for trained machine learning models that provide benchmarked evaluation across conditions such as cultural, demographic, or phenotypic groups (e.g., race, geographic location, sex, Fitzpatrick skin type) and intersectional groups (e.g., age and race, or sex and Fitzpatrick skin type) relevant to the intended application domains. Model cards also disclose the context in which models are intended to be used, details of the performance evaluation procedures, and other relevant information. In two years, it will seem insane that it was not standard to report how well ML models work when they're made available for general public use. </td></tr> <tr><td valign="top">Early October 2018:</td><td valign="top"><a href="https://arxiv.org/pdf/1801.07593.pdf">Adversarial bias mitigation work</a> featured in IBM's <a href="http://aif360.mybluemix.net/">fairness toolkit</a>, available <a href="https://github.com/IBM/AIF360/blob/master/aif360/algorithms/inprocessing/adversarial_debiasing.py">here</a>.</td></tr> <tr><td valign="top">September 2018:</td><td valign="top">Keynote at the <a href="http://montrealaisymposium.com">Montreal AI Symposium</a> on bias in artificial intelligence vision and language. Nice write-up <a href="http://www.montrealintechnology.com/the-best-place-in-north-america-to-eat-and-the-best-place-in-north-america-for-ai/">here</a>.</td></tr> <tr><td valign="top">June 2018:</td><td valign="top"><a href="https://arxiv.org/abs/1712.00193" target="_blank">Paper on improving inclusion in face attribute detection</a> accepted as oral to FAT/ML!</td></tr> <tr><td valign="top">May 2018:</td><td valign="top">Honored to have been selected to represent Google in the <a href="https://www.partnershiponai.org/">Partnership on AI</a>, working on Fair, Transparent, and Accountable AI.</td></tr> <tr><td valign="top">April 2018:</td><td valign="top"><a href="https://developers.googleblog.com/2018/04/text-embedding-models-contain-bias.html">Text Embedding Models Contain Bias. Here's Why That Matters.</a></td></tr> <tr><td valign="top">March 2018:</td><td valign="top">Leading Google's new Ethical AI team.</td></tr> <tr><td valign="top">Mid Feb 2018:</td><td valign="top"><a href="https://www.ted.com/talks/margaret_mitchell_how_we_can_build_ai_to_help_humans_not_hurt_us">TED talk is up!</a></td></tr> <tr><td valign="top">Early Feb 2018:</td><td valign="top">Submit to the workshops I'm co-organizing, <a href="http://www.visionandlanguage.net/workshop2018/">Storytelling</a> and <a href="http://ethicsinnlp.org/">Ethics in NLP</a>! Also check out the continuation of WiNLP, <a href="http://www.winlp.org">Widening NLP</a>!</td></tr> <tr><td valign="top">Mid Jan 2018:</td><td valign="top">Our response to Wang and Kosinski's "gaydar" research, which is both ethically and scientifically problematic: <a href="https://medium.com/@blaisea/do-algorithms-reveal-sexual-orientation-or-just-expose-our-stereotypes-d998fafdf477"><i>Do algorithms reveal sexual orientation or just expose our stereotypes?</i></a> Trended on Medium.</td></tr> <tr><td valign="top">Early Jan 2018:</td><td valign="top"><i>Mitigating Unwanted Biases with Adversarial Learning</i> paper accepted to <a href="http://www.aies-conference.com/">AAAI/ACM conference on Artificial Intelligence, Ethics, and Society</a>.</td></tr> <tr><td valign="top">Nov 2017:</td><td valign="top"><a href="https://arxiv.org/abs/1712.00193">Arxiv paper on smiling detection with intersectional race/gender categories</a> is up! We demonstrate how transfer learning with demographic categories in a held-out set of faces can improve face attribute tasks. Included are <i>intersectional analyses</i> (last page): Performance across different race x gender subgroups.</td></tr> <tr><td valign="top">1.Sep.2017:</td><td valign="top"><a href="http://www.winlp.org">The first workshop on Women and Underrepresented Minorities in NLP</a> had over 90 submission and over 150 participants! This has given rise to a new initiative, a Broad Interest Group on diversity and inclusion in <a href="http://www.aclweb.org">the ACL</a>, which I'm privileged to be helping with.</td></tr> <tr><td valign="top">12.July.2017:</td><td valign="top"> <a href="https://www.microsoft.com/en-us/seeing-ai/">Seeing AI Released!</a> From our CVPR 2015 first place image captioning system on one slow, disorganized computer, streamlined and brought through <a href="http://www.microsoft.com/cognitive">Microsoft Cognitive Services</a> so that the work can leverage others' work and others could leverage it, and foward to help people who are blind. Now in the hands of the users!!</td></tr> <tr><td valign="top">6.May.2017:</td><td valign="top"><a href="https://medium.com/@blaisea/physiognomys-new-clothes-f2d4b59fdd6a">Pushed out an opinion piece for the first time -- on automating discrimination with machine learning. We focused on the task of 'Physiognomy', interpreting people's character from their faces</a>. Relevant to emerging technology in computer vision. <br/>Response has been great. Endorsed by the ACLU, already translated into other languages. <a href="https://medium.com/@blaisea/physiognomys-new-clothes-f2d4b59fdd6a">Piece: Physiognomy's New Clothes</a>, with with my manager Blaise Agüera y Arcas and Princeton professor Alexander Todorov.</td></tr> <tr><td valign="top">21.April.2017:</td><td valign="top">The Bill Nye Saves the World show is out!! I'm on <a href="https://www.netflix.com/watch/80117907">Episode 3, "Machines Take Over The World"</a>, talking about my research in vision and language towards positive goals.</td></tr>    <tr><td valign="top">25.March.2017:</td><td valign="top"><a href="http://www.sueddeutsche.de/wissen/kuenstliche-intelligenz-der-computer-als-rassist-1.3432883">Ethics in NLP has hit the big time with coverage in Süddeutsche Zeitung</a>!</td></tr> <tr><td valign="top">ACL 2017:</td><td valign="top">Come check out the workshop I'm co-organizing, <a href="http://winlp.org">Women and Underrepresented Minorities in NLP</a>!</td></tr> <tr><td valign="top">EACL 2017:</td><td valign="top"><a href="http://m-mitchell.com/publications/multitask-blurb.html">Paper accepted on predicting suicide risk using multitask deep learning!</a> Also, come check out the workshop I'm co-organizing, <a href="http://ethicsinnlp.org">Ethics in NLP</a>!</td></tr> <tr><td valign="top">6.Feb.2017:</td><td valign="top"><a href="https://www.fastcompany.com/3067836/why-google-ideo-and-ibm-are-betting-on-ai-to-make-us-better-storytellers">New article on using Storytelling to advance Artificial Intelligence.</a></td></tr> <tr><td valign="top">NIPS 2016:</td><td valign="top">My former intern <a href="http://spandanagella.com/" target="_blank">Spandana Gella</a> is introducing our work from the summer: The first computational model for vision-to-language <i>specifically for people who are blind</i>. This is one of the most important perspectives to consider if you're interested in vision-language research. At <a href="http://wimlworkshop.org/" target="_blank">WiML</a>.</td></tr> <tr><td valign="top">Early November 2016:</td><td valign="top">Honored to begin new position as a Senior Research Scientist at <a href="http://research.google.com/pubs/MachineIntelligence.html" target="_blank">Google Research and Machine Intelligence</a>.</td></tr> <tr><td valign="top">Late October 2016:</td><td valign="top">I have left Microsoft Research. Please invite me out for cookies and beers, I could use it.</td></tr> <tr><td valign="top">October 2016:</td><td valign="top">I had the great privilege of being an AI expert on the <a href="https://www.netflix.com/title/80117748" target="_blank">new Bill Nye show!</a></td></tr> <tr><td valign="top">September 2016:</td><td valign="top"><a href="https://scholar.google.com/citations?view_op=view_citation&hl=en&user=5na92fcAAAAJ&citation_for_view=5na92fcAAAAJ:4DMP91E08xMC">The Visual Question Answering work</a> is taking the world by storm. </td></tr> <tr><td valign="top">Early September 2016:</td><td valign="top">Was offered the amazing opportunity of being a Senior Research Scientist in Google's Machine Intelligence group.</td></tr> <tr><td valign="top">Late August 2016:</td><td valign="top">Our Seeing AI project has <a href="https://www.facebook.com/BuzzFeedTech/videos/943223692465274/">over 1.7 million views on Buzzfeed</a>. =O<br/> Microsoft has let me know that I am not making enough impact, and promoted the men around me who helped with the project.</td></tr> <tr><td valign="top">Early August 2016:</td><td valign="top">Our former intern <a href="http://www.cs.cmu.edu/~imisra/", target="_blank">Ishan Misra</a> did a fantastic job <a href="http://www.cs.cmu.edu/~imisra/" target="_blank">explaining the reporting bias model on the Data Skeptic podcast</a>!</td></tr> <tr><td valign="top">Late July 2016:</td><td valign="top"> Helped advance clinical state-of-the-art in a <a href="http://www.clsp.jhu.edu/workshops/16-workshop/" target="_blank">one-of-a-kind workshop</a> on Detecting Risk and Protective Factors of Mental Health using Social Media Linked with Electronic Health Records, part of the <a href="http://www.clsp.jhu.edu/workshops/" target="_blank">JSALT Workshop series</a>. Definitely worth plugging in for a couple weeks. </td></tr> <tr><td valign="top">CVPR 2016: </td><td valign="top">Presented <a href="http://arxiv.org/abs/1512.06974">the first computational model for reporting bias</a>! Use variations on this <i>one handy trick</i> to significantly improve captioning, detection, etc. etc.!<Br/><BR/>Gave invited talk on vision-to-language generation at the <a href="http://visualqa.org/workshop.html" target="_blank">VQA Challenge</a>! <a href="images/invited_talk.jpg">[photo]</a> </td></tr> <tr><td valign="top">NAACL 2016:</td><td valign="top">Presented the <a href="https://arxiv.org/abs/1604.03968">storytelling work</a> to the masses!</td></tr> <tr><td valign="top">June 2016:</td><td valign="top">My kinda silly <a href="http://www.bloomberg.com/news/articles/2016-06-23/artificial-intelligence-has-a-sea-of-dudes-problem" target="_blank">"sea of dudes"</a> phrase, which I use to feel more relaxed as I go up and present, is taking the world by storm! Thanks, <a href="http://www.bloomberg.com/news/articles/2016-06-23/artificial-intelligence-has-a-sea-of-dudes-problem" target="_blank">Bloomberg</a>! </td></tr> <tr> <td valign="top">April 2016: </td><td valign="top">People are interested in both the Seeing AI work and the <a href="https://arxiv.org/abs/1604.03968">Visual Storytelling</a> work! <BR/>Here are some articles on <b><u>Seeing AI</u></b> from <a href="http://www.fastcodesign.com/3058905/the-real-reason-microsoft-is-building-so-many-computer-vision-apps" target="_blank">Fast Company</a>, <a href="https://www.technologyreview.com/s/601339/will-artificial-intelligence-win-the-caption-contest" target="_blank">MIT Technology Review</a>, and <a href="blogs.microsoft.com/next/2016/03/30/decades-of-computer-vision-research-one-swiss-army-knife" target="_blank">Microsoft</a>. <br/>Here are some articles on the <b><u>storytelling</u></b> work from <a href="http://www.livescience.com/54961-artificial-intelligence-tells-stories-from-photos.html" target="_blank">Live Science</a>, <a href="http://venturebeat.com/2016/04/14/microsoft-ai-visual-storytelling" target="_blank">Venture Beat</a>, and <a href="http://blogs.microsoft.com/next/2016/04/14/teaching-computers-to-describe-images-as-people-would">Microsoft</a>. The <a href="http://www.livescience.com/54961-artificial-intelligence-tells-stories-from-photos.html" target="_blank">Live Science</a> journalist was great, he was deeply concerned about how these things can be evaluated, and how you know when you're doing well. </td></tr> <tr> <td valign="top">30.March.2016: </td><td valign="top">Satya unveiled our project for assisting the visually impaired, <b>Seeing AI.</b> <a href="https://www.youtube.com/watch?v=rVF2duPVUTY" target="_blank">Check out the video here</a>!</td></tr> <tr><td valign="top">CVPR 2015: </td><td valign="top">The image captioning system I jockeyed won <a href="http://mscoco.org/dataset/#captions-challenge2015" target="_blank">first place in the COCO captioning challenge</a>, voted to be the most human-like! Let's hear it for thinking deeply about what kind of evaluation to focus on, and the power of logistic regression used well! Onwards with this technology towards its most real use case: Assisting people who are blind.</td></tr> </table> </div> <br/> <hr style="border: 1px solid #FFFFFF; width:50%;" /> <div id="whoami"> <h2><a name="About">About Me</a></h2> <br/>My research generally involves vision-language and grounded language generation, focusing on how to <span style="color:#42f44e;">evolve artificial intelligence towards positive goals</span>. This includes research on helping computers to communicate based on what they can process, as well as projects to create assistive and clinical technology from the state of the art in AI. <br/> <br/> My work combines computer vision, natural language processing, social media, many statistical methods, and insights from cognitive science. <br/> <br/> In a nutshell, I've worked on: <ul> <li>deep learning, structured learning, shallow learning, and probabilistic systems (<b>Math</b>)</li> <li>natural language generation, referring expression generation, reference to visible objects, conversation, image captioning, visual question answering, and storytelling (<b>Grounded Language</b>)</li> <li>dialogue assistance for people who are non-verbal (Cerebral Palsy and Autism), visual descriptions for people who are blind, automatic diagnosis/monitoring of Mild Cognitive Impairment (a precursor to Alzheimer's), Parkinson's, Apraxia, Autism, Depression, Post-Traumatic Stress Disorder, Suicide Risk, and Schizophrenia (<b>Assistive and Clinical Technology</b>)</li> </ul> </div> <br/> <hr style="border: 1px solid #FFFFFF; width:50%;" /> <br /> <div id="history"> Before founding Ethical AI and co-founding ML Fairness at Google Research, I was a founding member of Microsoft Research's "Cognition" group, focused on advancing artificial intelligence, and a researcher in <a href="https://www.microsoft.com/en-us/research/group/natural-language-processing/">Microsoft Research's Natural Language Processing group</a>. <br/><br/> Before MSR, I was a postdoctoral researcher at <a href="http://hltcoe.jhu.edu/" target="_blank">The Johns Hopkins University Center of Excellence</a>, where I focused on structured prediction, semantic role labeling, and sentiment analysis, working under <a href="http://www.cs.jhu.edu/~vandurme/">Benjamin Van Durme</a>. <br /> <br /> Before that, I was a postgraduate (PhD) student in the <a href="http://www.abdn.ac.uk/nlg/" target="_blank">natural language generation (NLG) group</a> at the University of Aberdeen, where I focused on how to naturally refer to visible, everyday objects. I primarily worked with <a href="https://www.abdn.ac.uk/ncs/profiles/k.vdeemter">Kees van Deemter</a> and <a href="http://www.abdn.ac.uk/ncs/profiles/e.reiter/">Ehud Reiter</a>. <br /> <br /> I spent a good chunk of 2008 getting a Master's in Computational Linguistics at the University of Washington, studying under <a href="http://faculty.washington.edu/ebender/">Emily Bender</a> and <a href="https://faculty.washington.edu/fxia/">Fei Xia</a>. <br /> <br /> Simultaneously (2005 - 2012), I worked on and off at the <a href="http://www.ogi.edu/bme/cslu" target="_blank">Center for Spoken Language Understanding</a>, part of OHSU, in Portland, Oregon. My title changed with time (research assistant/associate/visiting scholar), but throughout, I worked on technology that leverages syntactic and phonetic characteristics to aid those with neurological disorders. <br/><a href="http://www.lanzaroark.org/brian-roark">Brian Roark</a> was my boss/mentor/supervisor. <br/>I would not be in Computer Science if it were not for him encouraging me as an intelligent individual with ideas worth pursuing. <br/> </div> <br/> <hr style="border: 1px solid #FFFFFF; width:50%;" /> <br/> <div id="whoami"> I continue to balance my time between language generation, applications for clinical domains, and core AI research. </div>  </div> </div> <div id="footer"> </div> </div>   <script src="http://ajax.googleapis.com/ajax/libs/jquery/1.11.0/jquery.min.js"></script> <script>window.jQuery || document.write('<script src="js/minified/jquery-1.11.0.min.js"><\/script>')</script>  <script src="js/jquery.mCustomScrollbar.concat.min.js"></script> <script> (function($){ $(window).on("load",function(){ $("#content-1").mCustomScrollbar({ theme:"light-thick", scrollButtons:{enable:true}, scrollbarPosition: "outside" }); }); })(jQuery); </script> </body> </html>

CINXE.COM

Margaret Mitchell