Robotics

<!DOCTYPE html> <html lang="en"> <head> <meta charset="utf-8" /> <link rel="canonical" href="http://research.google/research-areas/robotics/" /><meta property="og:title" content="Robotics"><meta property="og:url" content="http://research.google/research-areas/robotics/"><meta property="og:image" content="https://storage.googleapis.com/gweb-research2023-media/images/Open_Graph.width-800.format-jpeg.jpg"><meta property="og:image:secure_url" content="https://storage.googleapis.com/gweb-research2023-media/images/Open_Graph.width-800.format-jpeg.jpg"><meta property="og:type" content="Website"> <title>Robotics</title> <meta name="viewport" content="width=device-width, initial-scale=1 viewport-fit=cover"/> <link rel="icon" type="image/png" href="/gr/static/assets/favicon.ico"> <link rel="preconnect" href="https://fonts.googleapis.com"> <link rel="preconnect" href="https://fonts.gstatic.com" crossorigin> <link rel="preload" href="https://fonts.googleapis.com/css2?family=Product+Sans&family=Google+Sans+Display:ital@0;1&family=Google+Sans:ital,wght@0,400;0,500;0,700;1,400;1,500;1,700&family=Google+Sans+Text:ital,wght@0,400;0,500;0,700;1,400;1,500;1,700&display=swap" as="style"> <link rel="stylesheet" href="https://fonts.googleapis.com/css2?family=Product+Sans&family=Google+Sans+Display:ital@0;1&family=Google+Sans:ital,wght@0,400;0,500;0,700;1,400;1,500;1,700&family=Google+Sans+Text:ital,wght@0,400;0,500;0,700;1,400;1,500;1,700&display=swap"> <link href="https://fonts.googleapis.com/css2?family=Roboto+Mono:wght@400;700&display=swap" rel="stylesheet"> <link href="https://www.gstatic.com/glue/cookienotificationbar/cookienotificationbar.min.css" rel="stylesheet" /> <link href="https://www.gstatic.com/glue/v27_1/glue-material.min.css" rel="stylesheet"> <link rel="stylesheet" type="text/css" href="/gr/static/css/googleresearch.css?id=0c26ea1fed8bdd0324f9f4fad1f6a470">  <script>(function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start': new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0], j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src= 'https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f); })(window,document,'script','dataLayer','GTM-K8QBZ7Q'); </script>  </head> <body class=" js-google-tag-wrapper" data-gt-page-path="http://research.google/research-areas/robotics/" data-env="production">  <noscript><iframe src="https://www.googletagmanager.com/ns.html?id=GTM-K8QBZ7Q" height="0" width="0" style="display:none;visibility:hidden"></iframe></noscript>  <header class="global-header glue-header glue-header--single not-glue"> <a href="#page-content" class="glue-header__skip-content">Jump to Content</a> <div class="glue-header__bar glue-header__bar--mobile not-glue"> <div class="glue-header__tier not-glue">  <div class="glue-header__container"> <div class="glue-header__lock-up">  <div class="glue-header__hamburger"> <button class="glue-header__drawer-toggle-btn" aria-label="Open the navigation drawer"> <svg class="glue-icon glue-icon--24px" role="presentation" aria-hidden="true"> <use href="/gr/static/assets/icons/glue-icons.svg#menu"></use> </svg> </button> </div> <div class="glue-header__logo"> <a class="glue-header__logo-link" href="/" title="Google Research">  <div class="glue-header__logo-container"> <svg role="presentation" aria-hidden="true" alt='Google' class="glue-icon glue-icon glue-header__logo-svg"> <use href="/gr/static/assets/icons/glue-icons.svg#google-color-logo"></use> </svg> </div> <span class="glue-header__logo--product">Research</span> </a> </div> </div> </div> </div> </div> <div class="glue-header__bar glue-header__bar--desktop glue-header__drawer"> <div class="glue-header__tier">  <div class="glue-header__container"> <div class="glue-header__lock-up"> <div class="glue-header__logo"> <a class="glue-header__logo-link" href="/" title="Google Research">  <div class="glue-header__logo-container"> <svg role="presentation" aria-hidden="true" alt='Google' class="glue-icon glue-icon glue-header__logo-svg not-glue --dark-logo"> <use href="/gr/static/assets/icons/glue-icons.svg#google-solid-logo"></use> </svg> <svg role="presentation" aria-hidden="true" alt='Google' class="glue-icon glue-icon glue-header__logo-svg --light-logo"> <use href="/gr/static/assets/icons/glue-icons.svg#google-color-logo"></use> </svg> </div> <span class="glue-header__logo--product">Research</span> </a> </div> </div> </div>  <div class="glue-header__container glue-header__container--linkbar"> <nav class="glue-header__link-bar navigation js-gt-global-nav-wrapper"> <ul class="glue-header__list"> <li class="glue-header__item js-sub-nav-parent --parent" data-gt-primary="Who we are" > <button class="glue-header__link js-sub-nav-target" aria-haspopup="true" aria-expanded="false" > <span class=""> Who we are <span class="icon icon--caret"></span> </span> </button> <div class="navigation__sub js-sub-nav" role="menu"> <div class="navigation__sub__container"> <div class="navigation__sub__mobile-heading"> <button class="glue-header__link js-sub-nav-close-mobile"> <span class="sr-text">Back to</span> <span class="icon icon--caret"></span> Who we are <span class="sr-text">menu</span> </button> <hr/> </div> <div class="block-nav_drawer_columns_content"> <div class="navigation__sub--content" data-gt-secondary="Defining the technology of today and tomorrow."> <div class="navigation__sub__wrapper"> <div class="navigation__sub__heading"> <h2 class="headline-3">Defining the technology of today and tomorrow.</h2> </div> <ul class="navigation__sub__columns"> <li data-gt-secondary="Philosophy"> <div class="navigation__sub__columns__desktop"> <h2 class="headline-6 navigation__sub__columns__heading"> Philosophy </h2> <p class="navigation__sub__columns__description caption">We strive to create an environment conducive to many different types of research across many different time scales and levels of risk.</p> <a href="http://research.google/philosophy/" class="glue-inline-link js-drawer-link" > <span class="sr-text">Learn more about our Philosophy</span> <span aria-hidden="true">Learn more</span> </a> </div> <div class="navigation__sub__columns__mobile"> <a class="glue-header__link" href="http://research.google/philosophy/" > Philosophy </a> </div> </li> <li data-gt-secondary="People"> <div class="navigation__sub__columns__desktop"> <h2 class="headline-6 navigation__sub__columns__heading"> People </h2> <p class="navigation__sub__columns__description caption">Our researchers drive advancements in computer science through both fundamental and applied research.</p> <a href="http://research.google/people/" class="glue-inline-link js-drawer-link" > <span class="sr-text">Learn more about our People</span> <span aria-hidden="true">Learn more</span> </a> </div> <div class="navigation__sub__columns__mobile"> <a class="glue-header__link" href="http://research.google/people/" > People </a> </div> </li> </ul> </div> </div> </div> </div> </div> </li> <li class="glue-header__item js-sub-nav-parent --parent" data-gt-primary="Research areas" > <button class="glue-header__link js-sub-nav-target" aria-haspopup="true" aria-expanded="false" > <span class=""> Research areas <span class="icon icon--caret"></span> </span> </button> <div class="navigation__sub js-sub-nav" role="menu"> <div class="navigation__sub__container"> <div class="navigation__sub__mobile-heading"> <button class="glue-header__link js-sub-nav-close-mobile"> <span class="sr-text">Back to</span> <span class="icon icon--caret"></span> Research areas <span class="sr-text">menu</span> </button> <hr/> </div> <div class="block-nav_drawer_columns_link_list"> <div class="navigation__sub--list"> <div class="navigation__sub__wrapper"> <ul class="navigation__sub__columns"> <li data-gt-secondary="Research areas"> <div class="navigation__sub__columns__desktop"> <h2 class="headline-6 navigation__sub__columns__heading">Research areas</h2> <ul> <li> <a class="navigation__sub__columns__list-link caption js-drawer-link" href="http://research.google/research-areas/" > Explore all research areas </a> </li> </ul> </div> <div class="navigation__sub__columns__mobile"> <button class="glue-header__link js-sub-nav-target" data-panel="nested" role="menuitem" aria-haspopup="true"> Research areas <span class="icon icon--caret"></span> </button> <div class="navigation__nested-sub js-sub-nav-parent"> <div class="navigation__sub__mobile-heading"> <button class="glue-header__link js-sub-nav-close-mobile" role="menuitem" aria-haspopup="true"> <span class="sr-text">Back to</span> <span class="icon icon--caret"></span> Research areas <span class="sr-text">menu</span> </button> <hr/> </div> <ul> <li role="menuitem"> <a href="http://research.google/research-areas/" class="navigation__sub__columns__mobile__link" > Explore all research areas <span> </span> </a> </li> </ul> </div> </div> </li> <li data-gt-secondary="Foundational ML & Algorithms"> <div class="navigation__sub__columns__desktop"> <h2 class="headline-6 navigation__sub__columns__heading">Foundational ML & Algorithms</h2> <ul> <li> <a class="navigation__sub__columns__list-link caption js-drawer-link" href="http://research.google/research-areas/algorithms-and-theory/" > Algorithms & Theory </a> </li> <li> <a class="navigation__sub__columns__list-link caption js-drawer-link" href="http://research.google/research-areas/data-management/" > Data Management </a> </li> <li> <a class="navigation__sub__columns__list-link caption js-drawer-link" href="http://research.google/research-areas/data-mining-and-modeling/" > Data Mining & Modeling </a> </li> <li> <a class="navigation__sub__columns__list-link caption js-drawer-link" href="http://research.google/research-areas/information-retrieval-and-the-web/" > Information Retrieval & the Web </a> </li> <li> <a class="navigation__sub__columns__list-link caption js-drawer-link" href="http://research.google/research-areas/machine-intelligence/" > Machine Intelligence </a> </li> <li> <a class="navigation__sub__columns__list-link caption js-drawer-link" href="http://research.google/research-areas/machine-perception/" > Machine Perception </a> </li> <li> <a class="navigation__sub__columns__list-link caption js-drawer-link" href="http://research.google/research-areas/machine-translation/" > Machine Translation </a> </li> <li> <a class="navigation__sub__columns__list-link caption js-drawer-link" href="http://research.google/research-areas/natural-language-processing/" > Natural Language Processing </a> </li> <li> <a class="navigation__sub__columns__list-link caption js-drawer-link" href="http://research.google/research-areas/speech-processing/" > Speech Processing </a> </li> </ul> </div> <div class="navigation__sub__columns__mobile"> <button class="glue-header__link js-sub-nav-target" data-panel="nested" role="menuitem" aria-haspopup="true"> Foundational ML & Algorithms <span class="icon icon--caret"></span> </button> <div class="navigation__nested-sub js-sub-nav-parent"> <div class="navigation__sub__mobile-heading"> <button class="glue-header__link js-sub-nav-close-mobile" role="menuitem" aria-haspopup="true"> <span class="sr-text">Back to</span> <span class="icon icon--caret"></span> Foundational ML & Algorithms <span class="sr-text">menu</span> </button> <hr/> </div> <ul> <li role="menuitem"> <a href="http://research.google/research-areas/algorithms-and-theory/" class="navigation__sub__columns__mobile__link" > Algorithms & Theory <span> </span> </a> </li> <li role="menuitem"> <a href="http://research.google/research-areas/data-management/" class="navigation__sub__columns__mobile__link" > Data Management <span> </span> </a> </li> <li role="menuitem"> <a href="http://research.google/research-areas/data-mining-and-modeling/" class="navigation__sub__columns__mobile__link" > Data Mining & Modeling <span> </span> </a> </li> <li role="menuitem"> <a href="http://research.google/research-areas/information-retrieval-and-the-web/" class="navigation__sub__columns__mobile__link" > Information Retrieval & the Web <span> </span> </a> </li> <li role="menuitem"> <a href="http://research.google/research-areas/machine-intelligence/" class="navigation__sub__columns__mobile__link" > Machine Intelligence <span> </span> </a> </li> <li role="menuitem"> <a href="http://research.google/research-areas/machine-perception/" class="navigation__sub__columns__mobile__link" > Machine Perception <span> </span> </a> </li> <li role="menuitem"> <a href="http://research.google/research-areas/machine-translation/" class="navigation__sub__columns__mobile__link" > Machine Translation <span> </span> </a> </li> <li role="menuitem"> <a href="http://research.google/research-areas/natural-language-processing/" class="navigation__sub__columns__mobile__link" > Natural Language Processing <span> </span> </a> </li> <li role="menuitem"> <a href="http://research.google/research-areas/speech-processing/" class="navigation__sub__columns__mobile__link" > Speech Processing <span> </span> </a> </li> </ul> </div> </div> </li> <li data-gt-secondary="Computing Systems & Quantum AI"> <div class="navigation__sub__columns__desktop"> <h2 class="headline-6 navigation__sub__columns__heading">Computing Systems & Quantum AI</h2> <ul> <li> <a class="navigation__sub__columns__list-link caption js-drawer-link" href="http://research.google/research-areas/distributed-systems-and-parallel-computing/" > Distributed Systems & Parallel Computing </a> </li> <li> <a class="navigation__sub__columns__list-link caption js-drawer-link" href="http://research.google/research-areas/hardware-and-architecture/" > Hardware & Architecture </a> </li> <li> <a class="navigation__sub__columns__list-link caption js-drawer-link" href="http://research.google/research-areas/mobile-systems/" > Mobile Systems </a> </li> <li> <a class="navigation__sub__columns__list-link caption js-drawer-link" href="http://research.google/research-areas/networking/" > Networking </a> </li> <li> <a class="navigation__sub__columns__list-link caption js-drawer-link" href="http://research.google/research-areas/quantum-computing/" > Quantum Computing </a> </li> <li> <a class="navigation__sub__columns__list-link caption js-drawer-link" href="http://research.google/research-areas/robotics/" > Robotics </a> </li> <li> <a class="navigation__sub__columns__list-link caption js-drawer-link" href="http://research.google/research-areas/security-privacy-and-abuse-prevention/" > Security, Privacy, & Abuse Prevention </a> </li> <li> <a class="navigation__sub__columns__list-link caption js-drawer-link" href="http://research.google/research-areas/software-engineering/" > Software Engineering </a> </li> <li> <a class="navigation__sub__columns__list-link caption js-drawer-link" href="http://research.google/research-areas/software-systems/" > Software Systems </a> </li> </ul> </div> <div class="navigation__sub__columns__mobile"> <button class="glue-header__link js-sub-nav-target" data-panel="nested" role="menuitem" aria-haspopup="true"> Computing Systems & Quantum AI <span class="icon icon--caret"></span> </button> <div class="navigation__nested-sub js-sub-nav-parent"> <div class="navigation__sub__mobile-heading"> <button class="glue-header__link js-sub-nav-close-mobile" role="menuitem" aria-haspopup="true"> <span class="sr-text">Back to</span> <span class="icon icon--caret"></span> Computing Systems & Quantum AI <span class="sr-text">menu</span> </button> <hr/> </div> <ul> <li role="menuitem"> <a href="http://research.google/research-areas/distributed-systems-and-parallel-computing/" class="navigation__sub__columns__mobile__link" > Distributed Systems & Parallel Computing <span> </span> </a> </li> <li role="menuitem"> <a href="http://research.google/research-areas/hardware-and-architecture/" class="navigation__sub__columns__mobile__link" > Hardware & Architecture <span> </span> </a> </li> <li role="menuitem"> <a href="http://research.google/research-areas/mobile-systems/" class="navigation__sub__columns__mobile__link" > Mobile Systems <span> </span> </a> </li> <li role="menuitem"> <a href="http://research.google/research-areas/networking/" class="navigation__sub__columns__mobile__link" > Networking <span> </span> </a> </li> <li role="menuitem"> <a href="http://research.google/research-areas/quantum-computing/" class="navigation__sub__columns__mobile__link" > Quantum Computing <span> </span> </a> </li> <li role="menuitem"> <a href="http://research.google/research-areas/robotics/" class="navigation__sub__columns__mobile__link" > Robotics <span> </span> </a> </li> <li role="menuitem"> <a href="http://research.google/research-areas/security-privacy-and-abuse-prevention/" class="navigation__sub__columns__mobile__link" > Security, Privacy, & Abuse Prevention <span> </span> </a> </li> <li role="menuitem"> <a href="http://research.google/research-areas/software-engineering/" class="navigation__sub__columns__mobile__link" > Software Engineering <span> </span> </a> </li> <li role="menuitem"> <a href="http://research.google/research-areas/software-systems/" class="navigation__sub__columns__mobile__link" > Software Systems <span> </span> </a> </li> </ul> </div> </div> </li> <li data-gt-secondary="Science, AI & Society"> <div class="navigation__sub__columns__desktop"> <h2 class="headline-6 navigation__sub__columns__heading">Science, AI & Society</h2> <ul> <li> <a class="navigation__sub__columns__list-link caption js-drawer-link" href="http://research.google/research-areas/climate-and-sustainability/" > Climate & Sustainability </a> </li> <li> <a class="navigation__sub__columns__list-link caption js-drawer-link" href="http://research.google/research-areas/economics-and-electronic-commerce/" > Economics & Electronic Commerce </a> </li> <li> <a class="navigation__sub__columns__list-link caption js-drawer-link" href="http://research.google/research-areas/education-innovation/" > Education Innovation </a> </li> <li> <a class="navigation__sub__columns__list-link caption js-drawer-link" href="http://research.google/research-areas/general-science/" > General Science </a> </li> <li> <a class="navigation__sub__columns__list-link caption js-drawer-link" href="http://research.google/research-areas/health-bioscience/" > Health & Bioscience </a> </li> <li> <a class="navigation__sub__columns__list-link caption js-drawer-link" href="http://research.google/research-areas/human-computer-interaction-and-visualization/" > Human-Computer Interaction and Visualization </a> </li> </ul> </div> <div class="navigation__sub__columns__mobile"> <button class="glue-header__link js-sub-nav-target" data-panel="nested" role="menuitem" aria-haspopup="true"> Science, AI & Society <span class="icon icon--caret"></span> </button> <div class="navigation__nested-sub js-sub-nav-parent"> <div class="navigation__sub__mobile-heading"> <button class="glue-header__link js-sub-nav-close-mobile" role="menuitem" aria-haspopup="true"> <span class="sr-text">Back to</span> <span class="icon icon--caret"></span> Science, AI & Society <span class="sr-text">menu</span> </button> <hr/> </div> <ul> <li role="menuitem"> <a href="http://research.google/research-areas/climate-and-sustainability/" class="navigation__sub__columns__mobile__link" > Climate & Sustainability <span> </span> </a> </li> <li role="menuitem"> <a href="http://research.google/research-areas/economics-and-electronic-commerce/" class="navigation__sub__columns__mobile__link" > Economics & Electronic Commerce <span> </span> </a> </li> <li role="menuitem"> <a href="http://research.google/research-areas/education-innovation/" class="navigation__sub__columns__mobile__link" > Education Innovation <span> </span> </a> </li> <li role="menuitem"> <a href="http://research.google/research-areas/general-science/" class="navigation__sub__columns__mobile__link" > General Science <span> </span> </a> </li> <li role="menuitem"> <a href="http://research.google/research-areas/health-bioscience/" class="navigation__sub__columns__mobile__link" > Health & Bioscience <span> </span> </a> </li> <li role="menuitem"> <a href="http://research.google/research-areas/human-computer-interaction-and-visualization/" class="navigation__sub__columns__mobile__link" > Human-Computer Interaction and Visualization <span> </span> </a> </li> </ul> </div> </div> </li> </ul> </div> </div></div> </div> </div> </li> <li class="glue-header__item js-sub-nav-parent --parent" data-gt-primary="Our work" > <button class="glue-header__link js-sub-nav-target" aria-haspopup="true" aria-expanded="false" > <span class=""> Our work <span class="icon icon--caret"></span> </span> </button> <div class="navigation__sub js-sub-nav" role="menu"> <div class="navigation__sub__container"> <div class="navigation__sub__mobile-heading"> <button class="glue-header__link js-sub-nav-close-mobile"> <span class="sr-text">Back to</span> <span class="icon icon--caret"></span> Our work <span class="sr-text">menu</span> </button> <hr/> </div> <div class="block-nav_drawer_columns_content"> <div class="navigation__sub--content" data-gt-secondary=""> <div class="navigation__sub__wrapper"> <ul class="navigation__sub__columns"> <li data-gt-secondary="Projects"> <div class="navigation__sub__columns__desktop"> <h2 class="headline-6 navigation__sub__columns__heading"> Projects </h2> <p class="navigation__sub__columns__description caption">We regularly open-source projects with the broader research community and apply our developments to Google products.</p> <a href="http://research.google/resources/our-projects/" class="glue-inline-link js-drawer-link" > <span class="sr-text">Learn more about our Projects</span> <span aria-hidden="true">Learn more</span> </a> </div> <div class="navigation__sub__columns__mobile"> <a class="glue-header__link" href="http://research.google/resources/our-projects/" > Projects </a> </div> </li> <li data-gt-secondary="Publications"> <div class="navigation__sub__columns__desktop"> <h2 class="headline-6 navigation__sub__columns__heading"> Publications </h2> <p class="navigation__sub__columns__description caption">Publishing our work allows us to share ideas and work collaboratively to advance the field of computer science.</p> <a href="http://research.google/pubs/" class="glue-inline-link js-drawer-link" > <span class="sr-text">Learn more about our Publications</span> <span aria-hidden="true">Learn more</span> </a> </div> <div class="navigation__sub__columns__mobile"> <a class="glue-header__link" href="http://research.google/pubs/" > Publications </a> </div> </li> <li data-gt-secondary="Resources"> <div class="navigation__sub__columns__desktop"> <h2 class="headline-6 navigation__sub__columns__heading"> Resources </h2> <p class="navigation__sub__columns__description caption">We make products, tools, and datasets available to everyone with the goal of building a more collaborative ecosystem.</p> <a href="http://research.google/resources/" class="glue-inline-link js-drawer-link" > <span class="sr-text">Learn more about our Resources</span> <span aria-hidden="true">Learn more</span> </a> </div> <div class="navigation__sub__columns__mobile"> <a class="glue-header__link" href="http://research.google/resources/" > Resources </a> </div> </li> </ul> </div> </div> </div> </div> </div> </li> <li class="glue-header__item js-sub-nav-parent --parent" data-gt-primary="Programs & events" > <button class="glue-header__link js-sub-nav-target" aria-haspopup="true" aria-expanded="false" > <span class=""> Programs & events <span class="icon icon--caret"></span> </span> </button> <div class="navigation__sub js-sub-nav" role="menu"> <div class="navigation__sub__container"> <div class="navigation__sub__mobile-heading"> <button class="glue-header__link js-sub-nav-close-mobile"> <span class="sr-text">Back to</span> <span class="icon icon--caret"></span> Programs & events <span class="sr-text">menu</span> </button> <hr/> </div> <div class="block-nav_drawer_columns_content"> <div class="navigation__sub--content" data-gt-secondary="Shaping the future, together."> <div class="navigation__sub__wrapper"> <div class="navigation__sub__heading"> <h2 class="headline-3">Shaping the future, together.</h2> <a href="http://research.google/programs-and-events/" class="js-drawer-link" > Collaborate with us </a> </div> <ul class="navigation__sub__columns"> <li data-gt-secondary="Student programs"> <div class="navigation__sub__columns__desktop"> <h2 class="headline-6 navigation__sub__columns__heading"> Student programs </h2> <p class="navigation__sub__columns__description caption">Supporting the next generation of researchers through a wide range of programming.</p> <a href="http://research.google/programs-and-events/student-engagement/" class="glue-inline-link js-drawer-link" > <span class="sr-text">Learn more about our Student programs</span> <span aria-hidden="true">Learn more</span> </a> </div> <div class="navigation__sub__columns__mobile"> <a class="glue-header__link" href="http://research.google/programs-and-events/student-engagement/" > Student programs </a> </div> </li> <li data-gt-secondary="Faculty programs"> <div class="navigation__sub__columns__desktop"> <h2 class="headline-6 navigation__sub__columns__heading"> Faculty programs </h2> <p class="navigation__sub__columns__description caption">Participating in the academic research community through meaningful engagement with university faculty.</p> <a href="http://research.google/programs-and-events/faculty-engagement/" class="glue-inline-link js-drawer-link" > <span class="sr-text">Learn more about our Faculty programs</span> <span aria-hidden="true">Learn more</span> </a> </div> <div class="navigation__sub__columns__mobile"> <a class="glue-header__link" href="http://research.google/programs-and-events/faculty-engagement/" > Faculty programs </a> </div> </li> <li data-gt-secondary="Conferences & events"> <div class="navigation__sub__columns__desktop"> <h2 class="headline-6 navigation__sub__columns__heading"> Conferences & events </h2> <p class="navigation__sub__columns__description caption">Connecting with the broader research community through events is essential for creating progress in every aspect of our work.</p> <a href="http://research.google/conferences-and-events/" class="glue-inline-link js-drawer-link" > <span class="sr-text">Learn more about our Conferences & events</span> <span aria-hidden="true">Learn more</span> </a> </div> <div class="navigation__sub__columns__mobile"> <a class="glue-header__link" href="http://research.google/conferences-and-events/" > Conferences & events </a> </div> </li> </ul> <div class="navigation__sub__cta"> <a class="glue-button glue-button--high-emphasis js-drawer-link" href="http://research.google/programs-and-events/" target="_blank" rel="noreferrer noopener" > Collaborate with us </a> </div> </div> </div> </div> </div> </div> </li> <li class="glue-header__item " data-gt-primary="Careers" > <a class="glue-header__link " href="http://research.google/careers/" > <span class=""> Careers </span> </a> </li> <li class="glue-header__item " data-gt-primary="Blog" > <a class="glue-header__link " href="http://research.google/blog/" > <span class=""> Blog </span> </a> </li> </ul> </nav> </div>  <div class="glue-header__search js-header-search"> <div class="glue-header__search__input"> <div class="search-input " data-type="header"> <input type="search" class="caption --empty-search js-search-bar js-gt-search-input" placeholder="Search"> <button class="search-input__button --search js-gt-search-btn"> <svg role="presentation" aria-hidden="true" class="glue-icon glue-icon--18px "> <use href="/gr/static/assets/icons/glue-icons.svg#search"></use> </svg> </button> <button class="search-input__button --clear"> <svg role="presentation" aria-hidden="true" class="glue-icon glue-icon--18px "> <use href="/gr/static/assets/icons/glue-icons.svg#close"></use> </svg> </button> </div> </div> <button class="glue-header__search__btn js-header-search-btn"> <svg role="presentation" aria-hidden="true" aria-hidden="true" class="glue-icon glue-icon--24px search"> <use href="/gr/static/assets/icons/glue-icons.svg#search"></use> </svg> <svg role="presentation" aria-hidden="true" aria-hidden="true" class="glue-icon glue-icon--24px close"> <use href="/gr/static/assets/icons/glue-icons.svg#close"></use> </svg> <span class="sr-text js-header-search-sr-text">Search</span> </button> </div> </div> </div> <div class="glue-header__drawer-backdrop"> <div class="glue-header__mobile_close"> <button class="glue-header__drawer-toggle-btn js-mobile-nav-close" aria-label="Close the navigation drawer"> <svg class="glue-icon glue-icon--24px" role="presentation" aria-hidden="true"> <use href="/gr/static/assets/icons/glue-icons.svg#close"></use> </svg> </button> </div> </div> </header> <main id="page-content"> <div class="research-area-detail"> <section class="basic-hero bhoig --theme-dark " data-gt-id="basic_hero" data-gt-component-name=""> <div class="glue-page"> <div class="glue-grid"> <div class="bhoig__image-wrapper glue-grid__col--span-2 glue-grid__col--span-3-md glue-grid__col--span-2-lg"> <div class="bhoig__image-bg" style="background-color: var(--glue-grey-900); "> <span class="icon icon--robotics" style="" ></span> </div> </div> <div class="bhoig__breadcrumb-wrapper glue-grid__col--span-10 glue-grid__col--span-9-md glue-grid__col--span-10-lg"> <nav class="glue-breadcrumbs" aria-label="Breadcrumbs"> <ol class="glue-breadcrumbs__list"> <li class="glue-breadcrumbs__item"> <a class="glue-breadcrumbs__link attribution" href="/">Home</a> <svg role="presentation" aria-hidden="true" class="glue-icon "> <use href="/gr/static/assets/icons/glue-icons.svg#chevron-right"></use> </svg> </li> <li class="glue-breadcrumbs__item"> <a class="glue-breadcrumbs__link attribution" href="/research-areas/">Research areas</a> <svg role="presentation" aria-hidden="true" class="glue-icon "> <use href="/gr/static/assets/icons/glue-icons.svg#chevron-right"></use> </svg> </li> </ol> </nav> </div> <h1 class="headline-1 bhoig__headline glue-grid__col--span-10 glue-grid__col--span-9-md glue-grid__col--span-10-lg">Robotics</h1> <div class="basic-hero__description bhoig__description glue-grid__col--span-10 glue-grid__col--span-9-md glue-grid__col--span-10-lg"> <p data-block-key="ctnw0">Having a machine learning agent interact with its environment requires true unsupervised learning, skill acquisition, active learning, exploration and reinforcement, all ingredients of human learning that are still not well understood or exploited through the supervised approaches that dominate deep learning today. Our goal is to improve robotics via machine learning, and improve machine learning via robotics. We foster close collaborations between machine learning researchers and roboticists to enable learning at scale on real and simulated robotic systems.</p> </div> <div class="bhoig__cta glue-grid__col--span-10 glue-grid__col--span-9-md glue-grid__col--span-10-lg"> </div> </div> </div> </section> <section class="offset-two-up"> <div class="glue-page glue-grid"> <div class="offset-two-up__left-col glue-grid__col glue-grid__col--span-4-sm glue-grid__col--span-12-md glue-grid__col--span-3-lg"> <h3 class="offset-two-up__headline headline-3">Recent Publications</h3> <div class="offset-two-up__cta-below-description"> <a class="glue-button glue-button--medium-emphasis" href="http://research.google/pubs/?category=robotics" > <span class="js-gt-item-id">See More</span> </a> </div> </div> <div class="glue-grid__col glue-grid__col--span-4-sm glue-grid__col--span-12-md glue-grid__col--span-9-lg"> <div class="publications-list --theme- row-card-list" data-hot-swap="pub-list" data-gt-id="publications_list" data-gt-component-name=""> <div class="row-card"> <div class="row-card__container"> <div class="row-card__body"> <a class="row-card__heading headline-6 glue-link" href=http://research.google/pubs/clara-classifying-and-disambiguating-user-commands-for-reliable-interactive-robotic-agents/ > CLARA: Classifying and Disambiguating User Commands for Reliable Interactive Robotic Agents </a> <div class="row-card__subheading"> <div class="row-card__subheading__item extra-small-text"> Jeongeun Park </div> <div class="row-card__subheading__spacer"></div> <div class="row-card__subheading__item extra-small-text"> Seungwon Lim </div> <div class="row-card__subheading__spacer"></div> <div class="row-card__subheading__item extra-small-text"> Joonhyung Lee </div> <div class="row-card__subheading__spacer"></div> <div class="row-card__subheading__item extra-small-text"> Sangbeom Park </div> <div class="row-card__subheading__spacer"></div> <div class="row-card__subheading__item extra-small-text"> <a class="row-card__small-link" href="/people/minsukchang/"> Minsuk Chang </a> </div> <div class="row-card__subheading__spacer"></div> <div class="row-card__subheading__item extra-small-text"> Sungjoon Choi </div> <div class="row-card__subheading__spacer"></div> <div class="row-card__subheading__item extra-small-text"> Youngjae Yu </div> <div class="row-card__subheading__spacer"></div> <div class="row-card__subheading__item extra-small-text"> IEEE Robotics and Automation Letters (2023) (to appear) </div> </div> </div> <div class="row-card__cta headline-6"> <div class="glue-tooltip" data-glue-tooltip-auto-position="false"> <button class="glue-button glue-button--low-emphasis glue-tooltip__trigger" aria-describedby=tooltip-contentin-this-paper-we-focus-on-inferri tabindex=0 > <span class="js-gt-item-id">Preview</span> </button> <span id="tooltip-contentin-this-paper-we-focus-on-inferri" class="glue-tooltip__content" role="tooltip"> <span data-tooltip-type="simple"> Preview abstract </span> <span data-tooltip-type="rich"> <span class="glue-tooltip__body">In this paper, we focus on inferring whether the given user command is clear, ambiguous, or infeasible in the context of interactive robotic agents utilizing large language models (LLMs). To tackle this problem, we first present an uncertainty estimation method for LLMs to classify whether the command is certain (i.e., clear) or not (i.e., ambiguous or infeasible). Once the command is classified as uncertain, we further distinguish it between ambiguous or infeasible commands leveraging LLMs with situational aware few-shot prompting in a zero-shot manner. For ambiguous commands, we further disambiguate the command by interacting with users via question generation with LLMs. We believe that proper recognition of the given commands could lead to a decrease in malfunction and undesired actions of the robot, enhancing the reliability of interactive robot agents. To evaluate the proposed system, we present a dataset consisting pair of high-level commands, scene descriptions, and labels of command type (i.e., clear, ambiguous, or infeasible). We validate the proposed method on the collected dataset, pick-and-place tabletop simulation. Furthermore, we demonstrate the approach in a real-world human-robot interaction environment, i.e., handover scenarios.</span> <a class="glue-button glue-button--low-emphasis" href="http://research.google/pubs/clara-classifying-and-disambiguating-user-commands-for-reliable-interactive-robotic-agents/" > <span class="js-gt-item-id">View details</span> </a> </span> </span> </div> </div> </div> </div> <div class="row-card"> <div class="row-card__container"> <div class="row-card__body"> <a class="row-card__heading headline-6 glue-link" href=http://research.google/pubs/robotic-table-tennis-a-case-study-into-a-high-speed-learning-system/ > Robotic Table Tennis: A Case Study into a High Speed Learning System </a> <div class="row-card__subheading"> <div class="row-card__subheading__item extra-small-text"> <a class="row-card__small-link" href="/people/davidbdambrosio/"> David B. D'Ambrosio </a> </div> <div class="row-card__subheading__spacer"></div> <div class="row-card__subheading__item extra-small-text"> Jon Abelian </div> <div class="row-card__subheading__spacer"></div> <div class="row-card__subheading__item extra-small-text"> Saminda Abeyruwan </div> <div class="row-card__subheading__spacer"></div> <div class="row-card__subheading__item extra-small-text"> Michael Ahn </div> <div class="row-card__subheading__spacer"></div> <div class="row-card__subheading__item extra-small-text"> <a class="row-card__small-link" href="/people/bewley/"> Alex Bewley </a> </div> <div class="row-card__subheading__spacer"></div> <div class="row-card__subheading__item extra-small-text"> Justin Boyd </div> <div class="row-card__subheading__spacer"></div> <div class="row-card__subheading__item extra-small-text"> <a class="row-card__small-link" href="/people/krzysztofchoromanski/"> Krzysztof Choromanski </a> </div> <div class="row-card__subheading__spacer"></div> <div class="row-card__subheading__item extra-small-text"> Erwin Johan Coumans </div> <div class="row-card__subheading__spacer"></div> <div class="row-card__subheading__item extra-small-text"> <a class="row-card__small-link" href="/people/107546/"> Tianli Ding </a> </div> <div class="row-card__subheading__spacer"></div> <div class="row-card__subheading__item extra-small-text"> Omar Escareno </div> <div class="row-card__subheading__spacer"></div> <div class="row-card__subheading__item extra-small-text"> Wenbo Gao </div> <div class="row-card__subheading__spacer"></div> <div class="row-card__subheading__item extra-small-text"> <a class="row-card__small-link" href="/people/108185/"> Laura Graesser </a> </div> <div class="row-card__subheading__spacer"></div> <div class="row-card__subheading__item extra-small-text"> <a class="row-card__small-link" href="/people/105360/"> Atil Iscen </a> </div> <div class="row-card__subheading__spacer"></div> <div class="row-card__subheading__item extra-small-text"> Navdeep Jaitly </div> <div class="row-card__subheading__spacer"></div> <div class="row-card__subheading__item extra-small-text"> <a class="row-card__small-link" href="/people/deepali/"> Deepali Jain </a> </div> <div class="row-card__subheading__spacer"></div> <div class="row-card__subheading__item extra-small-text"> Juhana Kangaspunta </div> <div class="row-card__subheading__spacer"></div> <div class="row-card__subheading__item extra-small-text"> Satoshi Kataoka </div> <div class="row-card__subheading__spacer"></div> <div class="row-card__subheading__item extra-small-text"> Gus Kouretas </div> <div class="row-card__subheading__spacer"></div> <div class="row-card__subheading__item extra-small-text"> Yuheng Kuang </div> <div class="row-card__subheading__spacer"></div> <div class="row-card__subheading__item extra-small-text"> <a class="row-card__small-link" href="/people/104936/"> Nevena Lazic </a> </div> <div class="row-card__subheading__spacer"></div> <div class="row-card__subheading__item extra-small-text"> Corey Lynch </div> <div class="row-card__subheading__spacer"></div> <div class="row-card__subheading__item extra-small-text"> <a class="row-card__small-link" href="/people/106158/"> Reza Mahjourian </a> </div> <div class="row-card__subheading__spacer"></div> <div class="row-card__subheading__item extra-small-text"> <a class="row-card__small-link" href="/people/104893/"> Sherry Moore </a> </div> <div class="row-card__subheading__spacer"></div> <div class="row-card__subheading__item extra-small-text"> Thinh Nguyen </div> <div class="row-card__subheading__spacer"></div> <div class="row-card__subheading__item extra-small-text"> Ken Oslund </div> <div class="row-card__subheading__spacer"></div> <div class="row-card__subheading__item extra-small-text"> Barney J. Reed </div> <div class="row-card__subheading__spacer"></div> <div class="row-card__subheading__item extra-small-text"> <a class="row-card__small-link" href="/people/107599/"> Krista Reymann </a> </div> <div class="row-card__subheading__spacer"></div> <div class="row-card__subheading__item extra-small-text"> <a class="row-card__small-link" href="/people/pannagrsanketi/"> Pannag Sanketi </a> </div> <div class="row-card__subheading__spacer"></div> <div class="row-card__subheading__item extra-small-text"> Anish Shankar </div> <div class="row-card__subheading__spacer"></div> <div class="row-card__subheading__item extra-small-text"> <a class="row-card__small-link" href="/people/pierresermanet/"> Pierre Sermanet </a> </div> <div class="row-card__subheading__spacer"></div> <div class="row-card__subheading__item extra-small-text"> <a class="row-card__small-link" href="/people/vikassindhwani/"> Vikas Sindhwani </a> </div> <div class="row-card__subheading__spacer"></div> <div class="row-card__subheading__item extra-small-text"> Avi Singh </div> <div class="row-card__subheading__spacer"></div> <div class="row-card__subheading__item extra-small-text"> <a class="row-card__small-link" href="/people/vincentvanhoucke/"> Vincent Vanhoucke </a> </div> <div class="row-card__subheading__spacer"></div> <div class="row-card__subheading__item extra-small-text"> Grace Vesom </div> <div class="row-card__subheading__spacer"></div> <div class="row-card__subheading__item extra-small-text"> Peng Xu </div> <div class="row-card__subheading__spacer"></div> <div class="row-card__subheading__item extra-small-text"> Robotics: Science and Systems (2023) </div> </div> </div> <div class="row-card__cta headline-6"> <div class="glue-tooltip" data-glue-tooltip-auto-position="false"> <button class="glue-button glue-button--low-emphasis glue-tooltip__trigger" aria-describedby=tooltip-contentwe-present-a-deep-dive-into-a-lear tabindex=0 > <span class="js-gt-item-id">Preview</span> </button> <span id="tooltip-contentwe-present-a-deep-dive-into-a-lear" class="glue-tooltip__content" role="tooltip"> <span data-tooltip-type="simple"> Preview abstract </span> <span data-tooltip-type="rich"> <span class="glue-tooltip__body">We present a deep-dive into a learning robotic system that, in previous work, was shown to be capable of hundreds of table tennis rallies with a human and has the ability to precisely return the ball to desired targets. This system puts together a highly optimized and novel perception subsystem, a high-speed low-latency robot controller, a simulation paradigm that can prevent damage in the real world and also train policies for zero-shot transfer, and automated real world environment resets that enable autonomous training and evaluation on physical robots. We complement a complete system description including numerous design decisions that are typically not widely disseminated, with a collection of ablation studies that clarify the importance of mitigating various sources of latency, accounting for training and deployment distribution shifts, robustness of the perception system, and sensitivity to policy hyper-parameters and choice of action space. A video demonstrating the components of our system and details of experimental results is included in the supplementary material.</span> <a class="glue-button glue-button--low-emphasis" href="http://research.google/pubs/robotic-table-tennis-a-case-study-into-a-high-speed-learning-system/" > <span class="js-gt-item-id">View details</span> </a> </span> </span> </div> </div> </div> </div> <div class="row-card"> <div class="row-card__container"> <div class="row-card__body"> <a class="row-card__heading headline-6 glue-link" href=http://research.google/pubs/a-connection-between-actor-regularization-and-critic-regularization-in-reinforcement-learning/ > A Connection between Actor Regularization and Critic Regularization in Reinforcement Learning </a> <div class="row-card__subheading"> <div class="row-card__subheading__item extra-small-text"> Benjamin Eysenbach </div> <div class="row-card__subheading__spacer"></div> <div class="row-card__subheading__item extra-small-text"> Matthieu Geist </div> <div class="row-card__subheading__spacer"></div> <div class="row-card__subheading__item extra-small-text"> Ruslan Salakhutdinov </div> <div class="row-card__subheading__spacer"></div> <div class="row-card__subheading__item extra-small-text"> Sergey Levine </div> <div class="row-card__subheading__spacer"></div> <div class="row-card__subheading__item extra-small-text"> International Conference on Machine Learning (ICML) (2023) </div> </div> </div> <div class="row-card__cta headline-6"> <div class="glue-tooltip" data-glue-tooltip-auto-position="false"> <button class="glue-button glue-button--low-emphasis glue-tooltip__trigger" aria-describedby=tooltip-contentas-with-any-machine-learning-probl tabindex=0 > <span class="js-gt-item-id">Preview</span> </button> <span id="tooltip-contentas-with-any-machine-learning-probl" class="glue-tooltip__content" role="tooltip"> <span data-tooltip-type="simple"> Preview abstract </span> <span data-tooltip-type="rich"> <span class="glue-tooltip__body">As with any machine learning problem with limited data, effective offline RL algorithms require careful regularization to avoid overfitting, with most methods regularizing either the actor or the critic. These methods appear distinct. Actor regularization (e.g., behavioral cloning penalties) is simpler and has appealing convergence properties, while critic regularization typically requires significantly more compute because it involves solving a game, but it has appealing lower-bound guarantees. Empirically, prior work alternates between claiming better results with actor regularization and critic regularization. In this paper, we show that these two regularization techniques can be equivalent under some assumptions: regularizing the critic using a CQL-like objective is equivalent to updating the actor with a BC- like regularizer and with a SARSA Q-value (i.e., “1-step RL”). Our experiments show that this theoretical model makes accurate, testable predictions about the performance of CQL and one-step RL. While our results do not definitively say whether users should prefer actor regularization or critic regularization, our results hint that actor regularization methods may be a simpler way to achieve the desirable properties of critic regularization. The results also suggest that the empirically- demonstrated benefits of both types of regularization might be more a function of implementation details rather than objective superiority.</span> <a class="glue-button glue-button--low-emphasis" href="http://research.google/pubs/a-connection-between-actor-regularization-and-critic-regularization-in-reinforcement-learning/" > <span class="js-gt-item-id">View details</span> </a> </span> </span> </div> </div> </div> </div> <div class="row-card"> <div class="row-card__container"> <div class="row-card__body"> <a class="row-card__heading headline-6 glue-link" href=http://research.google/pubs/robotic-skill-acquisition-via-instruction-augmentation-with-vision-language-models/ > Robotic Skill Acquisition via Instruction Augmentation with Vision-Language Models </a> <div class="row-card__subheading"> <div class="row-card__subheading__item extra-small-text"> <a class="row-card__small-link" href="/people/106928/"> Ted Xiao </a> </div> <div class="row-card__subheading__spacer"></div> <div class="row-card__subheading__item extra-small-text"> Harris Chan </div> <div class="row-card__subheading__spacer"></div> <div class="row-card__subheading__item extra-small-text"> <a class="row-card__small-link" href="/people/pierresermanet/"> Pierre Sermanet </a> </div> <div class="row-card__subheading__spacer"></div> <div class="row-card__subheading__item extra-small-text"> <a class="row-card__small-link" href="/people/ayzaanwahid/"> Ayzaan Wahid </a> </div> <div class="row-card__subheading__spacer"></div> <div class="row-card__subheading__item extra-small-text"> Anthony Brohan </div> <div class="row-card__subheading__spacer"></div> <div class="row-card__subheading__item extra-small-text"> Karol Hausman </div> <div class="row-card__subheading__spacer"></div> <div class="row-card__subheading__item extra-small-text"> Sergey Levine </div> <div class="row-card__subheading__spacer"></div> <div class="row-card__subheading__item extra-small-text"> <a class="row-card__small-link" href="/people/107216/"> Jonathan Tompson </a> </div> <div class="row-card__subheading__spacer"></div> <div class="row-card__subheading__item extra-small-text"> RSS 2023 (2023) </div> </div> </div> <div class="row-card__cta headline-6"> <div class="glue-tooltip" data-glue-tooltip-auto-position="false"> <button class="glue-button glue-button--low-emphasis glue-tooltip__trigger" aria-describedby=tooltip-contentin-recent-years-much-progress-has tabindex=0 > <span class="js-gt-item-id">Preview</span> </button> <span id="tooltip-contentin-recent-years-much-progress-has" class="glue-tooltip__content" role="tooltip"> <span data-tooltip-type="simple"> Preview abstract </span> <span data-tooltip-type="rich"> <span class="glue-tooltip__body">In recent years, much progress has been made in learning robotic manipulation policies that can follow natural language instructions. Common approaches involve learning methods that operate on offline datasets, such as task-specific teleoperated demonstrations or on hindsight labeled robotic experience. Such methods work reasonably but rely strongly on the assumption of clean data: teleoperated demonstrations are collected with specific tasks in mind, while hindsight language descriptions rely on expensive human labeling. Recently, large-scale pretrained language and vision-language models like CLIP have been applied to robotics in the form of learning representations and planners. However, can these pretrained models also be used to cheaply impart internet-scale knowledge onto offline datasets, providing access to skills contained in the offline dataset that weren't necessarily reflected in ground truth labels? We investigate fine-tuning a reward model on a small dataset of robot interactions with crowd-sourced natural language labels and using the model to relabel instructions of a large offline robot dataset. The resulting dataset with diverse language skills is used to train imitation learning policies, which outperform prior methods by up to 30% when evaluated on a diverse set of novel language instructions that were not contained in the original dataset.</span> <a class="glue-button glue-button--low-emphasis" href="http://research.google/pubs/robotic-skill-acquisition-via-instruction-augmentation-with-vision-language-models/" > <span class="js-gt-item-id">View details</span> </a> </span> </span> </div> </div> </div> </div> <div class="row-card"> <div class="row-card__container"> <div class="row-card__body"> <a class="row-card__heading headline-6 glue-link" href=http://research.google/pubs/agile-catching-with-whole-body-mpc-and-blackbox-policy-learning/ > Agile Catching with Whole-Body MPC and Blackbox Policy Learning </a> <div class="row-card__subheading"> <div class="row-card__subheading__item extra-small-text"> Saminda Abeyruwan </div> <div class="row-card__subheading__spacer"></div> <div class="row-card__subheading__item extra-small-text"> <a class="row-card__small-link" href="/people/bewley/"> Alex Bewley </a> </div> <div class="row-card__subheading__spacer"></div> <div class="row-card__subheading__item extra-small-text"> Nick Boffi </div> <div class="row-card__subheading__spacer"></div> <div class="row-card__subheading__item extra-small-text"> <a class="row-card__small-link" href="/people/krzysztofchoromanski/"> Krzysztof Choromanski </a> </div> <div class="row-card__subheading__spacer"></div> <div class="row-card__subheading__item extra-small-text"> <a class="row-card__small-link" href="/people/davidbdambrosio/"> David D'Ambrosio </a> </div> <div class="row-card__subheading__spacer"></div> <div class="row-card__subheading__item extra-small-text"> <a class="row-card__small-link" href="/people/deepali/"> Deepali Jain </a> </div> <div class="row-card__subheading__spacer"></div> <div class="row-card__subheading__item extra-small-text"> <a class="row-card__small-link" href="/people/pannagrsanketi/"> Pannag Sanketi </a> </div> <div class="row-card__subheading__spacer"></div> <div class="row-card__subheading__item extra-small-text"> Anish Shankar </div> <div class="row-card__subheading__spacer"></div> <div class="row-card__subheading__item extra-small-text"> <a class="row-card__small-link" href="/people/vikassindhwani/"> Vikas Sindhwani </a> </div> <div class="row-card__subheading__spacer"></div> <div class="row-card__subheading__item extra-small-text"> <a class="row-card__small-link" href="/people/sumeetsingh/"> Sumeet Singh </a> </div> <div class="row-card__subheading__spacer"></div> <div class="row-card__subheading__item extra-small-text"> Jean-Jacques Slotine </div> <div class="row-card__subheading__spacer"></div> <div class="row-card__subheading__item extra-small-text"> Stephen Tu </div> <div class="row-card__subheading__spacer"></div> <div class="row-card__subheading__item extra-small-text"> Learning for Dynamics and Control (2023) </div> </div> </div> <div class="row-card__cta headline-6"> <div class="glue-tooltip" data-glue-tooltip-auto-position="false"> <button class="glue-button glue-button--low-emphasis glue-tooltip__trigger" aria-describedby=tooltip-contentwe-address-a-benchmark-task-in-agi tabindex=0 > <span class="js-gt-item-id">Preview</span> </button> <span id="tooltip-contentwe-address-a-benchmark-task-in-agi" class="glue-tooltip__content" role="tooltip"> <span data-tooltip-type="simple"> Preview abstract </span> <span data-tooltip-type="rich"> <span class="glue-tooltip__body">We address a benchmark task in agile robotics: catching objects thrown at high-speed. This is a challenging task that involves tracking, intercepting, and cradling a thrown object with access only to visual observations of the object and the proprioceptive state of the robot, all within a fraction of a second. We present the relative merits of two fundamentally different solution strategies: (i) Model Predictive Control using accelerated constrained trajectory optimization, and (ii) Reinforcement Learning using zeroth-order optimization. We provide insights into various performance tradeoffs including sample efficiency, sim-to-real transfer, robustness to distribution shifts, and wholebody multimodality via extensive on-hardware experiments. We conclude with proposals on fusing “classical” and “learning-based” techniques for agile robot control. Videos of our experiments may be found here: https://sites.google.com/view/agile-catching.</span> <a class="glue-button glue-button--low-emphasis" href="http://research.google/pubs/agile-catching-with-whole-body-mpc-and-blackbox-policy-learning/" > <span class="js-gt-item-id">View details</span> </a> </span> </span> </div> </div> </div> </div> <div class="row-card"> <div class="row-card__container"> <div class="row-card__body"> <a class="row-card__heading headline-6 glue-link" href=http://research.google/pubs/scalable-multi-sensor-robot-imitation-learning-via-task-level-domain-consistency/ > Scalable Multi-Sensor Robot Imitation Learning via Task-Level Domain Consistency </a> <div class="row-card__subheading"> <div class="row-card__subheading__item extra-small-text"> Armando Fuentes </div> <div class="row-card__subheading__spacer"></div> <div class="row-card__subheading__item extra-small-text"> Daniel Ho </div> <div class="row-card__subheading__spacer"></div> <div class="row-card__subheading__item extra-small-text"> Eric Victor Jang </div> <div class="row-card__subheading__spacer"></div> <div class="row-card__subheading__item extra-small-text"> Matt Bennice </div> <div class="row-card__subheading__spacer"></div> <div class="row-card__subheading__item extra-small-text"> Mohi Khansari </div> <div class="row-card__subheading__spacer"></div> <div class="row-card__subheading__item extra-small-text"> Nicolas Sievers </div> <div class="row-card__subheading__spacer"></div> <div class="row-card__subheading__item extra-small-text"> <a class="row-card__small-link" href="/people/seankirmani/"> Sean Kirmani </a> </div> <div class="row-card__subheading__spacer"></div> <div class="row-card__subheading__item extra-small-text"> <a class="row-card__small-link" href="/people/105569/"> Yunfei Bai </a> </div> <div class="row-card__subheading__spacer"></div> <div class="row-card__subheading__item extra-small-text"> Yuqing Du </div> <div class="row-card__subheading__spacer"></div> <div class="row-card__subheading__item extra-small-text"> ICRA (2023) (to appear) </div> </div> </div> <div class="row-card__cta headline-6"> <div class="glue-tooltip" data-glue-tooltip-auto-position="false"> <button class="glue-button glue-button--low-emphasis glue-tooltip__trigger" aria-describedby=tooltip-contentrecent-work-in-visual-end-to-end-l tabindex=0 > <span class="js-gt-item-id">Preview</span> </button> <span id="tooltip-contentrecent-work-in-visual-end-to-end-l" class="glue-tooltip__content" role="tooltip"> <span data-tooltip-type="simple"> Preview abstract </span> <span data-tooltip-type="rich"> <span class="glue-tooltip__body">Recent work in visual end-to-end learning for robotics has shown the promise of imitation learning across a variety of tasks. However, such approaches are often expensive and require vast amounts of real world training demonstrations. Additionally, they rely on a time-consuming evaluation process for identifying the best model to deploy in the real world. These challenges can be mitigated by simulation - by supplementing real world data with simulated demonstrations and using simulated evaluations to identify strong policies. However, this introduces the well-known ``reality gap'' problem, where simulator inaccuracies decorrelates performance in simulation from reality. In this paper, we build on top of prior work in GAN-based domain adaptation and introduce the notion of a Task Consistency Loss (TCL), a self-supervised contrastive loss that encourages sim and real alignment both at the feature and action-prediction level. We demonstrate the effectiveness of our approach on the challenging task of latched-door opening with a 9 Degree-of-Freedom (DoF) mobile manipulator from raw RGB and depth images. While most prior work in vision-based manipulation operate from a fixed, third person view, mobile manipulation couples the challenges of locomotion and manipulation with greater visual diversity and action space complexity. We find that we are able to achieve 77% success on seen and unseen scenes, a +30% increase from the baseline, using only ~16 hours of teleoperation demonstrations in sim and real.</span> <a class="glue-button glue-button--low-emphasis" href="http://research.google/pubs/scalable-multi-sensor-robot-imitation-learning-via-task-level-domain-consistency/" > <span class="js-gt-item-id">View details</span> </a> </span> </span> </div> </div> </div> </div> </div> </div> </div> </section> <section class="offset-two-up --theme-dark both" data-gt-id="offset_two_up" data-gt-component-name=""> <div class="glue-page glue-grid"> <div class="offset-two-up__left-col glue-grid__col glue-grid__col--span-4-sm glue-grid__col--span-12-md glue-grid__col--span-3-lg"> <h2 class="offset-two-up__headline headline-3">Some of our teams</h2> </div> <div class="glue-grid__col glue-grid__col--span-4-sm glue-grid__col--span-12-md glue-grid__col--span-9-lg"> <ul class="card-stack--small nested-glue-grid-override" data-gt-id="team_small_card_stack" data-gt-component-name="Team - Small Card Stack"> <li class="glue-grid__col glue-grid__col--span-6-md glue-grid__col--span-4-sm"> <a class="glue-card not-glue glue-card--small" href="/teams/perception/"> <div class="glue-card__inner"> <div class="glue-card__content"> <div class="content-wrapper"> <p class="glue-headline body js-gt-item-id"> Perception </p> </div> <div class="glue-card__cta glue-card__cta--arrow"> <span class="glue-button glue-button--low-emphasis glue-button--icon"> <svg role="presentation" aria-hidden="true" class="glue-icon "> <use href="/gr/static/assets/icons/glue-icons.svg#arrow-forward"></use> </svg> </span> </div> </div> </div> </a> </li> </ul> </div> </div> </section> <section class="banner --theme-light" data-gt-id="banner" data-gt-component-name="None"> <div class="banner__wrapper glue-page glue-grid"> <div class="banner__copy glue-grid__col glue-grid__col--span-4-sm glue-grid__col--span-6-md"> <h2 class="banner__headline headline-3">Join us</h2> <p class="banner__body-copy body">We're always looking for more talented, passionate people.</p> <a class="glue-button glue-button--medium-emphasis" href="http://research.google/careers/" > <span class="js-gt-item-id">See opportunities</span> </a> </div> <div class="glue-grid__col glue-grid__col--span-0-sm glue-grid__col--span-1"></div> <div class="banner__image glue-grid__col glue-grid__col--span-4-sm glue-grid__col--span-5-md"> <img src="https://storage.googleapis.com/gweb-research2023-media/images/Careers.original.jpg" alt="Careers" /> </div> </div> </section> </main> <footer class="glue-footer"> <div class="glue-page"> <section class="glue-social"> <div class="glue-social__group glue-social--monochrome"> <p class="glue-social__title glue-social__title--inline"> Follow us </p> <nav class="js-gt-follow-us-wrapper" aria-label="Social media links"> <ul class="glue-social__list" role="list"> <li class="glue-social__item"> <a class="glue-social__link" href="https://twitter.com/GoogleAI" title="Follow us on x" target="_blank" rel="noopener" data-gt-method="x"" > <svg role="presentation" aria-hidden="true" class="glue-icon glue-icon--social glue-icon--24px"> <use href="/gr/static/assets/icons/twitter-x.svg#twitter-x"></use> </svg> </a> </li> <li class="glue-social__item"> <a class="glue-social__link" href="https://www.linkedin.com/showcase/googleresearch/" title="Follow us on linkedin" target="_blank" rel="noopener" data-gt-method="linkedin"" > <svg role="presentation" aria-hidden="true" class="glue-icon glue-icon--social glue-icon--24px"> <use href="/gr/static/assets/icons/glue-icons.svg#post-linkedin"></use> </svg> </a> </li> <li class="glue-social__item"> <a class="glue-social__link" href="https://www.youtube.com/c/GoogleResearch" title="Follow us on youtube" target="_blank" rel="noopener" data-gt-method="youtube"" > <svg role="presentation" aria-hidden="true" class="glue-icon glue-icon--social glue-icon--24px"> <use href="/gr/static/assets/icons/glue-icons.svg#video-youtube"></use> </svg> </a> </li> <li class="glue-social__item"> <a class="glue-social__link" href="https://github.com/google-research" title="Follow us on github" target="_blank" rel="noopener" data-gt-method="github"" > <svg role="presentation" aria-hidden="true" class="glue-icon glue-icon--social glue-icon--24px"> <use href="/gr/static/assets/icons/github.svg#github"></use> </svg> </a> </li> </ul> </nav> </div> </section> </div> <div class="glue-fullbleed"></div> <section class="glue-page"> <nav class="glue-footer__global" aria-label="Footer resource links"> <div class="glue-footer__logo"> <a href="https://www.google.com" title="Google" class="glue-footer__link"> <svg role="presentation" aria-hidden="true" class="glue-icon glue-footer__logo-img"> <use href="/gr/static/assets/icons/glue-icons.svg#google-solid-logo"></use> </svg> </a> </div> <ul class="glue-footer__global-links glue-no-bullet js-gt-global-nav-wrapper" role="list"> <li class="glue-footer__global-links-list-item" data-gt-primary="About Google"> <a class="glue-footer__link" href="https://about.google/" target="_blank" rel="noopener"> About Google </a> </li> <li class="glue-footer__global-links-list-item" data-gt-primary="Google Products"> <a class="glue-footer__link" href="https://about.google/intl/en/products/" target="_blank" rel="noopener"> Google Products </a> </li> <li class="glue-footer__global-links-list-item" data-gt-primary="Privacy"> <a class="glue-footer__link" href="https://policies.google.com/privacy" target="_blank" rel="noopener"> Privacy </a> </li> <li class="glue-footer__global-links-list-item" data-gt-primary="Terms"> <a class="glue-footer__link" href="https://policies.google.com/terms" target="_blank" rel="noopener"> Terms </a> </li> </ul> <ul class="glue-footer__global-links glue-footer__global-links--extra glue-no-bullet" role="list"> <li class="glue-footer__global-links-list-item glue-footer__global-links-list-item--extra"> <a class="glue-footer__link" href="https://support.google.com/?hl=en"> <svg role="presentation" aria-hidden="true" aria-hidden="true" class="glue-icon glue-icon--24px glue-icon--footer-help"> <use href="/gr/static/assets/icons/glue-icons.svg#help"></use> </svg> Help </a> </li> <li class="glue-footer__global-links-list-item glue-footer__global-links-list-item--extra"> <button class="glue-footer__link google-feedback js-feedback-button" href="" data-product-id="5137383" > Submit feedback </button> </li> </ul> </nav> </section> </footer> <script src="https://www.gstatic.com/glue/v27_1/material-components-web.min.js"></script> <script src="https://www.youtube.com/player_api"></script> <script type="text/javascript" src="/gr/static/js/googleresearch.js?id=b70549917812130af912601ad763f13e"></script> <script type="text/javascript" src="https://support.google.com/inapp/api.js"></script> <script src="https://www.gstatic.com/glue/cookienotificationbar/cookienotificationbar.min.js" data-glue-cookie-notification-bar-category="2B"> </script> </body> </html>

CINXE.COM

Robotics