Machine Translation

<!DOCTYPE html> <html lang="en"> <head> <meta charset="utf-8" /> <link rel="canonical" href="http://research.google/research-areas/machine-translation/" /><meta property="og:title" content="Machine Translation"><meta property="og:url" content="http://research.google/research-areas/machine-translation/"><meta property="og:image" content="https://storage.googleapis.com/gweb-research2023-media/images/Open_Graph.width-800.format-jpeg.jpg"><meta property="og:image:secure_url" content="https://storage.googleapis.com/gweb-research2023-media/images/Open_Graph.width-800.format-jpeg.jpg"><meta property="og:type" content="Website"> <title>Machine Translation</title> <meta name="viewport" content="width=device-width, initial-scale=1 viewport-fit=cover"/> <link rel="icon" type="image/png" href="/gr/static/assets/favicon.ico"> <link rel="preconnect" href="https://fonts.googleapis.com"> <link rel="preconnect" href="https://fonts.gstatic.com" crossorigin> <link rel="preload" href="https://fonts.googleapis.com/css2?family=Product+Sans&family=Google+Sans+Display:ital@0;1&family=Google+Sans:ital,wght@0,400;0,500;0,700;1,400;1,500;1,700&family=Google+Sans+Text:ital,wght@0,400;0,500;0,700;1,400;1,500;1,700&display=swap" as="style"> <link rel="stylesheet" href="https://fonts.googleapis.com/css2?family=Product+Sans&family=Google+Sans+Display:ital@0;1&family=Google+Sans:ital,wght@0,400;0,500;0,700;1,400;1,500;1,700&family=Google+Sans+Text:ital,wght@0,400;0,500;0,700;1,400;1,500;1,700&display=swap"> <link href="https://fonts.googleapis.com/css2?family=Roboto+Mono:wght@400;700&display=swap" rel="stylesheet"> <link href="https://www.gstatic.com/glue/cookienotificationbar/cookienotificationbar.min.css" rel="stylesheet" /> <link href="https://www.gstatic.com/glue/v27_1/glue-material.min.css" rel="stylesheet"> <link rel="stylesheet" type="text/css" href="/gr/static/css/googleresearch.css?id=0c26ea1fed8bdd0324f9f4fad1f6a470">  <script>(function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start': new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0], j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src= 'https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f); })(window,document,'script','dataLayer','GTM-K8QBZ7Q'); </script>  </head> <body class=" js-google-tag-wrapper" data-gt-page-path="http://research.google/research-areas/machine-translation/" data-env="production">  <noscript><iframe src="https://www.googletagmanager.com/ns.html?id=GTM-K8QBZ7Q" height="0" width="0" style="display:none;visibility:hidden"></iframe></noscript>  <header class="global-header glue-header glue-header--single not-glue"> <a href="#page-content" class="glue-header__skip-content">Jump to Content</a> <div class="glue-header__bar glue-header__bar--mobile not-glue"> <div class="glue-header__tier not-glue">  <div class="glue-header__container"> <div class="glue-header__lock-up">  <div class="glue-header__hamburger"> <button class="glue-header__drawer-toggle-btn" aria-label="Open the navigation drawer"> <svg class="glue-icon glue-icon--24px" role="presentation" aria-hidden="true"> <use href="/gr/static/assets/icons/glue-icons.svg#menu"></use> </svg> </button> </div> <div class="glue-header__logo"> <a class="glue-header__logo-link" href="/" title="Google Research">  <div class="glue-header__logo-container"> <svg role="presentation" aria-hidden="true" alt='Google' class="glue-icon glue-icon glue-header__logo-svg"> <use href="/gr/static/assets/icons/glue-icons.svg#google-color-logo"></use> </svg> </div> <span class="glue-header__logo--product">Research</span> </a> </div> </div> </div> </div> </div> <div class="glue-header__bar glue-header__bar--desktop glue-header__drawer"> <div class="glue-header__tier">  <div class="glue-header__container"> <div class="glue-header__lock-up"> <div class="glue-header__logo"> <a class="glue-header__logo-link" href="/" title="Google Research">  <div class="glue-header__logo-container"> <svg role="presentation" aria-hidden="true" alt='Google' class="glue-icon glue-icon glue-header__logo-svg not-glue --dark-logo"> <use href="/gr/static/assets/icons/glue-icons.svg#google-solid-logo"></use> </svg> <svg role="presentation" aria-hidden="true" alt='Google' class="glue-icon glue-icon glue-header__logo-svg --light-logo"> <use href="/gr/static/assets/icons/glue-icons.svg#google-color-logo"></use> </svg> </div> <span class="glue-header__logo--product">Research</span> </a> </div> </div> </div>  <div class="glue-header__container glue-header__container--linkbar"> <nav class="glue-header__link-bar navigation js-gt-global-nav-wrapper"> <ul class="glue-header__list"> <li class="glue-header__item js-sub-nav-parent --parent" data-gt-primary="Who we are" > <button class="glue-header__link js-sub-nav-target" aria-haspopup="true" aria-expanded="false" > <span class=""> Who we are <span class="icon icon--caret"></span> </span> </button> <div class="navigation__sub js-sub-nav" role="menu"> <div class="navigation__sub__container"> <div class="navigation__sub__mobile-heading"> <button class="glue-header__link js-sub-nav-close-mobile"> <span class="sr-text">Back to</span> <span class="icon icon--caret"></span> Who we are <span class="sr-text">menu</span> </button> <hr/> </div> <div class="block-nav_drawer_columns_content"> <div class="navigation__sub--content" data-gt-secondary="Defining the technology of today and tomorrow."> <div class="navigation__sub__wrapper"> <div class="navigation__sub__heading"> <h2 class="headline-3">Defining the technology of today and tomorrow.</h2> </div> <ul class="navigation__sub__columns"> <li data-gt-secondary="Philosophy"> <div class="navigation__sub__columns__desktop"> <h2 class="headline-6 navigation__sub__columns__heading"> Philosophy </h2> <p class="navigation__sub__columns__description caption">We strive to create an environment conducive to many different types of research across many different time scales and levels of risk.</p> <a href="http://research.google/philosophy/" class="glue-inline-link js-drawer-link" > <span class="sr-text">Learn more about our Philosophy</span> <span aria-hidden="true">Learn more</span> </a> </div> <div class="navigation__sub__columns__mobile"> <a class="glue-header__link" href="http://research.google/philosophy/" > Philosophy </a> </div> </li> <li data-gt-secondary="People"> <div class="navigation__sub__columns__desktop"> <h2 class="headline-6 navigation__sub__columns__heading"> People </h2> <p class="navigation__sub__columns__description caption">Our researchers drive advancements in computer science through both fundamental and applied research.</p> <a href="http://research.google/people/" class="glue-inline-link js-drawer-link" > <span class="sr-text">Learn more about our People</span> <span aria-hidden="true">Learn more</span> </a> </div> <div class="navigation__sub__columns__mobile"> <a class="glue-header__link" href="http://research.google/people/" > People </a> </div> </li> </ul> </div> </div> </div> </div> </div> </li> <li class="glue-header__item js-sub-nav-parent --parent" data-gt-primary="Research areas" > <button class="glue-header__link js-sub-nav-target" aria-haspopup="true" aria-expanded="false" > <span class=""> Research areas <span class="icon icon--caret"></span> </span> </button> <div class="navigation__sub js-sub-nav" role="menu"> <div class="navigation__sub__container"> <div class="navigation__sub__mobile-heading"> <button class="glue-header__link js-sub-nav-close-mobile"> <span class="sr-text">Back to</span> <span class="icon icon--caret"></span> Research areas <span class="sr-text">menu</span> </button> <hr/> </div> <div class="block-nav_drawer_columns_link_list"> <div class="navigation__sub--list"> <div class="navigation__sub__wrapper"> <ul class="navigation__sub__columns"> <li data-gt-secondary="Research areas"> <div class="navigation__sub__columns__desktop"> <h2 class="headline-6 navigation__sub__columns__heading">Research areas</h2> <ul> <li> <a class="navigation__sub__columns__list-link caption js-drawer-link" href="http://research.google/research-areas/" > Explore all research areas </a> </li> </ul> </div> <div class="navigation__sub__columns__mobile"> <button class="glue-header__link js-sub-nav-target" data-panel="nested" role="menuitem" aria-haspopup="true"> Research areas <span class="icon icon--caret"></span> </button> <div class="navigation__nested-sub js-sub-nav-parent"> <div class="navigation__sub__mobile-heading"> <button class="glue-header__link js-sub-nav-close-mobile" role="menuitem" aria-haspopup="true"> <span class="sr-text">Back to</span> <span class="icon icon--caret"></span> Research areas <span class="sr-text">menu</span> </button> <hr/> </div> <ul> <li role="menuitem"> <a href="http://research.google/research-areas/" class="navigation__sub__columns__mobile__link" > Explore all research areas <span> </span> </a> </li> </ul> </div> </div> </li> <li data-gt-secondary="Foundational ML & Algorithms"> <div class="navigation__sub__columns__desktop"> <h2 class="headline-6 navigation__sub__columns__heading">Foundational ML & Algorithms</h2> <ul> <li> <a class="navigation__sub__columns__list-link caption js-drawer-link" href="http://research.google/research-areas/algorithms-and-theory/" > Algorithms & Theory </a> </li> <li> <a class="navigation__sub__columns__list-link caption js-drawer-link" href="http://research.google/research-areas/data-management/" > Data Management </a> </li> <li> <a class="navigation__sub__columns__list-link caption js-drawer-link" href="http://research.google/research-areas/data-mining-and-modeling/" > Data Mining & Modeling </a> </li> <li> <a class="navigation__sub__columns__list-link caption js-drawer-link" href="http://research.google/research-areas/information-retrieval-and-the-web/" > Information Retrieval & the Web </a> </li> <li> <a class="navigation__sub__columns__list-link caption js-drawer-link" href="http://research.google/research-areas/machine-intelligence/" > Machine Intelligence </a> </li> <li> <a class="navigation__sub__columns__list-link caption js-drawer-link" href="http://research.google/research-areas/machine-perception/" > Machine Perception </a> </li> <li> <a class="navigation__sub__columns__list-link caption js-drawer-link" href="http://research.google/research-areas/machine-translation/" > Machine Translation </a> </li> <li> <a class="navigation__sub__columns__list-link caption js-drawer-link" href="http://research.google/research-areas/natural-language-processing/" > Natural Language Processing </a> </li> <li> <a class="navigation__sub__columns__list-link caption js-drawer-link" href="http://research.google/research-areas/speech-processing/" > Speech Processing </a> </li> </ul> </div> <div class="navigation__sub__columns__mobile"> <button class="glue-header__link js-sub-nav-target" data-panel="nested" role="menuitem" aria-haspopup="true"> Foundational ML & Algorithms <span class="icon icon--caret"></span> </button> <div class="navigation__nested-sub js-sub-nav-parent"> <div class="navigation__sub__mobile-heading"> <button class="glue-header__link js-sub-nav-close-mobile" role="menuitem" aria-haspopup="true"> <span class="sr-text">Back to</span> <span class="icon icon--caret"></span> Foundational ML & Algorithms <span class="sr-text">menu</span> </button> <hr/> </div> <ul> <li role="menuitem"> <a href="http://research.google/research-areas/algorithms-and-theory/" class="navigation__sub__columns__mobile__link" > Algorithms & Theory <span> </span> </a> </li> <li role="menuitem"> <a href="http://research.google/research-areas/data-management/" class="navigation__sub__columns__mobile__link" > Data Management <span> </span> </a> </li> <li role="menuitem"> <a href="http://research.google/research-areas/data-mining-and-modeling/" class="navigation__sub__columns__mobile__link" > Data Mining & Modeling <span> </span> </a> </li> <li role="menuitem"> <a href="http://research.google/research-areas/information-retrieval-and-the-web/" class="navigation__sub__columns__mobile__link" > Information Retrieval & the Web <span> </span> </a> </li> <li role="menuitem"> <a href="http://research.google/research-areas/machine-intelligence/" class="navigation__sub__columns__mobile__link" > Machine Intelligence <span> </span> </a> </li> <li role="menuitem"> <a href="http://research.google/research-areas/machine-perception/" class="navigation__sub__columns__mobile__link" > Machine Perception <span> </span> </a> </li> <li role="menuitem"> <a href="http://research.google/research-areas/machine-translation/" class="navigation__sub__columns__mobile__link" > Machine Translation <span> </span> </a> </li> <li role="menuitem"> <a href="http://research.google/research-areas/natural-language-processing/" class="navigation__sub__columns__mobile__link" > Natural Language Processing <span> </span> </a> </li> <li role="menuitem"> <a href="http://research.google/research-areas/speech-processing/" class="navigation__sub__columns__mobile__link" > Speech Processing <span> </span> </a> </li> </ul> </div> </div> </li> <li data-gt-secondary="Computing Systems & Quantum AI"> <div class="navigation__sub__columns__desktop"> <h2 class="headline-6 navigation__sub__columns__heading">Computing Systems & Quantum AI</h2> <ul> <li> <a class="navigation__sub__columns__list-link caption js-drawer-link" href="http://research.google/research-areas/distributed-systems-and-parallel-computing/" > Distributed Systems & Parallel鈥–omputing </a> </li> <li> <a class="navigation__sub__columns__list-link caption js-drawer-link" href="http://research.google/research-areas/hardware-and-architecture/" > Hardware & Architecture </a> </li> <li> <a class="navigation__sub__columns__list-link caption js-drawer-link" href="http://research.google/research-areas/mobile-systems/" > Mobile Systems </a> </li> <li> <a class="navigation__sub__columns__list-link caption js-drawer-link" href="http://research.google/research-areas/networking/" > Networking </a> </li> <li> <a class="navigation__sub__columns__list-link caption js-drawer-link" href="http://research.google/research-areas/quantum-computing/" > Quantum Computing </a> </li> <li> <a class="navigation__sub__columns__list-link caption js-drawer-link" href="http://research.google/research-areas/robotics/" > Robotics </a> </li> <li> <a class="navigation__sub__columns__list-link caption js-drawer-link" href="http://research.google/research-areas/security-privacy-and-abuse-prevention/" > Security, Privacy, & Abuse鈥≒revention </a> </li> <li> <a class="navigation__sub__columns__list-link caption js-drawer-link" href="http://research.google/research-areas/software-engineering/" > Software Engineering </a> </li> <li> <a class="navigation__sub__columns__list-link caption js-drawer-link" href="http://research.google/research-areas/software-systems/" > Software Systems </a> </li> </ul> </div> <div class="navigation__sub__columns__mobile"> <button class="glue-header__link js-sub-nav-target" data-panel="nested" role="menuitem" aria-haspopup="true"> Computing Systems & Quantum AI <span class="icon icon--caret"></span> </button> <div class="navigation__nested-sub js-sub-nav-parent"> <div class="navigation__sub__mobile-heading"> <button class="glue-header__link js-sub-nav-close-mobile" role="menuitem" aria-haspopup="true"> <span class="sr-text">Back to</span> <span class="icon icon--caret"></span> Computing Systems & Quantum AI <span class="sr-text">menu</span> </button> <hr/> </div> <ul> <li role="menuitem"> <a href="http://research.google/research-areas/distributed-systems-and-parallel-computing/" class="navigation__sub__columns__mobile__link" > Distributed Systems & Parallel鈥–omputing <span> </span> </a> </li> <li role="menuitem"> <a href="http://research.google/research-areas/hardware-and-architecture/" class="navigation__sub__columns__mobile__link" > Hardware & Architecture <span> </span> </a> </li> <li role="menuitem"> <a href="http://research.google/research-areas/mobile-systems/" class="navigation__sub__columns__mobile__link" > Mobile Systems <span> </span> </a> </li> <li role="menuitem"> <a href="http://research.google/research-areas/networking/" class="navigation__sub__columns__mobile__link" > Networking <span> </span> </a> </li> <li role="menuitem"> <a href="http://research.google/research-areas/quantum-computing/" class="navigation__sub__columns__mobile__link" > Quantum Computing <span> </span> </a> </li> <li role="menuitem"> <a href="http://research.google/research-areas/robotics/" class="navigation__sub__columns__mobile__link" > Robotics <span> </span> </a> </li> <li role="menuitem"> <a href="http://research.google/research-areas/security-privacy-and-abuse-prevention/" class="navigation__sub__columns__mobile__link" > Security, Privacy, & Abuse鈥≒revention <span> </span> </a> </li> <li role="menuitem"> <a href="http://research.google/research-areas/software-engineering/" class="navigation__sub__columns__mobile__link" > Software Engineering <span> </span> </a> </li> <li role="menuitem"> <a href="http://research.google/research-areas/software-systems/" class="navigation__sub__columns__mobile__link" > Software Systems <span> </span> </a> </li> </ul> </div> </div> </li> <li data-gt-secondary="Science, AI & Society"> <div class="navigation__sub__columns__desktop"> <h2 class="headline-6 navigation__sub__columns__heading">Science, AI & Society</h2> <ul> <li> <a class="navigation__sub__columns__list-link caption js-drawer-link" href="http://research.google/research-areas/climate-and-sustainability/" > Climate & Sustainability </a> </li> <li> <a class="navigation__sub__columns__list-link caption js-drawer-link" href="http://research.google/research-areas/economics-and-electronic-commerce/" > Economics & Electronic Commerce </a> </li> <li> <a class="navigation__sub__columns__list-link caption js-drawer-link" href="http://research.google/research-areas/education-innovation/" > Education Innovation </a> </li> <li> <a class="navigation__sub__columns__list-link caption js-drawer-link" href="http://research.google/research-areas/general-science/" > General Science </a> </li> <li> <a class="navigation__sub__columns__list-link caption js-drawer-link" href="http://research.google/research-areas/health-bioscience/" > Health & Bioscience </a> </li> <li> <a class="navigation__sub__columns__list-link caption js-drawer-link" href="http://research.google/research-areas/human-computer-interaction-and-visualization/" > Human-Computer Interaction and Visualization </a> </li> </ul> </div> <div class="navigation__sub__columns__mobile"> <button class="glue-header__link js-sub-nav-target" data-panel="nested" role="menuitem" aria-haspopup="true"> Science, AI & Society <span class="icon icon--caret"></span> </button> <div class="navigation__nested-sub js-sub-nav-parent"> <div class="navigation__sub__mobile-heading"> <button class="glue-header__link js-sub-nav-close-mobile" role="menuitem" aria-haspopup="true"> <span class="sr-text">Back to</span> <span class="icon icon--caret"></span> Science, AI & Society <span class="sr-text">menu</span> </button> <hr/> </div> <ul> <li role="menuitem"> <a href="http://research.google/research-areas/climate-and-sustainability/" class="navigation__sub__columns__mobile__link" > Climate & Sustainability <span> </span> </a> </li> <li role="menuitem"> <a href="http://research.google/research-areas/economics-and-electronic-commerce/" class="navigation__sub__columns__mobile__link" > Economics & Electronic Commerce <span> </span> </a> </li> <li role="menuitem"> <a href="http://research.google/research-areas/education-innovation/" class="navigation__sub__columns__mobile__link" > Education Innovation <span> </span> </a> </li> <li role="menuitem"> <a href="http://research.google/research-areas/general-science/" class="navigation__sub__columns__mobile__link" > General Science <span> </span> </a> </li> <li role="menuitem"> <a href="http://research.google/research-areas/health-bioscience/" class="navigation__sub__columns__mobile__link" > Health & Bioscience <span> </span> </a> </li> <li role="menuitem"> <a href="http://research.google/research-areas/human-computer-interaction-and-visualization/" class="navigation__sub__columns__mobile__link" > Human-Computer Interaction and Visualization <span> </span> </a> </li> </ul> </div> </div> </li> </ul> </div> </div></div> </div> </div> </li> <li class="glue-header__item js-sub-nav-parent --parent" data-gt-primary="Our work" > <button class="glue-header__link js-sub-nav-target" aria-haspopup="true" aria-expanded="false" > <span class=""> Our work <span class="icon icon--caret"></span> </span> </button> <div class="navigation__sub js-sub-nav" role="menu"> <div class="navigation__sub__container"> <div class="navigation__sub__mobile-heading"> <button class="glue-header__link js-sub-nav-close-mobile"> <span class="sr-text">Back to</span> <span class="icon icon--caret"></span> Our work <span class="sr-text">menu</span> </button> <hr/> </div> <div class="block-nav_drawer_columns_content"> <div class="navigation__sub--content" data-gt-secondary=""> <div class="navigation__sub__wrapper"> <ul class="navigation__sub__columns"> <li data-gt-secondary="Projects"> <div class="navigation__sub__columns__desktop"> <h2 class="headline-6 navigation__sub__columns__heading"> Projects </h2> <p class="navigation__sub__columns__description caption">We regularly open-source projects with the broader research community and apply our developments to Google products.</p> <a href="http://research.google/resources/our-projects/" class="glue-inline-link js-drawer-link" > <span class="sr-text">Learn more about our Projects</span> <span aria-hidden="true">Learn more</span> </a> </div> <div class="navigation__sub__columns__mobile"> <a class="glue-header__link" href="http://research.google/resources/our-projects/" > Projects </a> </div> </li> <li data-gt-secondary="Publications"> <div class="navigation__sub__columns__desktop"> <h2 class="headline-6 navigation__sub__columns__heading"> Publications </h2> <p class="navigation__sub__columns__description caption">Publishing our work allows us to share ideas and work collaboratively to advance the field of computer science.</p> <a href="http://research.google/pubs/" class="glue-inline-link js-drawer-link" > <span class="sr-text">Learn more about our Publications</span> <span aria-hidden="true">Learn more</span> </a> </div> <div class="navigation__sub__columns__mobile"> <a class="glue-header__link" href="http://research.google/pubs/" > Publications </a> </div> </li> <li data-gt-secondary="Resources"> <div class="navigation__sub__columns__desktop"> <h2 class="headline-6 navigation__sub__columns__heading"> Resources </h2> <p class="navigation__sub__columns__description caption">We make products, tools, and datasets available to everyone with the goal of building a more collaborative ecosystem.</p> <a href="http://research.google/resources/" class="glue-inline-link js-drawer-link" > <span class="sr-text">Learn more about our Resources</span> <span aria-hidden="true">Learn more</span> </a> </div> <div class="navigation__sub__columns__mobile"> <a class="glue-header__link" href="http://research.google/resources/" > Resources </a> </div> </li> </ul> </div> </div> </div> </div> </div> </li> <li class="glue-header__item js-sub-nav-parent --parent" data-gt-primary="Programs & events" > <button class="glue-header__link js-sub-nav-target" aria-haspopup="true" aria-expanded="false" > <span class=""> Programs & events <span class="icon icon--caret"></span> </span> </button> <div class="navigation__sub js-sub-nav" role="menu"> <div class="navigation__sub__container"> <div class="navigation__sub__mobile-heading"> <button class="glue-header__link js-sub-nav-close-mobile"> <span class="sr-text">Back to</span> <span class="icon icon--caret"></span> Programs & events <span class="sr-text">menu</span> </button> <hr/> </div> <div class="block-nav_drawer_columns_content"> <div class="navigation__sub--content" data-gt-secondary="Shaping the future, together."> <div class="navigation__sub__wrapper"> <div class="navigation__sub__heading"> <h2 class="headline-3">Shaping the future, together.</h2> <a href="http://research.google/programs-and-events/" class="js-drawer-link" > Collaborate with us </a> </div> <ul class="navigation__sub__columns"> <li data-gt-secondary="Student programs"> <div class="navigation__sub__columns__desktop"> <h2 class="headline-6 navigation__sub__columns__heading"> Student programs </h2> <p class="navigation__sub__columns__description caption">Supporting the next generation of researchers through a wide range of programming.</p> <a href="http://research.google/programs-and-events/student-engagement/" class="glue-inline-link js-drawer-link" > <span class="sr-text">Learn more about our Student programs</span> <span aria-hidden="true">Learn more</span> </a> </div> <div class="navigation__sub__columns__mobile"> <a class="glue-header__link" href="http://research.google/programs-and-events/student-engagement/" > Student programs </a> </div> </li> <li data-gt-secondary="Faculty programs"> <div class="navigation__sub__columns__desktop"> <h2 class="headline-6 navigation__sub__columns__heading"> Faculty programs </h2> <p class="navigation__sub__columns__description caption">Participating in the academic research community through meaningful engagement with university faculty.</p> <a href="http://research.google/programs-and-events/faculty-engagement/" class="glue-inline-link js-drawer-link" > <span class="sr-text">Learn more about our Faculty programs</span> <span aria-hidden="true">Learn more</span> </a> </div> <div class="navigation__sub__columns__mobile"> <a class="glue-header__link" href="http://research.google/programs-and-events/faculty-engagement/" > Faculty programs </a> </div> </li> <li data-gt-secondary="Conferences & events"> <div class="navigation__sub__columns__desktop"> <h2 class="headline-6 navigation__sub__columns__heading"> Conferences & events </h2> <p class="navigation__sub__columns__description caption">Connecting with the broader research community through events is essential for creating progress in every aspect of our work.</p> <a href="http://research.google/conferences-and-events/" class="glue-inline-link js-drawer-link" > <span class="sr-text">Learn more about our Conferences & events</span> <span aria-hidden="true">Learn more</span> </a> </div> <div class="navigation__sub__columns__mobile"> <a class="glue-header__link" href="http://research.google/conferences-and-events/" > Conferences & events </a> </div> </li> </ul> <div class="navigation__sub__cta"> <a class="glue-button glue-button--high-emphasis js-drawer-link" href="http://research.google/programs-and-events/" target="_blank" rel="noreferrer noopener" > Collaborate with us </a> </div> </div> </div> </div> </div> </div> </li> <li class="glue-header__item " data-gt-primary="Careers" > <a class="glue-header__link " href="http://research.google/careers/" > <span class=""> Careers </span> </a> </li> <li class="glue-header__item " data-gt-primary="Blog" > <a class="glue-header__link " href="http://research.google/blog/" > <span class=""> Blog </span> </a> </li> </ul> </nav> </div>  <div class="glue-header__search js-header-search"> <div class="glue-header__search__input"> <div class="search-input " data-type="header"> <input type="search" class="caption --empty-search js-search-bar js-gt-search-input" placeholder="Search"> <button class="search-input__button --search js-gt-search-btn"> <svg role="presentation" aria-hidden="true" class="glue-icon glue-icon--18px "> <use href="/gr/static/assets/icons/glue-icons.svg#search"></use> </svg> </button> <button class="search-input__button --clear"> <svg role="presentation" aria-hidden="true" class="glue-icon glue-icon--18px "> <use href="/gr/static/assets/icons/glue-icons.svg#close"></use> </svg> </button> </div> </div> <button class="glue-header__search__btn js-header-search-btn"> <svg role="presentation" aria-hidden="true" aria-hidden="true" class="glue-icon glue-icon--24px search"> <use href="/gr/static/assets/icons/glue-icons.svg#search"></use> </svg> <svg role="presentation" aria-hidden="true" aria-hidden="true" class="glue-icon glue-icon--24px close"> <use href="/gr/static/assets/icons/glue-icons.svg#close"></use> </svg> <span class="sr-text js-header-search-sr-text">Search</span> </button> </div> </div> </div> <div class="glue-header__drawer-backdrop"> <div class="glue-header__mobile_close"> <button class="glue-header__drawer-toggle-btn js-mobile-nav-close" aria-label="Close the navigation drawer"> <svg class="glue-icon glue-icon--24px" role="presentation" aria-hidden="true"> <use href="/gr/static/assets/icons/glue-icons.svg#close"></use> </svg> </button> </div> </div> </header> <main id="page-content"> <div class="research-area-detail"> <section class="basic-hero bhoig --theme-dark " data-gt-id="basic_hero" data-gt-component-name=""> <div class="glue-page"> <div class="glue-grid"> <div class="bhoig__image-wrapper glue-grid__col--span-2 glue-grid__col--span-3-md glue-grid__col--span-2-lg"> <div class="bhoig__image-bg" style="background-color: var(--glue-grey-900); "> <span class="icon icon--machine-translation" style="" ></span> </div> </div> <div class="bhoig__breadcrumb-wrapper glue-grid__col--span-10 glue-grid__col--span-9-md glue-grid__col--span-10-lg"> <nav class="glue-breadcrumbs" aria-label="Breadcrumbs"> <ol class="glue-breadcrumbs__list"> <li class="glue-breadcrumbs__item"> <a class="glue-breadcrumbs__link attribution" href="/">Home</a> <svg role="presentation" aria-hidden="true" class="glue-icon "> <use href="/gr/static/assets/icons/glue-icons.svg#chevron-right"></use> </svg> </li> <li class="glue-breadcrumbs__item"> <a class="glue-breadcrumbs__link attribution" href="/research-areas/">Research areas</a> <svg role="presentation" aria-hidden="true" class="glue-icon "> <use href="/gr/static/assets/icons/glue-icons.svg#chevron-right"></use> </svg> </li> </ol> </nav> </div> <h1 class="headline-1 bhoig__headline glue-grid__col--span-10 glue-grid__col--span-9-md glue-grid__col--span-10-lg">Machine Translation</h1> <div class="basic-hero__description bhoig__description glue-grid__col--span-10 glue-grid__col--span-9-md glue-grid__col--span-10-lg"> <p data-block-key="5yfrg">Machine Translation is an excellent example of how cutting-edge research and world-class infrastructure come together at Google. We focus our research efforts on developing statistical translation techniques that improve with more data and generalize well to new languages. Our large scale computing infrastructure allows us to rapidly experiment with new models trained on web-scale data to significantly improve translation quality. This research backs the translations served at translate.google.com, allowing our users to translate text, web pages and even speech. Deployed within a wide range of Google services like <a href="https://support.google.com/mail/answer/139503?hl=en" target="_blank" rel="noopener noreferrer">GMail</a>, <a href="https://booksearch.blogspot.com/2011/05/define-translate-and-search-for-words.html" target="_blank" rel="noopener noreferrer">Books</a>, <a href="https://googleresearch.blogspot.com/2015/07/how-google-translate-squeezes-deep.html">Android</a> and <a href="https://support.google.com/toolbar/answer/146786?hl=en" target="_blank" rel="noopener noreferrer">web search</a>, Google Translate is a high-impact, research-driven product that bridges language barriers and makes it possible to explore the multilingual web in 90 languages. Exciting research challenges abound as we pursue human quality translation and develop machine translation systems for new languages.</p> </div> <div class="bhoig__cta glue-grid__col--span-10 glue-grid__col--span-9-md glue-grid__col--span-10-lg"> </div> </div> </div> </section> <section class="offset-two-up"> <div class="glue-page glue-grid"> <div class="offset-two-up__left-col glue-grid__col glue-grid__col--span-4-sm glue-grid__col--span-12-md glue-grid__col--span-3-lg"> <h3 class="offset-two-up__headline headline-3">Recent Publications</h3> <div class="offset-two-up__cta-below-description"> <a class="glue-button glue-button--medium-emphasis" href="http://research.google/pubs/?category=machine-translation" > <span class="js-gt-item-id">See More</span> </a> </div> </div> <div class="glue-grid__col glue-grid__col--span-4-sm glue-grid__col--span-12-md glue-grid__col--span-9-lg"> <div class="publications-list --theme- row-card-list" data-hot-swap="pub-list" data-gt-id="publications_list" data-gt-component-name=""> <div class="row-card"> <div class="row-card__container"> <div class="row-card__body"> <a class="row-card__heading headline-6 glue-link" href=http://research.google/pubs/connecting-language-technologies-with-rich-diverse-data-sources-covering-thousands-of-languages/ > Connecting Language Technologies with Rich, Diverse Data Sources Covering Thousands of Languages </a> <div class="row-card__subheading"> <div class="row-card__subheading__item extra-small-text"> <a class="row-card__small-link" href="/people/daanvanesch/"> Daan van Esch </a> </div> <div class="row-card__subheading__spacer"></div> <div class="row-card__subheading__item extra-small-text"> <a class="row-card__small-link" href="/people/106369/"> Sandy Ritchie </a> </div> <div class="row-card__subheading__spacer"></div> <div class="row-card__subheading__item extra-small-text"> Sebastian Ruder </div> <div class="row-card__subheading__spacer"></div> <div class="row-card__subheading__item extra-small-text"> Julia Kreutzer </div> <div class="row-card__subheading__spacer"></div> <div class="row-card__subheading__item extra-small-text"> Clara Rivera </div> <div class="row-card__subheading__spacer"></div> <div class="row-card__subheading__item extra-small-text"> Ishank Saxena </div> <div class="row-card__subheading__spacer"></div> <div class="row-card__subheading__item extra-small-text"> <a class="row-card__small-link" href="/people/106896/"> Isaac Caswell </a> </div> <div class="row-card__subheading__spacer"></div> <div class="row-card__subheading__item extra-small-text"> Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024) </div> </div> </div> <div class="row-card__cta headline-6"> <div class="glue-tooltip" data-glue-tooltip-auto-position="false"> <button class="glue-button glue-button--low-emphasis glue-tooltip__trigger" aria-describedby=tooltip-contentcontrary-to-common-belief-there-a tabindex=0 > <span class="js-gt-item-id">Preview</span> </button> <span id="tooltip-contentcontrary-to-common-belief-there-a" class="glue-tooltip__content" role="tooltip"> <span data-tooltip-type="simple"> Preview abstract </span> <span data-tooltip-type="rich"> <span class="glue-tooltip__body">Contrary to common belief, there are rich and diverse data sources available for many thousands of languages, which can be used to develop technologies for these languages. In this paper, we provide an overview of some of the major online data sources, the types of data that they provide access to, potential applications of this data, and the number of languages that they cover. Even this covers only a small fraction of the data that exists; for example, printed books are published in many languages but few online aggregators exist.</span> <a class="glue-button glue-button--low-emphasis" href="http://research.google/pubs/connecting-language-technologies-with-rich-diverse-data-sources-covering-thousands-of-languages/" > <span class="js-gt-item-id">View details</span> </a> </span> </span> </div> </div> </div> </div> <div class="row-card"> <div class="row-card__container"> <div class="row-card__body"> <a class="row-card__heading headline-6 glue-link" href=http://research.google/pubs/ties-matter-meta-evaluating-modern-metrics-with-pairwise-accuracy-and-tie-calibration/ > Ties Matter: Meta-Evaluating Modern Metrics with Pairwise Accuracy and Tie Calibration </a> <div class="row-card__subheading"> <div class="row-card__subheading__item extra-small-text"> <a class="row-card__small-link" href="/people/danieldeutsch/"> Dan Deutsch </a> </div> <div class="row-card__subheading__spacer"></div> <div class="row-card__subheading__item extra-small-text"> George Foster </div> <div class="row-card__subheading__spacer"></div> <div class="row-card__subheading__item extra-small-text"> <a class="row-card__small-link" href="/people/markusfreitag/"> Markus Freitag </a> </div> <div class="row-card__subheading__spacer"></div> <div class="row-card__subheading__item extra-small-text"> Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, Association for Computational Linguistics, Singapore, pp. 12914-12929 </div> </div> </div> <div class="row-card__cta headline-6"> <div class="glue-tooltip" data-glue-tooltip-auto-position="false"> <button class="glue-button glue-button--low-emphasis glue-tooltip__trigger" aria-describedby=tooltip-contentkendalls-tau-is-frequently-used-t tabindex=0 > <span class="js-gt-item-id">Preview</span> </button> <span id="tooltip-contentkendalls-tau-is-frequently-used-t" class="glue-tooltip__content" role="tooltip"> <span data-tooltip-type="simple"> Preview abstract </span> <span data-tooltip-type="rich"> <span class="glue-tooltip__body">Kendall's tau is frequently used to meta-evaluate how well machine translation (MT) evaluation metrics score individual translations. Its focus on pairwise score comparisons is intuitive but raises the question of how ties should be handled, a gray area that has motivated different variants in the literature. We demonstrate that, in settings like modern MT meta-evaluation, existing variants have weaknesses arising from their handling of ties, and in some situations can even be gamed. We propose instead to meta-evaluate metrics with a version of pairwise accuracy that gives metrics credit for correctly predicting ties, in combination with a tie calibration procedure that automatically introduces ties into metric scores, enabling fair comparison between metrics that do and do not predict ties. We argue and provide experimental evidence that these modifications lead to fairer ranking-based assessments of metric performance.</span> <a class="glue-button glue-button--low-emphasis" href="http://research.google/pubs/ties-matter-meta-evaluating-modern-metrics-with-pairwise-accuracy-and-tie-calibration/" > <span class="js-gt-item-id">View details</span> </a> </span> </span> </div> </div> </div> </div> <div class="row-card"> <div class="row-card__container"> <div class="row-card__body"> <a class="row-card__heading headline-6 glue-link" href=http://research.google/pubs/bilex-rx-lexical-data-augmentation-for-massively-multilingual-machine-translation/ > BiLex Rx: Lexical Data Augmentation for Massively Multilingual Machine Translation </a> <div class="row-card__subheading"> <div class="row-card__subheading__item extra-small-text"> Alex Jones </div> <div class="row-card__subheading__spacer"></div> <div class="row-card__subheading__item extra-small-text"> <a class="row-card__small-link" href="/people/106896/"> Isaac Caswell </a> </div> <div class="row-card__subheading__spacer"></div> <div class="row-card__subheading__item extra-small-text"> <a class="row-card__small-link" href="/people/105696/"> Orhan Firat </a> </div> <div class="row-card__subheading__spacer"></div> <div class="row-card__subheading__item extra-small-text"> ArXiv (2023) </div> </div> </div> <div class="row-card__cta headline-6"> <div class="glue-tooltip" data-glue-tooltip-auto-position="false"> <button class="glue-button glue-button--low-emphasis glue-tooltip__trigger" aria-describedby=tooltip-contentneural-machine-translation-nmt-h tabindex=0 > <span class="js-gt-item-id">Preview</span> </button> <span id="tooltip-contentneural-machine-translation-nmt-h" class="glue-tooltip__content" role="tooltip"> <span data-tooltip-type="simple"> Preview abstract </span> <span data-tooltip-type="rich"> <span class="glue-tooltip__body">Neural machine translation (NMT) has progressed rapidly over the past several years, and modern models are able to achieve relatively high quality using only monolingual text data, an approach dubbed Unsupervised Machine Translation, or UNMT. However, these models still struggle in a variety of ways, including aspects of translation that for a human are the easiest---for instance, correctly translating common nouns. This work explores a cheap and abundant resource to combat this problem: bilingual lexicons (\textsc{BiLex}s). We test the efficacy of bilingual lexicons in a real-world set-up, on 200-language translation models trained on web-mined text. We present several findings: (1) we demonstrate the most effective ways to use this resource for MT by extensively experimenting with lexical data augmentation techniques, such as codeswitching and lexical prompting; (2) we pinpoint what settings and languages are benefited most from lexical data augmentation; and (3) we provide an empirical, per-language analysis of the quality of the public resource PanLex, a multilingual lexicon covering thousands of languages.</span> <a class="glue-button glue-button--low-emphasis" href="http://research.google/pubs/bilex-rx-lexical-data-augmentation-for-massively-multilingual-machine-translation/" > <span class="js-gt-item-id">View details</span> </a> </span> </span> </div> </div> </div> </div> <div class="row-card"> <div class="row-card__container"> <div class="row-card__body"> <a class="row-card__heading headline-6 glue-link" href=http://research.google/pubs/mu2slam-multitask-multilingual-speech-and-language-models/ > Mu2SLAM: Multitask, Multilingual Speech and Language Models </a> <div class="row-card__subheading"> <div class="row-card__subheading__item extra-small-text"> <a class="row-card__small-link" href="/people/106360/"> Yong Cheng </a> </div> <div class="row-card__subheading__spacer"></div> <div class="row-card__subheading__item extra-small-text"> Yu Zhang </div> <div class="row-card__subheading__spacer"></div> <div class="row-card__subheading__item extra-small-text"> <a class="row-card__small-link" href="/people/105147/"> Melvin Johnson </a> </div> <div class="row-card__subheading__spacer"></div> <div class="row-card__subheading__item extra-small-text"> <a class="row-card__small-link" href="/people/author7590/"> Wolfgang Macherey </a> </div> <div class="row-card__subheading__spacer"></div> <div class="row-card__subheading__item extra-small-text"> <a class="row-card__small-link" href="/people/105376/"> Ankur Bapna </a> </div> <div class="row-card__subheading__spacer"></div> <div class="row-card__subheading__item extra-small-text"> Submission to ACL 2023 </div> </div> </div> <div class="row-card__cta headline-6"> <div class="glue-tooltip" data-glue-tooltip-auto-position="false"> <button class="glue-button glue-button--low-emphasis glue-tooltip__trigger" aria-describedby=tooltip-contentwe-present-mu2slam-a-multilingual tabindex=0 > <span class="js-gt-item-id">Preview</span> </button> <span id="tooltip-contentwe-present-mu2slam-a-multilingual" class="glue-tooltip__content" role="tooltip"> <span data-tooltip-type="simple"> Preview abstract </span> <span data-tooltip-type="rich"> <span class="glue-tooltip__body">We present Mu2SLAM, a multilingual sequence-to-sequence model pre-trained jointly on un-labeled speech, unlabeled text and supervised data spanning Automatic Speech Recognition(ASR), Automatic Speech Translation (AST)and Machine Translation (MT), in over 100 languages. By leveraging a quantized representation of speech as a target, Mu2SLAM trains ona sequence-to-sequence masked denoising objective similar to T5 on both unlabeled speech and text, while utilizing the supervised tasks to improve cross-lingual and cross-modal representation alignment within the model. On CoVoSTAST, Mu2SLAM establishes a new state-of-the-art for models trained on public datasets, improv-ing on xx-en translation over the previous best by 1.9 Bleu points and on en-xx translation by 0.9 Bleu points. On Voxpopuli ASR, our model matches the performance of a mSLAM model finetuned with a RNN-T decoder, despite using a relatively weaker sequence-to-sequence architecture. On text understanding tasks, our model improves by more than 6% over mSLAM on XNLI, getting closer to the performance of mT5 models of comparable capacity on XNLI and TydiQA, paving the way towards a single model for all speech and text understanding tasks.</span> <a class="glue-button glue-button--low-emphasis" href="http://research.google/pubs/mu2slam-multitask-multilingual-speech-and-language-models/" > <span class="js-gt-item-id">View details</span> </a> </span> </span> </div> </div> </div> </div> <div class="row-card"> <div class="row-card__container"> <div class="row-card__body"> <a class="row-card__heading headline-6 glue-link" href=http://research.google/pubs/epsilon-sampling-rocks-investigating-sampling-strategies-for-minimum-bayes-risk-decoding-for-machine-translation/ > Epsilon Sampling Rocks: Investigating Sampling Strategies for Minimum Bayes Risk Decoding for Machine Translation </a> <div class="row-card__subheading"> <div class="row-card__subheading__item extra-small-text"> <a class="row-card__small-link" href="/people/markusfreitag/"> Markus Freitag </a> </div> <div class="row-card__subheading__spacer"></div> <div class="row-card__subheading__item extra-small-text"> Behrooz Ghorbani </div> <div class="row-card__subheading__spacer"></div> <div class="row-card__subheading__item extra-small-text"> Patrick Fernandes </div> <div class="row-card__subheading__spacer"></div> <div class="row-card__subheading__item extra-small-text"> Findings of the Association for Computational Linguistics: EMNLP 2023, Association for Computational Linguistics, Singapore, pp. 9198-9209 </div> </div> </div> <div class="row-card__cta headline-6"> <div class="glue-tooltip" data-glue-tooltip-auto-position="false"> <button class="glue-button glue-button--low-emphasis glue-tooltip__trigger" aria-describedby=tooltip-contentrecent-advances-in-machine-transla tabindex=0 > <span class="js-gt-item-id">Preview</span> </button> <span id="tooltip-contentrecent-advances-in-machine-transla" class="glue-tooltip__content" role="tooltip"> <span data-tooltip-type="simple"> Preview abstract </span> <span data-tooltip-type="rich"> <span class="glue-tooltip__body">Recent advances in machine translation (MT) have shown that Minimum Bayes Risk (MBR) decoding can be a powerful alternative to beam search decoding, especially when combined with neural-based utility functions. However, the performance of MBR decoding depends heavily on how and how many candidates are sampled from the model. In this paper, we explore how different sampling approaches for generating candidate lists for MBR decoding affect performance. We evaluate popular sampling approaches, such as ancestral, nucleus, and top-k sampling. Based on our insights into their limitations, we experiment with the recently proposed epsilon-sampling approach, which prunes away all tokens with a probability smaller than epsilon, ensuring that each token in a sample receives a fair probability mass. Through extensive human evaluations, we demonstrate that MBR decoding based on epsilon-sampling significantly outperforms not only beam search decoding, but also MBR decoding with all other tested sampling methods across four language pairs.</span> <a class="glue-button glue-button--low-emphasis" href="http://research.google/pubs/epsilon-sampling-rocks-investigating-sampling-strategies-for-minimum-bayes-risk-decoding-for-machine-translation/" > <span class="js-gt-item-id">View details</span> </a> </span> </span> </div> </div> </div> </div> <div class="row-card"> <div class="row-card__container"> <div class="row-card__body"> <a class="row-card__heading headline-6 glue-link" href=http://research.google/pubs/results-of-wmt23-metrics-shared-task-metrics-might-be-guilty-but-references-are-not-innocent/ > Results of WMT23 Metrics Shared Task: Metrics might be Guilty but References are not Innocent </a> <div class="row-card__subheading"> <div class="row-card__subheading__item extra-small-text"> <a class="row-card__small-link" href="/people/markusfreitag/"> Markus Freitag </a> </div> <div class="row-card__subheading__spacer"></div> <div class="row-card__subheading__item extra-small-text"> Nitika Mathur </div> <div class="row-card__subheading__spacer"></div> <div class="row-card__subheading__item extra-small-text"> Chi-kiu Lo </div> <div class="row-card__subheading__spacer"></div> <div class="row-card__subheading__item extra-small-text"> Eleftherios Avramidis </div> <div class="row-card__subheading__spacer"></div> <div class="row-card__subheading__item extra-small-text"> Ricardo Rei </div> <div class="row-card__subheading__spacer"></div> <div class="row-card__subheading__item extra-small-text"> Brian Thompson </div> <div class="row-card__subheading__spacer"></div> <div class="row-card__subheading__item extra-small-text"> Tom Kocmi </div> <div class="row-card__subheading__spacer"></div> <div class="row-card__subheading__item extra-small-text"> Fr茅d茅ric Blain </div> <div class="row-card__subheading__spacer"></div> <div class="row-card__subheading__item extra-small-text"> <a class="row-card__small-link" href="/people/danieldeutsch/"> Dan Deutsch </a> </div> <div class="row-card__subheading__spacer"></div> <div class="row-card__subheading__item extra-small-text"> Craig Stewart </div> <div class="row-card__subheading__spacer"></div> <div class="row-card__subheading__item extra-small-text"> Chrysoula Zerva </div> <div class="row-card__subheading__spacer"></div> <div class="row-card__subheading__item extra-small-text"> Sheila Castilho </div> <div class="row-card__subheading__spacer"></div> <div class="row-card__subheading__item extra-small-text"> Alon Lavie </div> <div class="row-card__subheading__spacer"></div> <div class="row-card__subheading__item extra-small-text"> George Foster </div> <div class="row-card__subheading__spacer"></div> <div class="row-card__subheading__item extra-small-text"> Proceedings of the Eighth Conference on Machine Translation, Association for Computational Linguistics, Singapore (2023), pp. 576-626 </div> </div> </div> <div class="row-card__cta headline-6"> <div class="glue-tooltip" data-glue-tooltip-auto-position="false"> <button class="glue-button glue-button--low-emphasis glue-tooltip__trigger" aria-describedby=tooltip-contentthis-paper-presents-the-results-of tabindex=0 > <span class="js-gt-item-id">Preview</span> </button> <span id="tooltip-contentthis-paper-presents-the-results-of" class="glue-tooltip__content" role="tooltip"> <span data-tooltip-type="simple"> Preview abstract </span> <span data-tooltip-type="rich"> <span class="glue-tooltip__body">This paper presents the results of the WMT23 Metrics Shared Task. Participants submitting automatic MT evaluation metrics were asked to score the outputs of the translation systems competing in the WMT23 News Translation Task. All metrics were evaluated on how well they correlate with human ratings at the system and segment level. Similar to last year, we acquired our own human ratings based on expert-based human evaluation via Multidimensional Quality Metrics (MQM). Following last year's success, we also included a challenge set subtask, where participants had to create contrastive test suites for evaluating metrics' ability to capture and penalise specific types of translation errors. Furthermore, we improved our meta-evaluation procedure by considering fewer tasks and calculating a global score by weighted averaging across the various tasks. We present an extensive analysis on how well metrics perform on three language pairs: Chinese-English, Hebrew-English on the sentence-level and English-German on the paragraph-level. The results strongly confirm the results reported last year, that neural-based metrics are significantly better than non-neural metrics in their levels of correlation with human judgments. Further, we investigate the impact of bad reference translations on the correlations of metrics with human judgment. We present a novel approach for generating synthetic reference translations based on the collection of MT system outputs and their corresponding MQM ratings, which has the potential to mitigate bad reference issues we observed this year for some language pairs. Finally, we also study the connections between the magnitude of metric differences and their expected significance in human evaluation, which should help the community to better understand and adopt new metrics.</span> <a class="glue-button glue-button--low-emphasis" href="http://research.google/pubs/results-of-wmt23-metrics-shared-task-metrics-might-be-guilty-but-references-are-not-innocent/" > <span class="js-gt-item-id">View details</span> </a> </span> </span> </div> </div> </div> </div> </div> </div> </div> </section> <section class="banner --theme-dark" data-gt-id="banner" data-gt-component-name="None"> <div class="banner__wrapper glue-page glue-grid"> <div class="banner__copy glue-grid__col glue-grid__col--span-4-sm glue-grid__col--span-6-md"> <h2 class="banner__headline headline-3">Join us</h2> <p class="banner__body-copy body">We're always looking for more talented, passionate people.</p> <a class="glue-button glue-button--medium-emphasis glue-button--reversed" href="http://research.google/careers/" > <span class="js-gt-item-id">See opportunities</span> </a> </div> <div class="glue-grid__col glue-grid__col--span-0-sm glue-grid__col--span-1"></div> <div class="banner__image glue-grid__col glue-grid__col--span-4-sm glue-grid__col--span-5-md"> <img src="https://storage.googleapis.com/gweb-research2023-media/images/Careers.original.jpg" alt="Careers" /> </div> </div> </section> </main> <footer class="glue-footer"> <div class="glue-page"> <section class="glue-social"> <div class="glue-social__group glue-social--monochrome"> <p class="glue-social__title glue-social__title--inline"> Follow us </p> <nav class="js-gt-follow-us-wrapper" aria-label="Social media links"> <ul class="glue-social__list" role="list"> <li class="glue-social__item"> <a class="glue-social__link" href="https://twitter.com/GoogleAI" title="Follow us on x" target="_blank" rel="noopener" data-gt-method="x"" > <svg role="presentation" aria-hidden="true" class="glue-icon glue-icon--social glue-icon--24px"> <use href="/gr/static/assets/icons/twitter-x.svg#twitter-x"></use> </svg> </a> </li> <li class="glue-social__item"> <a class="glue-social__link" href="https://www.linkedin.com/showcase/googleresearch/" title="Follow us on linkedin" target="_blank" rel="noopener" data-gt-method="linkedin"" > <svg role="presentation" aria-hidden="true" class="glue-icon glue-icon--social glue-icon--24px"> <use href="/gr/static/assets/icons/glue-icons.svg#post-linkedin"></use> </svg> </a> </li> <li class="glue-social__item"> <a class="glue-social__link" href="https://www.youtube.com/c/GoogleResearch" title="Follow us on youtube" target="_blank" rel="noopener" data-gt-method="youtube"" > <svg role="presentation" aria-hidden="true" class="glue-icon glue-icon--social glue-icon--24px"> <use href="/gr/static/assets/icons/glue-icons.svg#video-youtube"></use> </svg> </a> </li> <li class="glue-social__item"> <a class="glue-social__link" href="https://github.com/google-research" title="Follow us on github" target="_blank" rel="noopener" data-gt-method="github"" > <svg role="presentation" aria-hidden="true" class="glue-icon glue-icon--social glue-icon--24px"> <use href="/gr/static/assets/icons/github.svg#github"></use> </svg> </a> </li> </ul> </nav> </div> </section> </div> <div class="glue-fullbleed"></div> <section class="glue-page"> <nav class="glue-footer__global" aria-label="Footer resource links"> <div class="glue-footer__logo"> <a href="https://www.google.com" title="Google" class="glue-footer__link"> <svg role="presentation" aria-hidden="true" class="glue-icon glue-footer__logo-img"> <use href="/gr/static/assets/icons/glue-icons.svg#google-solid-logo"></use> </svg> </a> </div> <ul class="glue-footer__global-links glue-no-bullet js-gt-global-nav-wrapper" role="list"> <li class="glue-footer__global-links-list-item" data-gt-primary="About Google"> <a class="glue-footer__link" href="https://about.google/" target="_blank" rel="noopener"> About Google </a> </li> <li class="glue-footer__global-links-list-item" data-gt-primary="Google Products"> <a class="glue-footer__link" href="https://about.google/intl/en/products/" target="_blank" rel="noopener"> Google Products </a> </li> <li class="glue-footer__global-links-list-item" data-gt-primary="Privacy"> <a class="glue-footer__link" href="https://policies.google.com/privacy" target="_blank" rel="noopener"> Privacy </a> </li> <li class="glue-footer__global-links-list-item" data-gt-primary="Terms"> <a class="glue-footer__link" href="https://policies.google.com/terms" target="_blank" rel="noopener"> Terms </a> </li> </ul> <ul class="glue-footer__global-links glue-footer__global-links--extra glue-no-bullet" role="list"> <li class="glue-footer__global-links-list-item glue-footer__global-links-list-item--extra"> <a class="glue-footer__link" href="https://support.google.com/?hl=en"> <svg role="presentation" aria-hidden="true" aria-hidden="true" class="glue-icon glue-icon--24px glue-icon--footer-help"> <use href="/gr/static/assets/icons/glue-icons.svg#help"></use> </svg> Help </a> </li> <li class="glue-footer__global-links-list-item glue-footer__global-links-list-item--extra"> <button class="glue-footer__link google-feedback js-feedback-button" href="" data-product-id="5137383" > Submit feedback </button> </li> </ul> </nav> </section> </footer> <script src="https://www.gstatic.com/glue/v27_1/material-components-web.min.js"></script> <script src="https://www.youtube.com/player_api"></script> <script type="text/javascript" src="/gr/static/js/googleresearch.js?id=b70549917812130af912601ad763f13e"></script> <script type="text/javascript" src="https://support.google.com/inapp/api.js"></script> <script src="https://www.gstatic.com/glue/cookienotificationbar/cookienotificationbar.min.js" data-glue-cookie-notification-bar-category="2B"> </script> </body> </html>

CINXE.COM

Machine Translation