CINXE.COM
Federated Learning
<!doctype html> <html lang="en"> <head> <meta charset="utf-8"> <title>Federated Learning</title> <meta name="description" content="Building better products with on-device data and privacy by default. An online comic from Google AI."> <meta name="viewport" content="width=device-width, initial-scale=1"> <meta name="twitter:card" content="summary_large_image"> <meta name="twitter:site" content="@GoogleAI"> <meta name="twitter:title" content="Federated Learning"> <meta name="twitter:description" content="Building better products with on-device data and privacy by default. An online comic from Google AI."> <meta name="twitter:image" content="https://federated.withgoogle.com/assets/meta/twitter.png?1557190726349"> <meta property="og:type" content="article"> <meta property="og:url" content="https://federated.withgoogle.com"> <meta property="og:site_name" content="Federated Learning"> <meta property="og:title" content="Federated Learning"> <meta property="og:description" content="Building better products with on-device data and privacy by default. An online comic from Google AI."> <meta property="og:image" content="https://federated.withgoogle.com/assets/meta/facebook.png?1557190726349"> <meta property="og:image:width" content="1200"> <meta property="og:image:height" content="630"> <link rel="icon" type="image/png" href="/assets/meta/favicon.png"> <link rel="stylesheet" href="/assets/css/main.css"> <link rel="stylesheet" href="//fonts.googleapis.com/css?family=Google+Sans:400,500|Roboto:400,500|Material+Icons"> </head> <body> <header> <div class="limit-width"> <div class="title"> <div class="material-icons" id="nav-expand"></div> <h1><a href="#top">Federated Learning</a></h1> </div> <ul id="nav"> <li><a href="#about">About</a></li> <li><a href="#learn">Learn more</a></li> <li><a href="#research">Research</a></li> </ul> </div> </header> <a class="anchor" id="top"></a> <main> <div class="social" id="social-top"> <span class="share">Share</span> <a target="_blank" class="logo" href="https://twitter.com/home?status=https%3A//federated.withgoogle.com"> <svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 28 28" class="share-links-icon twitter" aria-label="Share on Twitter"><path d="M25.76,6.32A9.63,9.63,0,0,1,23,7.1,5,5,0,0,0,25.1,4.36,9.67,9.67,0,0,1,22,5.56,4.83,4.83,0,0,0,18.45,4a4.91,4.91,0,0,0-4.88,4.95,5,5,0,0,0,.13,1.13,13.78,13.78,0,0,1-10-5.17,5,5,0,0,0,1.51,6.61A4.8,4.8,0,0,1,3,10.89V11a4.93,4.93,0,0,0,3.91,4.85,4.82,4.82,0,0,1-2.2.09,4.89,4.89,0,0,0,4.55,3.44,9.69,9.69,0,0,1-7.22,2A13.65,13.65,0,0,0,9.47,23.6c9,0,13.87-7.54,13.87-14.08q0-.32,0-.64a10,10,0,0,0,2.43-2.56"></path></svg> </a> <a target="_blank" class="logo" href="https://www.facebook.com/sharer/sharer.php?u=https%3A//federated.withgoogle.com"> <svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 28 28" class="share-links-icon facebook" aria-label="Share on Facebook"><path d="M24.68,2H3.32A1.32,1.32,0,0,0,2,3.32V24.68A1.32,1.32,0,0,0,3.32,26H14.82V16.71H11.69V13.08h3.13V10.41c0-3.1,1.89-4.79,4.66-4.79a25.65,25.65,0,0,1,2.79.14V9H20.35c-1.5,0-1.8.71-1.8,1.76v2.31h3.59l-.47,3.62H18.56V26h6.12A1.32,1.32,0,0,0,26,24.68V3.32A1.32,1.32,0,0,0,24.68,2Z"></path></svg> </a> <a target="_blank" class="logo" href="https://www.linkedin.com/shareArticle?mini=true&url=https%3A//federated.withgoogle.com&title=Federated Learning"> <svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 28 28" class="share-links-icon linkedin" aria-label="Share on LinkedIn"><path d="M22.57,22.45H19V16.88c0-1.33,0-3-1.88-3s-2.17,1.45-2.17,2.94v5.66H11.31V11h3.46v1.56h.05a3.8,3.8,0,0,1,3.42-1.85c3.65,0,4.33,2.37,4.33,5.46ZM7.24,9.43A2.06,2.06,0,1,1,9.34,7.37,2.08,2.08,0,0,1,7.24,9.43ZM5.44,11H9.05V22.45H5.44V11ZM24.37,2H3.62a1.76,1.76,0,0,0-1.8,1.73V24.27A1.77,1.77,0,0,0,3.62,26H24.37a1.77,1.77,0,0,0,1.8-1.73V3.73A1.77,1.77,0,0,0,24.37,2Z"></path></svg> </a> </div> <section class="limit-width-3up"> <img src="/assets/images/header.png" srcset="/assets/images/header.png 761w, /assets/images/header_1x.png 1523w, /assets/images/header_2x.png 3045w" id="cover" class="cover-image" aria-labelledby="cover-caption" tabindex="0"> <img src="/assets/images/header-2up.png" srcset="/assets/images/header-2up.png 1523w, /assets/images/header-2up_1x.png 3045w, /assets/images/header-2up_2x.png 6090w" id="cover-2up" class="cover-image" aria-labelledby="cover-caption" tabindex="0"> <img src="/assets/images/header-3up.png" srcset="/assets/images/header-3up.png 2284w, /assets/images/header-3up_1x.png 4568w, /assets/images/header-3up_2x.png 9135w" id="cover-3up" class="cover-image" aria-labelledby="cover-caption" tabindex="0"> <p id="cover-caption">Building better products with on-device data and privacy by default. An online comic from Google AI.</p> <div id="comic"> <figure class="progressive" data-panel="1"> <img class="preview" src="assets/comic/panel001.png" data-src="assets/comic/panel001_1x.png" data-srcset="assets/comic/panel001.png 750w, assets/comic/panel001_1x.png 1500w, assets/comic/panel001_2x.png 3045w" aria-labelledby="caption1" tabindex="0"/> <figcaption class="caption"> <p id="caption1"> Martha, a caucasian woman in her mid-thirties, bursts into a run-down office. Her Boss, a balding caucasian man in his fifties, sits behind his desk in despair. There’s a dead cactus by his elbow, an anxious-looking photo of him on the wall, and exposed wires hanging from the ceiling. Martha shouts “Boss! I’m back from the conference! And I know how we can win back our users!” “About time!” The Boss says. “Our brand is in shambles.” </p> </figcaption> </figure> <figure class="progressive" data-panel="2"> <img class="preview" src="assets/comic/panel002.png" data-src="assets/comic/panel002_1x.png" data-srcset="assets/comic/panel002.png 750w, assets/comic/panel002_1x.png 1500w, assets/comic/panel002_2x.png 3045w" aria-labelledby="caption2" tabindex="0"/> <figcaption class="caption"> <p id="caption2"> Martha leans eagerly across the desk. “Don’t worry,” she says. “I learned all about a new approach that can handle our privacy concerns and improve functionality. It’s called federated learning…” Her Boss interrupts. “Federated what now?” </p> </figcaption> </figure> <figure class="progressive" data-panel="3"> <img class="preview" src="assets/comic/panel003.png" data-src="assets/comic/panel003_1x.png" data-srcset="assets/comic/panel003.png 750w, assets/comic/panel003_1x.png 1500w, assets/comic/panel003_2x.png 3045w" aria-labelledby="caption3" tabindex="0"/> <figcaption class="caption"> <p id="caption3"> Martha waves her hands in excitement while she talks. “It lets us do machine learning while keeping data on-device. It’s resilient, low-impact, secure–” The Boss leaps out of his chair and hurries off-panel. “Whatever, I’m sold! I’ll give you a team of our very best–” </p> </figcaption> </figure> <figure class="progressive" data-panel="4"> <img class="preview" src="assets/comic/panel004.png" data-src="assets/comic/panel004_1x.png" data-srcset="assets/comic/panel004.png 750w, assets/comic/panel004_1x.png 1500w, assets/comic/panel004_2x.png 3045w" aria-labelledby="caption4" tabindex="0"/> <figcaption class="caption"> <p id="caption4"> The Boss gestures grandly to a rag-tag group of twenty-somethings while shouting “Interns!” The five interns all look up. Brad, a burly caucasian jock, waves hello overenthusiastically. Kai, a nonbinary Japanese-American hacker, plays with a Rubix cube. Devi, a bubbly Indian-American networker, snaps a selfie. Mateo, a scrawny Hispanic bookworm, pauses in the middle of eating a sandwich. Aliyah, a sharply-dressed African-American security enthusiast, looks unimpressed. </p> </figcaption> </figure> <figure class="progressive" data-panel="5"> <img class="preview" src="assets/comic/panel005.png" data-src="assets/comic/panel005_1x.png" data-srcset="assets/comic/panel005.png 750w, assets/comic/panel005_1x.png 1500w, assets/comic/panel005_2x.png 3045w" aria-labelledby="caption5" tabindex="0"/> <figcaption class="caption"> <p id="caption5"> Martha glares at the interns. </p> </figcaption> </figure> <figure class="progressive" data-panel="6"> <img class="preview" src="assets/comic/panel006.png" data-src="assets/comic/panel006_1x.png" data-srcset="assets/comic/panel006.png 750w, assets/comic/panel006_1x.png 1500w, assets/comic/panel006_2x.png 3045w" aria-labelledby="caption6" tabindex="0"/> <figcaption class="caption"> <p id="caption6"> Martha narrows her eyes and says “Challenge accepted.” </p> </figcaption> </figure> <figure class="progressive" data-panel="7"> <img class="preview" src="assets/comic/panel007.png" data-src="assets/comic/panel007_1x.png" data-srcset="assets/comic/panel007.png 750w, assets/comic/panel007_1x.png 1500w, assets/comic/panel007_2x.png 3045w" aria-labelledby="caption7" tabindex="0"/> <figcaption class="caption"> <p id="caption7"> The interns sit around a conference table while Martha presents in front of a white-board. “Okay, everybody!” Martha says. “We’re going to start with a hypothetical problem. Let’s say we want to train a machine learning model on sensitive user data.” </p> </figcaption> </figure> <figure class="progressive" data-panel="8"> <img class="preview" src="assets/comic/panel008.png" data-src="assets/comic/panel008_1x.png" data-srcset="assets/comic/panel008.png 750w, assets/comic/panel008_1x.png 1500w, assets/comic/panel008_2x.png 3045w" aria-labelledby="caption8" tabindex="0"/> <figcaption class="caption"> <p id="caption8"> Kai leans over to Mateo, who suddenly looks very sweaty. Kai says “You okay, buddy? You’re a little, uh, damp.” Mateo tugs at his collar and says “Sorry, I just get kinda worried about all this AI stuff.” Brad sits up in his seat eagerly. Martha, speaking from off-panel, says “Where would you start?” </p> </figcaption> </figure> <figure class="progressive" data-panel="9"> <img class="preview" src="assets/comic/panel009.png" data-src="assets/comic/panel009_1x.png" data-srcset="assets/comic/panel009.png 750w, assets/comic/panel009_1x.png 1500w, assets/comic/panel009_2x.png 3045w" aria-labelledby="caption9" tabindex="0"/> <figcaption class="caption"> <p id="caption9"> Brad waves his hand while shouting “Ooh! Ooh! Ooh! Ooh!” Matha calls on him while saying “Uh…yes?” </p> </figcaption> </figure> <figure class="progressive" data-panel="10"> <img class="preview" src="assets/comic/panel010.png" data-src="assets/comic/panel010_1x.png" data-srcset="assets/comic/panel010.png 750w, assets/comic/panel010_1x.png 1500w, assets/comic/panel010_2x.png 3045w" aria-labelledby="caption10" tabindex="0"/> <figcaption class="caption"> <p id="caption10"> Brad grabs Mateo’s phone, beaming. “It’s easy!” Brad says. “Get your app to upload the user’s data to the server, and you can use it however you want!” Mateo looks panicked and says “Hey, that’s personal!” </p> </figcaption> </figure> <figure class="progressive" data-panel="11"> <img class="preview" src="assets/comic/panel011.png" data-src="assets/comic/panel011_1x.png" data-srcset="assets/comic/panel011.png 750w, assets/comic/panel011_1x.png 1500w, assets/comic/panel011_2x.png 3045w" aria-labelledby="caption11" tabindex="0"/> <figcaption class="caption"> <p id="caption11"> Aliyah looks up from her notepad and says “Wow. Can you say, ‘privacy nightmare’?” In the background, Mateo anxiously tries to switch his phone off while muttering “Airplane mode. Airplane mode.” Brad spreads his hands and whines. “Come on, I just wanna make the app better.” </p> </figcaption> </figure> <figure class="progressive" data-panel="12"> <img class="preview" src="assets/comic/panel012.png" data-src="assets/comic/panel012_1x.png" data-srcset="assets/comic/panel012.png 750w, assets/comic/panel012_1x.png 1500w, assets/comic/panel012_2x.png 3045w" aria-labelledby="caption12" tabindex="0"/> <figcaption class="caption"> <p id="caption12"> A close-up of Martha’s face, lit dramatically from below. She turns and looks over her shoulder. “Ah,” she says. “That’s what we said before...The Incident.” Kai looks up at her and asks “Whoa, what happened?” </p> </figcaption> </figure> <figure class="progressive" data-panel="13"> <img class="preview" src="assets/comic/panel013.png" data-src="assets/comic/panel013_1x.png" data-srcset="assets/comic/panel013.png 750w, assets/comic/panel013_1x.png 1500w, assets/comic/panel013_2x.png 3045w" aria-labelledby="caption13" tabindex="0"/> <figcaption class="caption"> <p id="caption13"> A little icon of Martha’s head in the corner says “It seemed like a good idea at the time…” A smartphone screen showing an app-store advertisement for “Paw Pilot: Your Constant Canine Companion!” The advertised features include: Plot your perfect pet path with suggested routes Fewer vet visits with health tracking Earn cash for chow with Rover Rewards The app has a logo of a cute corgi in a hat sitting above the words “Always watching! Always listening!” </p> </figcaption> </figure> <figure class="progressive" data-panel="14"> <img class="preview" src="assets/comic/panel014.png" data-src="assets/comic/panel014_1x.png" data-srcset="assets/comic/panel014.png 750w, assets/comic/panel014_1x.png 1500w, assets/comic/panel014_2x.png 3045w" aria-labelledby="caption14" tabindex="0"/> <figcaption class="caption"> <p id="caption14"> A laptop showing app store reviews of Paw Pilot. It has a dismal rating of 0.001 Angry reviews say “Leaked doggy cam footage!” “Ruined my marriage!” and “Banned from Arby’s for life!” Grouchy red emojis abound. Devi and Aliyah observe from the bottom corner of the panel. “Ooh,” says Devi, wincing. “That was a bad leak.” Aliyah says “Like I said…” </p> </figcaption> </figure> <figure class="progressive" data-panel="15"> <img class="preview" src="assets/comic/panel015.png" data-src="assets/comic/panel015_1x.png" data-srcset="assets/comic/panel015.png 750w, assets/comic/panel015_1x.png 1500w, assets/comic/panel015_2x.png 3045w" aria-labelledby="caption15" tabindex="0"/> <figcaption class="caption"> <p id="caption15"> Aliyah finishes speaking from off-panel and says “...privacy nightmare.” An assortment of dramatic headlines from The Guardian (“Royal Corgi Hostage Debacle”) and The Times (“Pooch Pilfered by Pirates). A tablet devices plays footage of protesters carrying signs that say “Our dogs, our data.” </p> </figcaption> </figure> <figure class="progressive" data-panel="16"> <img class="preview" src="assets/comic/panel016.png" data-src="assets/comic/panel016_1x.png" data-srcset="assets/comic/panel016.png 750w, assets/comic/panel016_1x.png 1500w, assets/comic/panel016_2x.png 3045w" aria-labelledby="caption16" tabindex="0"/> <figcaption class="caption"> <p id="caption16"> Back in the conference room, Martha sketches a neural network model on a whiteboard. Martha says: “But look…say you have an app, any app–that relies on machine learning.” Devi and Aliyah sit in the foreground and snicker to each other. “Just as long as it’s not Paw Pilot,” says Devi. </p> </figcaption> </figure> <figure class="progressive" data-panel="17"> <img class="preview" src="assets/comic/panel017.png" data-src="assets/comic/panel017_1x.png" data-srcset="assets/comic/panel017.png 750w, assets/comic/panel017_1x.png 1500w, assets/comic/panel017_2x.png 3045w" aria-labelledby="caption17" tabindex="0"/> <figcaption class="caption"> <p id="caption17"> Martha reveals her whiteboard diagram, showing a central brain-like network querying a variety of shapes around it. Martha says “The real-world performance of your machine learning model depends on the relevance of the data used to train it.” </p> </figcaption> </figure> <figure class="progressive" data-panel="18"> <img class="preview" src="assets/comic/panel018.png" data-src="assets/comic/panel018_1x.png" data-srcset="assets/comic/panel018.png 750w, assets/comic/panel018_1x.png 1500w, assets/comic/panel018_2x.png 3045w" aria-labelledby="caption18" tabindex="0"/> <figcaption class="caption"> <p id="caption18"> Martha leans over to gesture at Mateo’s phone, saying “And the best data lives right here at the source on the devices we use every day.” Mateo recoils and says “Eep!” In the background, Brad claps his hands in front of the whiteboard and squeals “Ooh, diagrams!” </p> </figcaption> </figure> <figure class="progressive" data-panel="19"> <img class="preview" src="assets/comic/panel019.png" data-src="assets/comic/panel019_1x.png" data-srcset="assets/comic/panel019.png 750w, assets/comic/panel019_1x.png 1500w, assets/comic/panel019_2x.png 3045w" aria-labelledby="caption19" tabindex="0"/> <figcaption class="caption"> <p id="caption19"> Kai looks skeptically at Martha and says “Hang on, isn’t accessing that sensitive data the whole reason your app tanked?” Mateo looks nervous between them. Martha looks sly and says “Ah, but what if the data never leaves your device?” </p> </figcaption> </figure> <figure class="progressive" data-panel="20"> <img class="preview" src="assets/comic/panel020.png" data-src="assets/comic/panel020_1x.png" data-srcset="assets/comic/panel020.png 750w, assets/comic/panel020_1x.png 1500w, assets/comic/panel020_2x.png 3045w" aria-labelledby="caption20" tabindex="0"/> <figcaption class="caption"> <p id="caption20"> Brad pouts, gesturing to the white-board, where he has annotated the diagram with a bunch of red hearts and the words “Masie + Brad, BFFs.” He says “Aww, then I can’t use it to train Masie.” Martha grins and snaps her fingers in the foreground saying “Oh, but you can.” Aliyah looks at Brad incredulously and mutters “You named it ‘Masie’?” </p> </figcaption> </figure> <figure class="progressive" data-panel="21"> <img class="preview" src="assets/comic/panel021.png" data-src="assets/comic/panel021_1x.png" data-srcset="assets/comic/panel021.png 750w, assets/comic/panel021_1x.png 1500w, assets/comic/panel021_2x.png 3045w" aria-labelledby="caption21" tabindex="0"/> <figcaption class="caption"> <p id="caption21"> Martha pops into view among a field of smartphones matched to each of the interns, who all sit inside their screens as tiny versions of themselves. Martha is surrounded by icons depicting neural networks, security features, and training models. Martha shouts “Welcome to the world of federated learning! We’re about to train a centralized model on decentralized data.” Brad looks awed. </p> </figcaption> </figure> <figure class="progressive" data-panel="22"> <img class="preview" src="assets/comic/panel022.png" data-src="assets/comic/panel022_1x.png" data-srcset="assets/comic/panel022.png 750w, assets/comic/panel022_1x.png 1500w, assets/comic/panel022_2x.png 3045w" aria-labelledby="caption22" tabindex="0"/> <figcaption class="caption"> <p id="caption22"> Martha stands between a central neural network model and a floating smartphone, holding out her hands to stop them from interacting. She says “On-device data can be used to train a smarter central model and improve our users’ experience. But since there’s no way we’d wanna bring that data to the server…” </p> </figcaption> </figure> <figure class="progressive" data-panel="23"> <img class="preview" src="assets/comic/panel023.png" data-src="assets/comic/panel023_1x.png" data-srcset="assets/comic/panel023.png 750w, assets/comic/panel023_1x.png 1500w, assets/comic/panel023_2x.png 3045w" aria-labelledby="caption23" tabindex="0"/> <figcaption class="caption"> <p id="caption23"> A small neural network training model floats above Martha’s hand. She’s ready to toss it to the five interns’ mobile phones. Martha says “..the training can be brought to the device!” </p> </figcaption> </figure> <figure class="progressive" data-panel="24"> <img class="preview" src="assets/comic/panel024.png" data-src="assets/comic/panel024_1x.png" data-srcset="assets/comic/panel024.png 750w, assets/comic/panel024_1x.png 1500w, assets/comic/panel024_2x.png 3045w" aria-labelledby="caption24" tabindex="0"/> <figcaption class="caption"> <p id="caption24"> Devi and Martha float through space. A planet drifts by. Devi is busy typing on her phone. She says “This better not drain my battery.” Martha says “Not to worry. Devices only participate when they’re eligible.” </p> </figcaption> </figure> <figure class="progressive" data-panel="25"> <img class="preview" src="assets/comic/panel025.png" data-src="assets/comic/panel025_1x.png" data-srcset="assets/comic/panel025.png 750w, assets/comic/panel025_1x.png 1500w, assets/comic/panel025_2x.png 3045w" aria-labelledby="caption25" tabindex="0"/> <figcaption class="caption"> <p id="caption25"> Devi and Martha float past a large smartphone, plugged in and displaying a Do Not Disturb icon on its screen. Devi looks startled and says “Whoa!” The smartphone is surrounded by labels that say “charging,” “on wi-fi,” and “idle,” the basic requirements for eligibility. Martha says “We’re not impacting our users or their phones at all!” </p> </figcaption> </figure> <figure class="progressive" data-panel="26"> <img class="preview" src="assets/comic/panel026.png" data-src="assets/comic/panel026_1x.png" data-srcset="assets/comic/panel026.png 750w, assets/comic/panel026_1x.png 1500w, assets/comic/panel026_2x.png 3045w" aria-labelledby="caption26" tabindex="0"/> <figcaption class="caption"> <p id="caption26"> All five interns appear in their phones at the bottom of the panel. Mateo and Brad’s phones float slightly higher in the lineup with tiny check marks below them. Martha floats further above them and says “Let’s say you five represent all our users. Some of you probably have relevant data for the problem I’m trying to solve, and so—” Brad interrupts her, practically hopping out of his phone saying “Ooh! Pick me! Pick me! I’m eligible.” Mateo looks anxious and says “Uhh.” </p> </figcaption> </figure> <figure class="progressive" data-panel="27"> <img class="preview" src="assets/comic/panel027.png" data-src="assets/comic/panel027_1x.png" data-srcset="assets/comic/panel027.png 750w, assets/comic/panel027_1x.png 1500w, assets/comic/panel027_2x.png 3045w" aria-labelledby="caption27" tabindex="0"/> <figcaption class="caption"> <p id="caption27"> Martha rolls her eyes and forges ahead, ignoring Brad. “–and so, a subset of devices are selected to receive a training model.” The central neural network model points at Mateo’s phone. He lets out a nervous yelp. </p> </figcaption> </figure> <figure class="progressive" data-panel="28"> <img class="preview" src="assets/comic/panel028.png" data-src="assets/comic/panel028_1x.png" data-srcset="assets/comic/panel028.png 750w, assets/comic/panel028_1x.png 1500w, assets/comic/panel028_2x.png 3045w" aria-labelledby="caption28" tabindex="0"/> <figcaption class="caption"> <p id="caption28"> Martha and Mateo float above the following diagram: The central neural network model sends a small yellow training model over to Mateo’s phone. Martha reassures him by saying “It’s just a few megs.” Mateo doesn’t look convinced and says “Hmm.” </p> </figcaption> </figure> <figure class="progressive" data-panel="29"> <img class="preview" src="assets/comic/panel029.png" data-src="assets/comic/panel029_1x.png" data-srcset="assets/comic/panel029.png 750w, assets/comic/panel029_1x.png 1500w, assets/comic/panel029_2x.png 3045w" aria-labelledby="caption29" tabindex="0"/> <figcaption class="caption"> <p id="caption29"> The small neural network training model appears inside of Mateo’s phone. Martha holds up a stopwatch and says “It trains on your data in just a few minutes...” Mateo looks surprised and says “Oh!” </p> </figcaption> </figure> <figure class="progressive" data-panel="30"> <img class="preview" src="assets/comic/panel030.png" data-src="assets/comic/panel030_1x.png" data-srcset="assets/comic/panel030.png 750w, assets/comic/panel030_1x.png 1500w, assets/comic/panel030_2x.png 3045w" aria-labelledby="caption30" tabindex="0"/> <figcaption class="caption"> <p id="caption30"> The stopwatch goes ding! Martha continues: “...sends its training results (not your data) to the server…” The training model now floats above Mateo’s phone, as a delta symbol gets sent back toward the central neural network. Mateo looks more intrigued, saying “Oooh!” </p> </figcaption> </figure> <figure class="progressive" data-panel="31"> <img class="preview" src="assets/comic/panel031.png" data-src="assets/comic/panel031_1x.png" data-srcset="assets/comic/panel031.png 750w, assets/comic/panel031_1x.png 1500w, assets/comic/panel031_2x.png 3045w" aria-labelledby="caption31" tabindex="0"/> <figcaption class="caption"> <p id="caption31"> Martha snaps her fingers, saying “…and disappears!” The model vanishes from above Mateo’s phone with a poof. Mateo smiles and looks rueful, saying “Aw! I was just starting to like that little guy.” </p> </figcaption> </figure> <figure class="progressive" data-panel="32"> <img class="preview" src="assets/comic/panel032.png" data-src="assets/comic/panel032_1x.png" data-srcset="assets/comic/panel032.png 750w, assets/comic/panel032_1x.png 1500w, assets/comic/panel032_2x.png 3045w" aria-labelledby="caption32" tabindex="0"/> <figcaption class="caption"> <p id="caption32"> Aliyah and Kai investigate the passage of the delta symbol toward the central neural network. Aliyah holds up a magnifying glass and says “Hold up. Can’t you reconstruct the data from the results that are sent to the server?” Kai floats below and gestures to the right-hand side of the panel, where there’s a lock. “No way!” Kai says, “It’s gotta be encrypted from the start, right?” </p> </figcaption> </figure> <figure class="progressive" data-panel="33"> <img class="preview" src="assets/comic/panel033.png" data-src="assets/comic/panel033_1x.png" data-srcset="assets/comic/panel033.png 750w, assets/comic/panel033_1x.png 1500w, assets/comic/panel033_2x.png 3045w" aria-labelledby="caption33" tabindex="0"/> <figcaption class="caption"> <p id="caption33"> Martha leans in from off-panel, saying “Exactly. And it’s not just encrypted, it can be encrypted with a key that the server doesn’t have.” Below Martha, a line of phones all send delta symbols back toward the central model. Before they reach it, the deltas combine into a larger delta symbol, rendering their individual characteristics anonymous. </p> </figcaption> </figure> <figure class="progressive has-footnote" data-panel="34"> <img class="preview" src="assets/comic/panel034.png" data-src="assets/comic/panel034_1x.png" data-srcset="assets/comic/panel034.png 750w, assets/comic/panel034_1x.png 1500w, assets/comic/panel034_2x.png 3045w" aria-labelledby="caption34 footnote34" tabindex="0"/> <figcaption class="caption"> <p id="caption34"> Martha narrates from off-panel, saying “Secure aggregation enables the server to combine the encrypted results, and only decrypt the aggregate.” </p> </figcaption> <figcaption class="footnote"> <p id="footnote34"> <strong>Secure aggregation</strong> is an interactive cryptographic protocol for computing sums of masked vectors, like model weights. It works by coordinating the exchange of random masks among pairs of participating clients, such that the masks cancel out when a sufficient number of inputs are received. To read more about secure aggregation, see <a target="_blank" rel="noopener" href="https://ai.google/research/pubs/pub47246">Practical Secure Aggregation for Privacy-Preserving Machine Learning</a>. </p> </figcaption> <div class="footnote-icon"></div> </figure> <figure class="progressive" data-panel="35"> <img class="preview" src="assets/comic/panel035.png" data-src="assets/comic/panel035_1x.png" data-srcset="assets/comic/panel035.png 750w, assets/comic/panel035_1x.png 1500w, assets/comic/panel035_2x.png 3045w" aria-labelledby="caption35" tabindex="0"/> <figcaption class="caption"> <p id="caption35"> Martha draws wavy lines in two colors across incoming information from a phone, demonstrating zero-sum masking. She says “On each device, before anything is sent, the secure aggregation protocol adds zero-sum masks to scramble the training results. When you add up all those training results” Kai jumps in excitedly and says “the masks exactly cancel out. Nice!” </p> </figcaption> </figure> <figure class="progressive" data-panel="36"> <img class="preview" src="assets/comic/panel036.png" data-src="assets/comic/panel036_1x.png" data-srcset="assets/comic/panel036.png 750w, assets/comic/panel036_1x.png 1500w, assets/comic/panel036_2x.png 3045w" aria-labelledby="caption36" tabindex="0"/> <figcaption class="caption"> <p id="caption36"> Aliyah points her magnifying glass towards the centralized neural network and says “OK, so the server can’t see any single phone’s result. But what if one phone has really unique data? Could that data be compromised by showing up inside the model?” </p> </figcaption> </figure> <figure class="progressive" data-panel="37"> <img class="preview" src="assets/comic/panel037.png" data-src="assets/comic/panel037_1x.png" data-srcset="assets/comic/panel037.png 750w, assets/comic/panel037_1x.png 1500w, assets/comic/panel037_2x.png 3045w" aria-labelledby="caption37" tabindex="0"/> <figcaption class="caption"> <p id="caption37"> Martha and Mateo stand on opposite sides of the neural network. Martha considers Aliyah’s question thoughtfully, and says “Well, it’s possible, but we don’t want it to happen. For machine learning to work best, models need to capture common patterns in the data, not memorize things that are specific to one phone.” Mateo looks frightened as he inspects the neural network through a magnifying glass, saying “Wait a second, is that me?” </p> </figcaption> </figure> <figure class="progressive" data-panel="38"> <img class="preview" src="assets/comic/panel038.png" data-src="assets/comic/panel038_1x.png" data-srcset="assets/comic/panel038.png 750w, assets/comic/panel038_1x.png 1500w, assets/comic/panel038_2x.png 3045w" aria-labelledby="caption38" tabindex="0"/> <figcaption class="caption"> <p id="caption38"> A close-up view of the neural network is shown under a magnifying glass, with a blocky but recognizable Mateo-shaped outline within it, among many background lines. Martha floats to the side of the magnifying glass, looking concerned. She says “Ah, looks like you’re the rare data! And we don’t want the model memorizing that. This is why we have ways to measure and control how much a model might be memorizing.” </p> </figcaption> </figure> <figure class="progressive" data-panel="39"> <img class="preview" src="assets/comic/panel039.png" data-src="assets/comic/panel039_1x.png" data-srcset="assets/comic/panel039.png 750w, assets/comic/panel039_1x.png 1500w, assets/comic/panel039_2x.png 3045w" aria-labelledby="caption39" tabindex="0"/> <figcaption class="caption"> <p id="caption39"> The close-up under the magnifying glass transforms into abstract shapes that are no longer recognizably Mateo. Martha and Brad float on either side of the magnifying glass. Martha happily says, “For example, watch what happens if we limit how much any one phone can contribute and add noise to obscure rare data.” Brad gleefully chimes in “Ooh, I know this one! It’s differential privacy!” </p> </figcaption> </figure> <figure class="progressive has-footnote" data-panel="40"> <img class="preview" src="assets/comic/panel040.png" data-src="assets/comic/panel040_1x.png" data-srcset="assets/comic/panel040.png 750w, assets/comic/panel040_1x.png 1500w, assets/comic/panel040_2x.png 3045w" aria-labelledby="caption40 footnote40" tabindex="0"/> <figcaption class="caption"> <p id="caption40"> An off-panel narrator says “Differential privacy is a well-established way to deal with the risk of model memorization,* where a shared model’s parameters might be too influenced by a single contributor.” </p> </figcaption> <figcaption class="footnote"> <p id="footnote40"> Understanding and mitigating the risks of <strong>model memorization</strong> is an active area of research. Techniques to measure memorization are explored, e.g. in the 2018 paper <a target="_blank" rel="noopener" href="https://g.co/research/SecretSharerPaper"> The Secret Sharer: Measuring Unintended Neural Network Memorization & Extracting Secrets</a>. Memorization risk can be mitigated by pre-filtering rare or sensitive information before training. More sophisticated mitigation techniques include differentially private model training as explored, for example, in the 2018 paper <a target="_blank" rel="noopener" href="https://g.co/research/DPLanguageModelsPaper">Learning Differentially Private Recurrent Language Models</a>, which shows how to learn model weights that are not too dependent on any one device’s data. For more information on differential privacy, the canonical textbook “The Algorithmic Foundations of Differential Privacy” by Cynthia Dwork and Aaron Roth is available from NOW publishers and <a target="_blank" rel="noopener" href="https://www.cis.upenn.edu/~aaroth/privacybook.html">online</a>. </p> </figcaption> <div class="footnote-icon"></div> </figure> <figure class="progressive" data-panel="41"> <img class="preview" src="assets/comic/panel041.png" data-src="assets/comic/panel041_1x.png" data-srcset="assets/comic/panel041.png 750w, assets/comic/panel041_1x.png 1500w, assets/comic/panel041_2x.png 3045w" aria-labelledby="caption41" tabindex="0"/> <figcaption class="caption"> <p id="caption41"> Brad jumps up from behind the large central model, shouting “Aw, yeah! Now we can update our model and push out Version 2, right?” Martha says “Not so fast. We’ve got more training rounds ahead of us, and then we’ll wanna test the model before rolling it out.” </p> </figcaption> </figure> <figure class="progressive" data-panel="42"> <img class="preview" src="assets/comic/panel042.png" data-src="assets/comic/panel042_1x.png" data-srcset="assets/comic/panel042.png 750w, assets/comic/panel042_1x.png 1500w, assets/comic/panel042_2x.png 3045w" aria-labelledby="caption42" tabindex="0"/> <figcaption class="caption"> <p id="caption42"> Brad and Aliyah stand on top of the centralized model. Brad says “But we don’t have the data to test the improved model…” Aliyah continues “…because it’s all on the users’ devices, where it belongs…” </p> </figcaption> </figure> <figure class="progressive" data-panel="43"> <img class="preview" src="assets/comic/panel043.png" data-src="assets/comic/panel043_1x.png" data-srcset="assets/comic/panel043.png 750w, assets/comic/panel043_1x.png 1500w, assets/comic/panel043_2x.png 3045w" aria-labelledby="caption43" tabindex="0"/> <figcaption class="caption"> <p id="caption43"> Brad and Aliyah think for a moment. </p> </figcaption> </figure> <figure class="progressive" data-panel="44"> <img class="preview" src="assets/comic/panel044.png" data-src="assets/comic/panel044_1x.png" data-srcset="assets/comic/panel044.png 750w, assets/comic/panel044_1x.png 1500w, assets/comic/panel044_2x.png 3045w" aria-labelledby="caption44" tabindex="0"/> <figcaption class="caption"> <p id="caption44"> Light bulbs appear over Brad and Aliyah’s heads. They look at each other and shout “so that’s where we’ll test it!” </p> </figcaption> </figure> <figure class="progressive" data-panel="45"> <img class="preview" src="assets/comic/panel045.png" data-src="assets/comic/panel045_1x.png" data-srcset="assets/comic/panel045.png 750w, assets/comic/panel045_1x.png 1500w, assets/comic/panel045_2x.png 3045w" aria-labelledby="caption45" tabindex="0"/> <figcaption class="caption"> <p id="caption45"> Martha appears and says “Yes! We safely trained on-device, so now we can safely test the quality of that training where it matters most… Devi holds her phone, which is receiving a copy of the small training model. She says “In the hands of our users!” </p> </figcaption> </figure> <figure class="progressive" data-panel="46"> <img class="preview" src="assets/comic/panel046.png" data-src="assets/comic/panel046_1x.png" data-srcset="assets/comic/panel046.png 750w, assets/comic/panel046_1x.png 1500w, assets/comic/panel046_2x.png 3045w" aria-labelledby="caption46" tabindex="0"/> <figcaption class="caption"> <p id="caption46"> The central neural network model sits in a circle of phones, querying some of them, sending models to others. It’s a busy hive of training activity. Martha narrates “While some phones are training our models, others are testing them. Training, testing, analytics – they’re all tasks we can tackle privately and securely with federation!” </p> </figcaption> </figure> <figure class="progressive" data-panel="47"> <img class="preview" src="assets/comic/panel047.png" data-src="assets/comic/panel047_1x.png" data-srcset="assets/comic/panel047.png 750w, assets/comic/panel047_1x.png 1500w, assets/comic/panel047_2x.png 3045w" aria-labelledby="caption47" tabindex="0"/> <figcaption class="caption"> <p id="caption47"> Back in the conference room, Brad becomes impatient. “Okay, okay, we get it,” he groans. “Can we update everyone’s model already?” Martha leans over and grins at him. “Ah,” she says, “but that was just a little improvement. Bet we can make it even better.” </p> </figcaption> </figure> <figure class="progressive" data-panel="48"> <img class="preview" src="assets/comic/panel048.png" data-src="assets/comic/panel048_1x.png" data-srcset="assets/comic/panel048.png 750w, assets/comic/panel048_1x.png 1500w, assets/comic/panel048_2x.png 3045w" aria-labelledby="caption48" tabindex="0"/> <figcaption class="caption"> <p id="caption48"> A caption reads “Three days and several thousand iterations later…” Brad is doubled over, exhausted. “Nowwww can we update everyone’s model?” he asks. Martha grins at him from her computer, lifting her hand to press a large green button. “Ohh, all right,” she says. </p> </figcaption> </figure> <figure class="progressive" data-panel="49"> <img class="preview" src="assets/comic/panel049.png" data-src="assets/comic/panel049_1x.png" data-srcset="assets/comic/panel049.png 750w, assets/comic/panel049_1x.png 1500w, assets/comic/panel049_2x.png 3045w" aria-labelledby="caption49" tabindex="0"/> <figcaption class="caption"> <p id="caption49"> Back in diagram-world, a central model floats in the middle of the panel, with five arrows pointing out to the five interns’ phones. A small neural model, now purple instead of yellow, floats above each phone. The interns all appear within their phone screens. Martha says “This new model has learned directly from our user’s data without centralizing any of it!” Kai coos “Ooh, such accuracy.” Mateo cheers “And my data’s still mine!” </p> </figcaption> </figure> <figure class="progressive" data-panel="50"> <img class="preview" src="assets/comic/panel050.png" data-src="assets/comic/panel050_1x.png" data-srcset="assets/comic/panel050.png 750w, assets/comic/panel050_1x.png 1500w, assets/comic/panel050_2x.png 3045w" aria-labelledby="caption50" tabindex="0"/> <figcaption class="caption"> <p id="caption50"> Devi looks at her phone and asks “Is this model going to keep learning as I use it?” “No,” says Martha, “the new model is static. It’s as smart as it’s going to be until the next update. But it’s really smart, because it’s learned from thousands of users like you.” </p> </figcaption> </figure> <figure class="progressive" data-panel="51"> <img class="preview" src="assets/comic/panel051.png" data-src="assets/comic/panel051_1x.png" data-srcset="assets/comic/panel051.png 750w, assets/comic/panel051_1x.png 1500w, assets/comic/panel051_2x.png 3045w" aria-labelledby="caption51" tabindex="0"/> <figcaption class="caption"> <p id="caption51"> A smartphone appears with little networks of association on either side of it. To the left is a tiny icon of Commander Riker from Star Trek, to the right is Captain Picard. Kai says “So if I was binge-watching Star Trek for the first-time, my keyboard app would already guess that “Picard”: should follow “Captain” every time “Riker” follows “Commander”?” Martha says “Right! Although I’m more of a Captain Ahab woman, myself.” </p> </figcaption> </figure> <figure class="progressive" data-panel="52"> <img class="preview" src="assets/comic/panel052.png" data-src="assets/comic/panel052_1x.png" data-srcset="assets/comic/panel052.png 750w, assets/comic/panel052_1x.png 1500w, assets/comic/panel052_2x.png 3045w" aria-labelledby="caption52" tabindex="0"/> <figcaption class="caption"> <p id="caption52"> Devi looks thoughtful and says “It’s not so much personalized learning as it is...collaborative learning? Martha smiles and says “Sure! Or, you could just call it…” </p> </figcaption> </figure> <figure class="progressive" data-panel="53"> <img class="preview" src="assets/comic/panel053.png" data-src="assets/comic/panel053_1x.png" data-srcset="assets/comic/panel053.png 750w, assets/comic/panel053_1x.png 1500w, assets/comic/panel053_2x.png 3045w" aria-labelledby="caption53" tabindex="0"/> <figcaption class="caption"> <p id="caption53"> Mateo triumphantly shouts “Federated learning!” </p> </figcaption> </figure> <figure class="progressive" data-panel="54"> <img class="preview" src="assets/comic/panel054.png" data-src="assets/comic/panel054_1x.png" data-srcset="assets/comic/panel054.png 750w, assets/comic/panel054_1x.png 1500w, assets/comic/panel054_2x.png 3045w" aria-labelledby="caption54" tabindex="0"/> <figcaption class="caption"> <p id="caption54"> Brad looks thoughtful and says “There are a lot of places where utility and privacy seem to be in conflict.” Mateo looks determined in the foreground, saying “But they don’t have to be!” </p> </figcaption> </figure> <figure class="progressive has-footnote" data-panel="55"> <img class="preview" src="assets/comic/panel055.png" data-src="assets/comic/panel055_1x.png" data-srcset="assets/comic/panel055.png 750w, assets/comic/panel055_1x.png 1500w, assets/comic/panel055_2x.png 3045w" aria-labelledby="caption55 footnote55" tabindex="0"/> <figcaption class="caption"> <p id="caption55"> Martha and the interns appear under a large tree, with equally large roots exposed beneath the ground. Martha leans against the trunk while the interns all look up at the branches and enjoy the shade. Martha says “Federated learning and analytics are new fields, with established roots* and tremendous room to grow. And they allow us to test and train on all kinds of devices–not just phones and tablets!” </p> </figcaption> <figcaption class="footnote"> <p id="footnote55"> <strong>Federated learning and analytics</strong> come from a rich heritage of distributed optimization, machine learning and privacy research. They are inspired by many systems and tools, including <a target="_blank" rel="noopener" href="https://en.wikipedia.org/wiki/MapReduce">MapReduce</a> for distributed computation, <a target="_blank" rel="noopener" href="https://www.tensorflow.org/">TensorFlow</a> for machine learning and <a target="_blank" rel="noopener" href="https://security.googleblog.com/2014/10/learning-statistics-with-privacy-aided.html">RAPPOR</a> for privacy-preserving analytics. The federated learning approach for training deep networks was first articulated in a 2016 paper published by Google AI researchers: <a target="_blank" rel="noopener" href="https://arxiv.org/abs/1602.05629">Communication-Efficient Learning of Deep Networks from Decentralized Data</a>. </p> </figcaption> <div class="footnote-icon"></div> </figure> <figure class="progressive" data-panel="56"> <img class="preview" src="assets/comic/panel056.png" data-src="assets/comic/panel056_1x.png" data-srcset="assets/comic/panel056.png 750w, assets/comic/panel056_1x.png 1500w, assets/comic/panel056_2x.png 3045w" aria-labelledby="caption56" tabindex="0"/> <figcaption class="caption"> <p id="caption56"> Martha leans over two iconographic cars, one driven by a person and one driven by AI. They circle a small training model. Martha says “Imagine training self-driving cars on aggregated real-world driver behavior.” </p> </figcaption> </figure> <figure class="progressive" data-panel="57"> <img class="preview" src="assets/comic/panel057.png" data-src="assets/comic/panel057_1x.png" data-srcset="assets/comic/panel057.png 750w, assets/comic/panel057_1x.png 1500w, assets/comic/panel057_2x.png 3045w" aria-labelledby="caption57" tabindex="0"/> <figcaption class="caption"> <p id="caption57"> Aliyah strokes her chin while looking at two hospital buildings. A training model sits between them. Arrows show clipboards of information circulating between the buildings. Aliyah says “Or how about helping hospitals improve diagnostics while maintaining patient privacy!” </p> </figcaption> </figure> <figure class="progressive" data-panel="58"> <img class="preview" src="assets/comic/panel058.png" data-src="assets/comic/panel058_1x.png" data-srcset="assets/comic/panel058.png 750w, assets/comic/panel058_1x.png 1500w, assets/comic/panel058_2x.png 3045w" aria-labelledby="caption58" tabindex="0"/> <figcaption class="caption"> <p id="caption58"> Martha beams and says “And that’s only the beginning! So, what have we learned?” </p> </figcaption> </figure> <figure class="progressive" data-panel="59"> <img class="preview" src="assets/comic/panel059.png" data-src="assets/comic/panel059_1x.png" data-srcset="assets/comic/panel059.png 750w, assets/comic/panel059_1x.png 1500w, assets/comic/panel059_2x.png 3045w" aria-labelledby="caption59" tabindex="0"/> <figcaption class="caption"> <p id="caption59"> The interns stand together and all chime in to say: “That we can learn from everyone without learning about any one!” Mateo looks wide-eyed with excitement and says “I so wanna learn more!” </p> </figcaption> </figure> <figure class="progressive" data-panel="60"> <img class="preview" src="assets/comic/panel060.png" data-src="assets/comic/panel060_1x.png" data-srcset="assets/comic/panel060.png 750w, assets/comic/panel060_1x.png 1500w, assets/comic/panel060_2x.png 3045w" aria-labelledby="caption60" tabindex="0"/> <figcaption class="caption"> <p id="caption60"> Martha regards the interns proudly and says “You’ve all come so far. I think there’s a forever home for all of you here at this compan-” The Boss bursts through the door to the conference room and shouts “Change of plan, everyone!” </p> </figcaption> </figure> <figure class="progressive" data-panel="61"> <img class="preview" src="assets/comic/panel061.png" data-src="assets/comic/panel061_1x.png" data-srcset="assets/comic/panel061.png 750w, assets/comic/panel061_1x.png 1500w, assets/comic/panel061_2x.png 3045w" aria-labelledby="caption61" tabindex="0"/> <figcaption class="caption"> <p id="caption61"> Closeup on the Boss, beaming like a maniac. He says “I have just learned that we are extremely bankrupt.” </p> </figcaption> </figure> <figure class="progressive" data-panel="62"> <img class="preview" src="assets/comic/panel062.png" data-src="assets/comic/panel062_1x.png" data-srcset="assets/comic/panel062.png 750w, assets/comic/panel062_1x.png 1500w, assets/comic/panel062_2x.png 3045w" aria-labelledby="caption62" tabindex="0"/> <figcaption class="caption"> <p id="caption62"> The interns and Martha all stare at the Boss. He looks at them blankly. </p> </figcaption> </figure> <figure class="progressive" data-panel="63"> <img class="preview" src="assets/comic/panel063.png" data-src="assets/comic/panel063_1x.png" data-srcset="assets/comic/panel063.png 750w, assets/comic/panel063_1x.png 1500w, assets/comic/panel063_2x.png 3045w" aria-labelledby="caption63" tabindex="0"/> <figcaption class="caption"> <p id="caption63"> Martha turns to the interns and says “Who wants to go start a new company?” </p> </figcaption> </figure> <figure class="progressive" data-panel="64"> <img class="preview" src="assets/comic/panel064.png" data-src="assets/comic/panel064_1x.png" data-srcset="assets/comic/panel064.png 750w, assets/comic/panel064_1x.png 1500w, assets/comic/panel064_2x.png 3045w" aria-labelledby="caption64" tabindex="0"/> <figcaption class="caption"> <p id="caption64"> The interns immediately raise their hands and grin. Martha looks pleased. The Boss keels over. </p> </figcaption> </figure> <figure class="progressive" data-panel="65"> <img class="preview" src="assets/comic/panel065.png" data-src="assets/comic/panel065_1x.png" data-srcset="assets/comic/panel065.png 750w, assets/comic/panel065_1x.png 1500w, assets/comic/panel065_2x.png 3045w" aria-labelledby="caption65" tabindex="0"/> <figcaption class="caption"> <p id="caption65"> The End! A corgi chases a machine learning model across the panel, saying “Yip!” </p> </figcaption> </figure> <div class="social" id="social-bottom"> <span class="share">Share</span> <a target="_blank" class="logo" href="https://twitter.com/home?status=https%3A//federated.withgoogle.com"> <svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 28 28" class="share-links-icon twitter" aria-label="Share on Twitter"><path d="M25.76,6.32A9.63,9.63,0,0,1,23,7.1,5,5,0,0,0,25.1,4.36,9.67,9.67,0,0,1,22,5.56,4.83,4.83,0,0,0,18.45,4a4.91,4.91,0,0,0-4.88,4.95,5,5,0,0,0,.13,1.13,13.78,13.78,0,0,1-10-5.17,5,5,0,0,0,1.51,6.61A4.8,4.8,0,0,1,3,10.89V11a4.93,4.93,0,0,0,3.91,4.85,4.82,4.82,0,0,1-2.2.09,4.89,4.89,0,0,0,4.55,3.44,9.69,9.69,0,0,1-7.22,2A13.65,13.65,0,0,0,9.47,23.6c9,0,13.87-7.54,13.87-14.08q0-.32,0-.64a10,10,0,0,0,2.43-2.56"></path></svg> </a> <a target="_blank" class="logo" href="https://www.facebook.com/sharer/sharer.php?u=https%3A//federated.withgoogle.com"> <svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 28 28" class="share-links-icon facebook" aria-label="Share on Facebook"><path d="M24.68,2H3.32A1.32,1.32,0,0,0,2,3.32V24.68A1.32,1.32,0,0,0,3.32,26H14.82V16.71H11.69V13.08h3.13V10.41c0-3.1,1.89-4.79,4.66-4.79a25.65,25.65,0,0,1,2.79.14V9H20.35c-1.5,0-1.8.71-1.8,1.76v2.31h3.59l-.47,3.62H18.56V26h6.12A1.32,1.32,0,0,0,26,24.68V3.32A1.32,1.32,0,0,0,24.68,2Z"></path></svg> </a> <a target="_blank" class="logo" href="https://www.linkedin.com/shareArticle?mini=true&url=https%3A//federated.withgoogle.com&title=Federated Learning"> <svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 28 28" class="share-links-icon linkedin" aria-label="Share on LinkedIn"><path d="M22.57,22.45H19V16.88c0-1.33,0-3-1.88-3s-2.17,1.45-2.17,2.94v5.66H11.31V11h3.46v1.56h.05a3.8,3.8,0,0,1,3.42-1.85c3.65,0,4.33,2.37,4.33,5.46ZM7.24,9.43A2.06,2.06,0,1,1,9.34,7.37,2.08,2.08,0,0,1,7.24,9.43ZM5.44,11H9.05V22.45H5.44V11ZM24.37,2H3.62a1.76,1.76,0,0,0-1.8,1.73V24.27A1.77,1.77,0,0,0,3.62,26H24.37a1.77,1.77,0,0,0,1.8-1.73V3.73A1.77,1.77,0,0,0,24.37,2Z"></path></svg> </a> </div> </div> </section> <section class="limit-width"> <p>Story by <a target="_blank" rel="noopener" href="https://lucybellwood.com/">Lucy Bellwood</a> and <a target="_blank" rel="noopener" href="http://scottmccloud.com/">Scott McCloud</a>. Art by Lucy Bellwood. This comic is licensed under the <a target="_blank" rel="noopener" href="https://creativecommons.org/licenses/by-nc-nd/3.0/">Creative Commons Attribution-Noncommercial-NoDerivative Works 3.0 license</a>.<br/>Translation is permitted.</p> </section> <section class="limit-width"> <a class="anchor" id="about"></a> <h2>About</h2> <p>This site is brought to you by the federated learning and analytics team at <a target="_blank" rel="noopener" href="https://research.google/">Google Research</a>. We are a team of researchers and engineers that develop foundational technologies that enable strong privacy guarantees for AI and analytics systems. We are inventors in this space, with work spanning research and deployment of federated learning, analytics, private aggregation, differential privacy, and more. We aim to invent and deploy technologies for a private-by-default ecosystem that empowers users with transparent and verifiable privacy claims.</p> <p>We believe that strong privacy guarantees are essential to the future of tech, and are committed to driving innovation in this space. We maintain a collection of open source projects under <a target="_blank" rel="noopener" href="https://github.com/google-parfait">Parfait</a>.</p> <p>Questions, feedback, ideas? We’d love to hear from you: <a target="_blank" rel="noopener" href="mailto:federated-feedback@google.com">federated-feedback@google.com</a></p> </section> <section class="limit-width"> <a class="anchor" id="learn"></a> <h2>Learn more</h2> <p>Introduction to Federated Learning and Federated Analytics</p> <ul> <li><a target="_blank" rel="noopener" href="https://ai.googleblog.com/2017/04/federated-learning-collaborative.html">Federated Learning: Collaborative Machine Learning without Centralized Training Data</a></li> <li><a target="_blank" rel="noopener" href="https://ai.googleblog.com/2020/05/federated-analytics-collaborative-data.html">Federated Analytics: Collaborative Data Science without Data Collection</a></li> <li><a target="_blank" rel="noopener" href="https://pair.withgoogle.com/explorables/federated-learning/">How Federated Learning Protects Privacy: Interactive Introduction to Federated Learning</a></li> <li><a target="_blank" rel="noopener" href="https://queue.acm.org/detail.cfm?id=3501293">Building Privacy-Preserving Systems for Machine Learning and Data Science on Decentralized Data</a></li> <li><a target="_blank" rel="noopener" href="https://www.youtube.com/watch?v=89BGjQYA0uE">(Google I/O'19) Federated Learning: Machine Learning on Decentralized Data</a></li> </ul> <p>Use Cases in Google Products</p> <ul> <li><a target="_blank" rel="noopener" href="https://ai.googleblog.com/2022/02/federated-learning-with-formal.html">Federated Learning with Formal Differential Privacy Guarantee in Gboard</a></li> <li><a target="_blank" rel="noopener" href="https://ai.googleblog.com/2021/11/predicting-text-selections-with.html">Predicting Text Selections within Federated Learning</a></li> <li><a target="_blank" rel="noopener" href="https://security.googleblog.com/2022/12/trust-in-transparency-private-compute.html">Federated Learning in Android Private Compute Core</a></li> <li><a target="_blank" rel="noopener" href="https://support.google.com/assistant/answer/11140942?hl=en">Google Uses Federated Learning to Improve Speech Models</a></li> <li><a target="_blank" rel="noopener" href="https://support.google.com/messages/answer/9327902?hl=en#zippy=%2Chow-federated-technology-works">How Messages Improves Suggestions with Federated Technology</a></li> </ul> <p>TensorFlow Federated</p> <ul> <li>TensorFlow Federated <a target="_blank" rel="noopener" href="https://www.tensorflow.org/federated">Site</a> and <a target="_blank" rel="noopener" href="https://github.com/tensorflow/federated">GitHub</a></li> <li><a target="_blank" rel="noopener" href="https://medium.com/tensorflow/introducing-tensorflow-federated-a4147aa20041">Introducing TensorFlow Federated</a></li> <li><a target="_blank" rel="noopener" href="https://www.tensorflow.org/federated/tutorials/tutorials_overview">TensorFlow Federated Tutorial Overview</a></li> <li><a target="_blank" rel="noopener" href="https://www.youtube.com/watch?v=JBNas6Yd30A">Google TechTalk Tutorial for TensorFlow Federated </a></li> <li><a target="_blank" rel="noopener" href="https://discuss.tensorflow.org/t/welcome-to-sig-federated/">SIG (Special Interest Groups) Federated</a></li> </ul> </section> <section class="limit-width"> <a class="anchor" id="research"></a> <h2>Research</h2> <p>Best Practices, Challenges and Open Problems</p> <ul> <li><a target="_blank" rel="noopener" href="https://arxiv.org/abs/1912.04977">Advances and Open Problems in Federated Learning</a></li> <li><a target="_blank" rel="noopener" href="https://arxiv.org/pdf/2107.06917">Field Guide to Federated Optimization</a></li> <li><a target="_blank" rel="noopener" href="https://arxiv.org/abs/2206.09262">Motley: Benchmarking Heterogeneity and Personalization in Federated Learning</a></li> </ul> <p>YouTube Playlists of Workshop for Federated Learning and Analytics</p> <ul> <li><a target="_blank" rel="noopener" href="https://www.youtube.com/playlist?list=PLSIUOFhnxEiA-Ky_BuirTS6Amb9A60uSH">2022 Workshop for Federated Learning and Analytics</a></li> <li><a target="_blank" rel="noopener" href="https://www.youtube.com/playlist?list=PLSIUOFhnxEiD9uihG5t9ABdPhSVqQ3HWA">2021 Workshop for Federated Learning and Analytics</a></li> <li><a target="_blank" rel="noopener" href="https://www.youtube.com/playlist?list=PLSIUOFhnxEiCJS8q6SYdc0944xlV_6Jbu">2020 Workshop for Federated Learning and Analytics</a></li> </ul> <p>Federated Differential Privacy Research</p> <ul> <li><a target="_blank" rel="noopener" href="https://arxiv.org/abs/1902.08534">Federated Heavy Hitters Discovery with Differential Privacy</a></li> <li><a target="_blank" rel="noopener" href="https://arxiv.org/pdf/2103.00039">Practical and Private (Deep) Learning without Sampling or Shuffling</a></li> <li><a target="_blank" rel="noopener" href="https://arxiv.org/abs/2102.06387">Improved Differential Privacy for SGD via Optimal Private Linear Operators on Adaptive Streams</a></li> <li><a target="_blank" rel="noopener" href="https://arxiv.org/pdf/2111.02356">Towards Sparse Federated Analytics: Location Heatmaps under Distributed Differential Privacy with Secure Aggregation</a></li> <li><a target="_blank" rel="noopener" href="https://arxiv.org/pdf/2202.08312">The Distributed Discrete Gaussian Mechanism for Federated Learning with Secure Aggregation</a></li> <li><a target="_blank" rel="noopener" href="https://arxiv.org/pdf/2203.03761">The Fundamental Price of Secure Aggregation in Differentially Private Federated Learning</a></li> </ul> <p>Federated Learning Research in Gboard</p> <ul> <li><a target="_blank" rel="noopener" href="https://arxiv.org/abs/1903.10635">Federated Learning of Out-Of-Vocabulary Words</a></li> <li><a target="_blank" rel="noopener" href="https://arxiv.org/abs/1812.02903">Applied Federated Learning: Improving Google Keyboard Query Suggestions</a></li> <li><a target="_blank" rel="noopener" href="https://arxiv.org/abs/1811.03604">Federated Learning for Mobile Keyboard Prediction</a></li> </ul> <p>Fundamental Research in Federated Learning</p> <ul> <li><a target="_blank" rel="noopener" href="https://arxiv.org/abs/1602.05629">Communication-Efficient Learning of Deep Networks from Decentralized Data</a></li> <li><a target="_blank" rel="noopener" href="https://arxiv.org/abs/2003.00295">Adaptive Federated Optimization</a></li> <li><a target="_blank" rel="noopener" href="https://arxiv.org/abs/2208.09432">Federated Select: A Primitive for Communication and Memory-Efficient Federated Learning</a></li> <li><a target="_blank" rel="noopener" href="https://arxiv.org/abs/2205.13655">Mixed Federated Learning: Joint Decentralized and Centralized Learning</a></li> </ul> <p>Secure Aggregation</p> <ul> <li><a target="_blank" rel="noopener" href="https://ai.google/research/pubs/pub47246">Practical Secure Aggregation for Privacy Preserving Machine Learning</a></li> <li><a target="_blank" rel="noopener" href="https://dl.acm.org/doi/10.1145/3372297.3417885">Secure Single-Server Aggregation with (Poly)Logarithmic Overhead</a></li> </ul> <p>Federated Learning System Design</p> <ul> <li><a target="_blank" rel="noopener" href="https://arxiv.org/abs/1902.01046">Towards Federated Learning at Scale: System Design</a></li> </ul> </section> </main> <footer> <div class="limit-width"> <a target="_blank" rel="noopener" href="https://ai.google/" class="google-logo"></a> <ul id="nav-bottom"> <li><a href="#top">Back to top</a></li> <li><a href="#about">About</a></li> <li><a href="#learn">Learn more</a></li> <li><a href="#research">Research</a></li> <li><a target="_blank" rel="noopener" href="https://policies.google.com/">Privacy & Terms</a></li> </ul> </div> </footer> <script> window.ga=window.ga||function(){(ga.q=ga.q||[]).push(arguments)};ga.l=+new Date; ga('create', 'UA-139530313-1', 'auto'); ga('send', 'pageview'); </script> <script async src='https://www.google-analytics.com/analytics.js'></script> <script src="/assets/js/comic-loader.js"></script> </body> </html>