CINXE.COM

<!DOCTYPE html> <html dir="ltr" xml:lang="en" lang="en"> <head> <title>Differential privacy in (a bit) more detail - Ted is writing things</title> <meta http-equiv="Content-type" content="text/html; charset=utf-8" /> <meta name="author" content="Damien Desfontaines" /> <meta name="twitter:creator" content="@TedOnPrivacy" />  <meta name="viewport" content="width=device-width, initial-scale=1" /> <link rel="stylesheet" href="/style/menu.css" type="text/css" /> <link rel="stylesheet" href="/style/blog.css" type="text/css" media="screen" /> <link rel="stylesheet" href="/style/blog-mobile.css" type="text/css" media="(max-width: 580px)" /> <link rel="stylesheet" href="/style/blog-print.css" type="text/css" media="print" /> <link rel="stylesheet" href="/style/pygments.css" type="text/css" /> <link rel="contents" href="posts.html" /> <link rel="icon" href="/favicon.ico" sizes="any"> <link rel="icon" href="/favicon.svg" type="image/svg+xml"> <link rel="apple-touch-icon" href="/apple-touch-icon.png"> <link rel="manifest" href="/site.webmanifest"> <link href="https://desfontain.es/blog/" type="application/rss+xml" rel="alternate" title="Ted is writing things - RSS Feed" /> <meta name="title" property="og:title" content="Differential privacy in (a bit) more detail - Ted is writing things" /> <meta property="twitter:title" content="Differential privacy in (a bit) more detail - Ted is writing things" /> <meta name="description" property="og:description" content="Why does differential privacy work so well? Let's look at it more closely." /> <meta property="twitter:description" content="Why does differential privacy work so well? Let's look at it more closely." /> <meta property="summary" content="Why does differential privacy work so well? Let's look at it more closely." /> <meta name="twitter:card" content="summary"/> <link rel="canonical" href="https://desfontain.es/blog/differential-privacy-in-more-detail.html" /> <link rel="prev" href="differential-privacy-awesomeness.html" /> <link rel="next" href="part-time-phd.html" /> <style type="text/css">  </style> </head> <body id="index" class="home">  <a aria-label="Skip to content" href="#contenu"></a> <div id="menuGlobal"> <table> <tr> <td> <a href="../index.html"> ..@..♦.D. <img src="../flag-uk.png" alt=""/> </a> </td> <td> <a href="../serious.html">About <img src="../flag-uk.png" alt=""/></a> <a href="../serious-fr.html"><img src="../flag-france.gif" alt=""/></a> </td> <td id="menuCourant"> Blog <img src="../flag-uk.png" alt=""/> <a href="../blogue/index.html"><img src="../flag-france.gif" alt=""/></a> </td> <td> <a href="../recettes/index.html">Recipes <img src="../flag-france.gif" alt=""/></a> </td> </tr> <tr id="sousMenu"> <td colspan="4"> <a href="index.html">latest</a> — <a href="rss.xml">rss</a> — <a href="posts.html">archives</a> <a href="differential-privacy-awesomeness.html">← previous</a> — <a href="part-time-phd.html">next →</a> </td> </tr> </table> </div> <div id="container"> <header> <h1><a href="./"> Ted is writing things </a></h1> On privacy, research, and privacy research. </header> <article id="contenu"> <header> <h1> <a href="./differential-privacy-in-more-detail.html">Differential privacy in (a bit) more detail</a> </h1> </header> <footer> <time datetime="2018-08-16T00:00:00+02:00"> 2018-08-16 </time> — updated <time datetime="2019-02-20T00:00:00+01:00"> 2019-02-20 </time> </footer> <div> This post is part of a <a href="friendly-intro-to-differential-privacy.html">series on differential privacy</a>. Check out the <a href="friendly-intro-to-differential-privacy.html">table of contents</a> to see the other articles! <hr> As I mentioned in the <a href="differential-privacy-awesomeness.html">previous article</a>, differential privacy is pretty awesome. If I did a good job, you're now wondering what the real definition looks like. So in this post, I will go into a bit more detail into what differential privacy actually means, and why it works so well. There will be some math! But I promise I will explain all the concepts I use, and give lots of intuition. <h1 id="the-definition">The definition</h1> We saw that a process satisfies differential privacy if its output is basically the same if you change the data of one individual. And by "basically the same", we meant "the probabilities are close". <center> <svg role="img" aria-label="Diagram showing two fake "Guess Who" boards, the second one having one of the people missing. Each board, representing a database, has an arrow going to a silly "magic" gif, and this has an arrow going to a cloud labeled "output". A double arrow labeled "basically the same" points to the two outputs." xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:cc="http://creativecommons.org/ns#" xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#" xmlns:svg="http://www.w3.org/2000/svg" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd" xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape" version="1.1" viewBox="0 0 734.41882 358.09845" stroke-miterlimit="10" id="svg4123" sodipodi:docname="full-diagram.svg" inkscape:version="0.92.2pre0 (973e216, 2017-07-25)" width="734.41882" height="358.09845" style="fill:none;stroke:none;stroke-linecap:square;stroke-miterlimit:10"> <title>Same diagram as before, duplicated, with the bottom line missing one person in the database. A double arrow labeled "basically the same" points to the two outputs.</title> <desc>Same diagram as before, duplicated, with the bottom line missing one person in the database. A double arrow labeled "basically the same" points to the two outputs.</desc> <metadata id="metadata4129"> <rdf:RDF> <cc:Work rdf:about=""> <dc:format>image/svg+xml</dc:format> <dc:type rdf:resource="http://purl.org/dc/dcmitype/StillImage" /> <dc:title></dc:title> </cc:Work> </rdf:RDF> </metadata> <defs id="defs4127"> <clipPath id="p.3-4"> <path d="M 0,0 H 275 V 252 H 0 Z" id="path4077-0" inkscape:connector-curvature="0" style="clip-rule:evenodd" /> </clipPath> </defs> <sodipodi:namedview pagecolor="#ffffff" bordercolor="#666666" borderopacity="1" objecttolerance="10" gridtolerance="10" guidetolerance="10" inkscape:pageopacity="0" inkscape:pageshadow="2" inkscape:window-width="1918" inkscape:window-height="1078" id="namedview4125" showgrid="false" inkscape:zoom="1.3111111" inkscape:cx="532.33439" inkscape:cy="127.30279" inkscape:window-x="0" inkscape:window-y="0" inkscape:window-maximized="0" inkscape:current-layer="g4121" fit-margin-top="0" fit-margin-left="0" fit-margin-right="0" fit-margin-bottom="0" viewbox-x="82" viewbox-width="650" /> <clipPath id="p.0"> <path d="M 0,0 H 960 V 720 H 0 Z" id="path4036" inkscape:connector-curvature="0" style="clip-rule:nonzero" /> </clipPath> <g clip-path="url(#p.0)" id="g4121" transform="translate(-76.086586,-124.34907)"> <path d="M 0,0 H 960 V 720 H 0 Z" id="path4039" inkscape:connector-curvature="0" style="fill:#000000;fill-opacity:0;fill-rule:evenodd" /> <path d="m 76.08921,124.34908 h 223.2992 V 267.11813 H 76.08921 Z" id="path4041" inkscape:connector-curvature="0" style="fill:#000000;fill-opacity:0;fill-rule:evenodd" /> <g transform="matrix(0.22307612,0,0,0.22307664,76.089214,124.34908)" id="g4048"> <clipPath id="p.1"> <path d="M 0,0 H 1001 V 640 H 0 Z" id="path4043" inkscape:connector-curvature="0" style="clip-rule:evenodd" /> </clipPath> <image clip-path="url(#p.1)" width="1001" height="640" x="0" y="0" preserveAspectRatio="none" xlink:href="https://desfontain.es/privacy/images/quiestce.jpeg" id="image4046" style="fill:#000000" /> </g> <path d="M 76.086586,339.67847 H 299.38579 V 482.44751 H 76.086586 Z" id="path4050" inkscape:connector-curvature="0" style="fill:#000000;fill-opacity:0;fill-rule:evenodd" /> <g transform="matrix(0.22307612,0,0,0.22307664,76.086589,339.67847)" id="g4057"> <clipPath id="p.2"> <path d="M 0,0 H 1001 V 640 H 0 Z" id="path4052" inkscape:connector-curvature="0" style="clip-rule:evenodd" /> </clipPath> <image clip-path="url(#p.2)" width="1001" height="640" x="0" y="0" preserveAspectRatio="none" xlink:href="https://desfontain.es/privacy/images/quiestce.jpeg" id="image4055" style="fill:#000000" /> </g> <path d="m 604.81974,178.70273 v 0 c -1.51233,-10.41394 3.45307,-20.72311 12.78925,-26.55296 9.33618,-5.82984 21.40576,-6.15791 31.08722,-0.84502 v 0 c 3.42944,-6.0551 9.70636,-10.23573 16.93207,-11.27733 7.2257,-1.04159 14.55151,1.17818 19.76159,5.98789 v 0 c 2.92145,-5.48995 8.65778,-9.17848 15.1734,-9.75675 6.51569,-0.57824 12.88855,2.03563 16.85718,6.9141 v 0 c 5.27802,-5.81934 13.67554,-8.2695 21.55884,-6.29033 7.8833,1.97917 13.83649,8.03222 15.28363,15.53996 v 0 c 6.46649,1.65271 11.8529,5.85413 14.76764,11.51872 2.91473,5.66462 3.07178,12.23673 0.43066,18.01831 v 0 c 6.36756,7.76532 7.85712,18.11251 3.91272,27.18018 -3.9444,9.06767 -12.7301,15.49351 -23.07849,16.87953 -0.0729,8.51033 -5.05414,16.31931 -13.02356,20.41701 -7.96942,4.09772 -17.68268,3.84427 -25.39587,-0.66263 -3.2854,10.1926 -12.53272,17.6922 -23.74671,19.25873 -11.21405,1.56653 -22.38446,-3.08087 -28.68524,-11.93434 -7.72339,4.3639 -16.99084,5.621 -25.71185,3.48773 -8.72095,-2.13329 -16.16077,-7.47725 -20.64124,-14.8264 v 0 c -7.89233,0.86539 -15.52319,-2.96604 -19.10534,-9.59276 -3.58216,-6.62671 -2.35303,-14.63803 3.07733,-20.05798 v 0 c -7.04022,-3.8826 -10.63251,-11.58696 -8.90375,-19.09555 1.72882,-7.50861 8.38696,-13.11998 16.50256,-13.90799 z" id="path4059" inkscape:connector-curvature="0" style="fill:#f6cd4c;fill-rule:evenodd" /> <path d="m 597.06497,212.10764 v 0 c 3.32227,1.83222 7.16028,2.66335 10.99872,2.38181 m 5.02759,27.2696 c 1.65063,-0.18098 3.26861,-0.56427 4.81219,-1.13993 m 41.53858,12.47812 c -1.16095,-1.63134 -2.133,-3.37456 -2.89954,-5.19996 m 55.3335,-2.12474 v 0 c 0.59893,-1.85826 0.98706,-3.77081 1.15777,-5.70575 m 37.26019,-14.04791 c 0.0776,-9.06056 -5.41461,-17.3565 -14.11767,-21.32448 m 33.2832,-22.73425 c -1.40942,3.08528 -3.56109,5.8222 -6.28625,7.99617 m -8.91071,-37.53403 v 0 c 0.24011,1.24591 0.35126,2.51055 0.33185,3.77639 m -37.17371,-13.02565 v 0 c -1.31659,1.45164 -2.4013,3.07382 -3.22033,4.81604 m -28.81055,-1.97402 v 0 c -0.70166,1.31851 -1.22559,2.71372 -1.55975,4.15347 m -35.13446,1.13608 v 0 c 2.04858,1.12421 3.94384,2.47731 5.64404,4.02959 m -49.5199,23.36855 v 0 c 0.20844,1.4353 0.53784,2.85292 0.98499,4.23957" id="path4061" inkscape:connector-curvature="0" style="fill:#000000;fill-opacity:0;fill-rule:evenodd" /> <path d="m 636.30257,211.8428 h -0.1875 q -0.78125,0 -1.25,-0.48437 -0.46875,-0.48438 -0.46875,-1.17188 0,-0.46875 0.5,-2.92187 l 1.375,-7.10938 q 0.46875,-2.42187 2.0625,-13.4375 l 0.39063,-2.70312 q 0.14062,-1.01563 1.10937,-1.84375 0.96875,-0.84375 1.78125,-0.84375 0.4375,0 2.9375,1.3125 2.70313,1.45312 3.20313,1.59375 5.09375,1.875 8.375,5.6875 3.29687,3.8125 3.29687,8.23437 0,4.04688 -2.40625,7.92188 -2.39062,3.875 -5.78125,5.71875 -3.39062,1.84375 -8.67187,1.84375 -1.53125,0 -3.46875,-0.53125 -1.9375,-0.51563 -2.79688,-1.26563 z m 6,-25.40625 -1.64062,11.20313 -2,10.67187 q 0.0625,0.0469 0.14062,0.0781 1.84375,1.32812 4.375,1.32812 5.01563,0 7.375,-1.79687 2.35938,-1.79688 3.60938,-4.21875 1.25,-2.42188 1.25,-4.92188 0,-2.51562 -1.40625,-4.64062 -1.39063,-2.125 -3.76563,-3.57813 -2.35937,-1.46875 -7.9375,-4.125 z m 32.16406,24.28125 q -4.21875,2.03125 -6.01562,2.03125 -7.29688,0 -7.29688,-6.95312 0,-6.46875 4.07813,-10.46875 4.09375,-4 8.98437,-4 1.89063,0 3.90625,0.96875 2.03125,0.96875 2.03125,2.25 0,0.79687 -0.53125,1.25 -0.39062,0.76562 -1.10937,4.40625 -0.70313,3.64062 -0.70313,5.25 0,1.96875 1.01563,5.42187 l 0.0781,0.3125 q -0.4375,1.82813 -2.17187,1.82813 -0.29688,0 -1.09375,-0.79688 -0.79688,-0.79687 -1.17188,-1.5 z m 1.42188,-15.1875 q -1.21875,-0.73437 -1.95313,-0.73437 -3.25,0 -6.07812,2.95312 -2.82813,2.9375 -2.82813,7.3125 0,4.29688 3.71875,4.29688 2.90625,0 5.46875,-2.15625 0.3125,-8.1875 1.67188,-11.67188 z m 16.45312,-0.34375 q -0.23437,0.9375 -0.60937,2.1875 -1.64063,5.42188 -1.64063,8.26563 0,3.35937 1.5625,3.35937 1.54688,0 2.28125,-0.95312 1.01563,-1.32813 1.875,-1.32813 0.65625,0 1.25,0.48438 0.59375,0.48437 0.59375,1.26562 0,1.71875 -2.15625,2.95313 -2.15625,1.21875 -4.29687,1.21875 -5.03125,0 -5.03125,-6.28125 0,-3.4375 1.54687,-8.39063 0.40625,-1.3125 0.78125,-2.625 -4.25,-0.20312 -4.85937,-0.70312 -0.60938,-0.5 -0.60938,-1.5625 0,-0.85938 0.57813,-1.42188 0.59375,-0.57812 1.60937,-0.57812 l 4.03125,0.32812 q 0.125,-0.51562 0.1875,-1.17187 0.0312,-0.67188 0.125,-1.48438 l 0.34375,-2.42187 q 0.10938,-0.79688 0.71875,-1.34375 0.625,-0.54688 1.46875,-0.54688 1.64063,0 1.64063,2.625 0,0.76563 -0.23438,1.98438 l -0.39062,2.35937 q 2.60937,-0.32812 2.78125,-0.32812 2.03125,0 2.79687,0.40625 0.76563,0.39062 0.76563,1.46875 0,0.95312 -0.65625,1.54687 -0.65625,0.57813 -1.57813,0.57813 l -1.92187,-0.0781 q -1.45313,0 -2.95313,0.1875 z m 21.44141,15.53125 q -4.21875,2.03125 -6.01563,2.03125 -7.29687,0 -7.29687,-6.95312 0,-6.46875 4.07812,-10.46875 4.09375,-4 8.98438,-4 1.89062,0 3.90625,0.96875 2.03125,0.96875 2.03125,2.25 0,0.79687 -0.53125,1.25 -0.39063,0.76562 -1.10938,4.40625 -0.70312,3.64062 -0.70312,5.25 0,1.96875 1.01562,5.42187 l 0.0781,0.3125 q -0.4375,1.82813 -2.17188,1.82813 -0.29687,0 -1.09375,-0.79688 -0.79687,-0.79687 -1.17187,-1.5 z m 1.42187,-15.1875 q -1.21875,-0.73437 -1.95312,-0.73437 -3.25,0 -6.07813,2.95312 -2.82812,2.9375 -2.82812,7.3125 0,4.29688 3.71875,4.29688 2.90625,0 5.46875,-2.15625 0.3125,-8.1875 1.67187,-11.67188 z" id="path4063" inkscape:connector-curvature="0" style="fill:#000000;fill-rule:nonzero" /> <path d="m 604.81711,394.03214 v 0 c -1.51233,-10.41397 3.45307,-20.72311 12.78925,-26.55298 9.33618,-5.82983 21.40576,-6.1579 31.08722,-0.845 v 0 c 3.42944,-6.05511 9.70636,-10.23575 16.93207,-11.27734 7.2257,-1.0416 14.55151,1.17819 19.76159,5.98788 v 0 c 2.92145,-5.48993 8.65778,-9.17847 15.1734,-9.75674 6.51569,-0.57825 12.88855,2.03564 16.85718,6.91409 v 0 c 5.27802,-5.81934 13.67554,-8.2695 21.55884,-6.29031 7.8833,1.97915 13.83649,8.03222 15.28363,15.53994 v 0 c 6.46649,1.65271 11.8529,5.85413 14.76764,11.51874 2.91473,5.66461 3.07178,12.23673 0.43066,18.01831 v 0 c 6.36756,7.76532 7.85712,18.11249 3.91272,27.18018 -3.9444,9.06766 -12.7301,15.4935 -23.07849,16.87952 -0.0729,8.51034 -5.05414,16.3193 -13.02356,20.41702 -7.96942,4.09769 -17.68268,3.84427 -25.39587,-0.66266 -3.2854,10.19263 -12.53272,17.69223 -23.74671,19.25876 -11.21405,1.56653 -22.38446,-3.08087 -28.68524,-11.93436 -7.72339,4.36392 -16.99084,5.621 -25.71185,3.48773 -8.72095,-2.13327 -16.16077,-7.47723 -20.64124,-14.82638 v 0 c -7.89233,0.86539 -15.52319,-2.96607 -19.10534,-9.59277 -3.58216,-6.62671 -2.35303,-14.63804 3.07733,-20.05799 v 0 c -7.04022,-3.8826 -10.63257,-11.58694 -8.90375,-19.09555 1.72882,-7.5086 8.38696,-13.11996 16.50256,-13.90799 z" id="path4065" inkscape:connector-curvature="0" style="fill:#f6cd4c;fill-rule:evenodd" /> <path d="m 597.06237,427.43705 v 0 c 3.32227,1.83218 7.16028,2.66333 10.99872,2.38181 m 5.02759,27.26959 c 1.65063,-0.181 3.26861,-0.56427 4.81219,-1.13992 m 41.53858,12.47811 c -1.16095,-1.63134 -2.133,-3.37457 -2.89954,-5.19998 m 55.3335,-2.12472 v 0 c 0.59893,-1.85825 0.98706,-3.77082 1.15777,-5.70575 m 37.26019,-14.04792 c 0.0776,-9.06054 -5.41461,-17.3565 -14.11767,-21.32449 m 33.2832,-22.73425 c -1.40942,3.0853 -3.56109,5.82223 -6.28625,7.99618 m -8.91071,-37.53402 v 0 c 0.24011,1.24591 0.35126,2.51056 0.33185,3.77639 m -37.17371,-13.02566 v 0 c -1.31659,1.45163 -2.4013,3.07382 -3.22033,4.81604 m -28.81055,-1.974 v 0 c -0.70166,1.31848 -1.22559,2.71372 -1.55975,4.15347 m -35.13446,1.13608 v 0 c 2.04858,1.12421 3.94384,2.4773 5.64404,4.02957 m -49.5199,23.36856 v 0 c 0.20844,1.43531 0.53784,2.85291 0.98499,4.23957" id="path4067" inkscape:connector-curvature="0" style="fill:#000000;fill-opacity:0;fill-rule:evenodd" /> <path d="m 636.29997,427.17219 h -0.1875 q -0.78125,0 -1.25,-0.48437 -0.46875,-0.48438 -0.46875,-1.17188 0,-0.46875 0.5,-2.92187 l 1.375,-7.10938 q 0.46875,-2.42187 2.0625,-13.4375 l 0.39063,-2.70312 q 0.14062,-1.01563 1.10937,-1.84375 0.96875,-0.84375 1.78125,-0.84375 0.4375,0 2.9375,1.3125 2.70313,1.45312 3.20313,1.59375 5.09375,1.875 8.375,5.6875 3.29687,3.8125 3.29687,8.23437 0,4.04688 -2.40625,7.92188 -2.39062,3.875 -5.78125,5.71875 -3.39062,1.84375 -8.67187,1.84375 -1.53125,0 -3.46875,-0.53125 -1.9375,-0.51563 -2.79688,-1.26563 z m 6,-25.40625 -1.64062,11.20313 -2,10.67187 q 0.0625,0.0469 0.14062,0.0781 1.84375,1.32812 4.375,1.32812 5.01563,0 7.375,-1.79687 2.35938,-1.79688 3.60938,-4.21875 1.25,-2.42188 1.25,-4.92188 0,-2.51562 -1.40625,-4.64062 -1.39063,-2.125 -3.76563,-3.57813 -2.35937,-1.46875 -7.9375,-4.125 z m 32.16406,24.28125 q -4.21875,2.03125 -6.01562,2.03125 -7.29688,0 -7.29688,-6.95312 0,-6.46875 4.07813,-10.46875 4.09375,-4 8.98437,-4 1.89063,0 3.90625,0.96875 2.03125,0.96875 2.03125,2.25 0,0.79687 -0.53125,1.25 -0.39062,0.76562 -1.10937,4.40625 -0.70313,3.64062 -0.70313,5.25 0,1.96875 1.01563,5.42187 l 0.0781,0.3125 q -0.4375,1.82813 -2.17187,1.82813 -0.29688,0 -1.09375,-0.79688 -0.79688,-0.79687 -1.17188,-1.5 z m 1.42188,-15.1875 q -1.21875,-0.73437 -1.95313,-0.73437 -3.25,0 -6.07812,2.95312 -2.82813,2.9375 -2.82813,7.3125 0,4.29688 3.71875,4.29688 2.90625,0 5.46875,-2.15625 0.3125,-8.1875 1.67188,-11.67188 z m 16.45312,-0.34375 q -0.23437,0.9375 -0.60937,2.1875 -1.64063,5.42188 -1.64063,8.26563 0,3.35937 1.5625,3.35937 1.54688,0 2.28125,-0.95312 1.01563,-1.32813 1.875,-1.32813 0.65625,0 1.25,0.48438 0.59375,0.48437 0.59375,1.26562 0,1.71875 -2.15625,2.95313 -2.15625,1.21875 -4.29687,1.21875 -5.03125,0 -5.03125,-6.28125 0,-3.4375 1.54687,-8.39063 0.40625,-1.3125 0.78125,-2.625 -4.25,-0.20312 -4.85937,-0.70312 -0.60938,-0.5 -0.60938,-1.5625 0,-0.85938 0.57813,-1.42188 0.59375,-0.57812 1.60937,-0.57812 l 4.03125,0.32812 q 0.125,-0.51562 0.1875,-1.17187 0.0312,-0.67188 0.125,-1.48438 l 0.34375,-2.42187 q 0.10938,-0.79688 0.71875,-1.34375 0.625,-0.54688 1.46875,-0.54688 1.64063,0 1.64063,2.625 0,0.76563 -0.23438,1.98438 l -0.39062,2.35937 q 2.60937,-0.32812 2.78125,-0.32812 2.03125,0 2.79687,0.40625 0.76563,0.39062 0.76563,1.46875 0,0.95312 -0.65625,1.54687 -0.65625,0.57813 -1.57813,0.57813 l -1.92187,-0.0781 q -1.45313,0 -2.95313,0.1875 z m 21.44141,15.53125 q -4.21875,2.03125 -6.01563,2.03125 -7.29687,0 -7.29687,-6.95312 0,-6.46875 4.07812,-10.46875 4.09375,-4 8.98438,-4 1.89062,0 3.90625,0.96875 2.03125,0.96875 2.03125,2.25 0,0.79687 -0.53125,1.25 -0.39063,0.76562 -1.10938,4.40625 -0.70312,3.64062 -0.70312,5.25 0,1.96875 1.01562,5.42187 l 0.0781,0.3125 q -0.4375,1.82813 -2.17188,1.82813 -0.29687,0 -1.09375,-0.79688 -0.79687,-0.79687 -1.17187,-1.5 z m 1.42187,-15.1875 q -1.21875,-0.73437 -1.95312,-0.73437 -3.25,0 -6.07813,2.95312 -2.82812,2.9375 -2.82812,7.3125 0,4.29688 3.71875,4.29688 2.90625,0 5.46875,-2.15625 0.3125,-8.1875 1.67187,-11.67188 z" id="path4069" inkscape:connector-curvature="0" style="fill:#000000;fill-rule:nonzero" /> <path d="m 299.38846,189.04987 h 70.36065 v -6.68504 l 8.40525,13.37008 -8.40525,13.37007 v -6.68502 h -70.36065 z" id="path4071" inkscape:connector-curvature="0" style="fill:#7c7ce0;fill-rule:evenodd;stroke-width:0.79288208" /> <path d="M 427.88773,353.66272 H 553.16596 V 468.46329 H 427.88773 Z" id="path4075" inkscape:connector-curvature="0" style="fill:#000000;fill-opacity:0;fill-rule:evenodd" /> <g transform="matrix(0.45555722,0,0,0.45555774,379.83689,353.66273)" id="g4082"> <clipPath id="p.3"> <path d="M 0,0 H 275 V 252 H 0 Z" id="path4077" inkscape:connector-curvature="0" style="clip-rule:evenodd" /> </clipPath> <image clip-path="url(#p.3)" width="275" height="252" x="0" y="0" preserveAspectRatio="none" xlink:href="https://desfontain.es/privacy/images/magic.gif" id="image4080" style="fill:#000000" /> </g> <path d="m 660.00147,264.20093 21.66931,-21.66928 21.66932,21.66928 h -10.83466 v 85.7638 h 10.83466 l -21.66932,21.66928 -21.66931,-21.66928 h 10.83466 v -85.7638 z" id="path4088" inkscape:connector-curvature="0" style="fill:#db4437;fill-rule:evenodd" /> <path d="m 660.00147,264.20093 21.66931,-21.66928 21.66932,21.66928 h -10.83466 v 85.7638 h 10.83466 l -21.66932,21.66928 -21.66931,-21.66928 h 10.83466 v -85.7638 z" id="path4090" inkscape:connector-curvature="0" style="fill-rule:evenodd;stroke:#666666;stroke-width:1;stroke-linecap:butt;stroke-linejoin:round" /> <path d="m 554.53672,285.04017 c 42.66144,-12.09317 85.32288,12.0932 127.98438,0 42.66143,-12.09317 85.32287,12.0932 127.98431,0 l -1.70062,50.79135 c -42.66144,12.09317 -85.32288,-12.09317 -127.98432,0 -42.66143,12.09317 -85.32294,-12.09317 -127.98437,0 z" id="path4092" inkscape:connector-curvature="0" style="fill:#db4437;fill-rule:evenodd" /> <path d="m 587.72404,321.45273 q -0.84375,-0.5 -1.26562,-1.0625 -0.40625,-0.5625 -0.40625,-1.07812 0,-0.67188 0.67187,-1.07813 0.125,-0.0625 0.35938,-0.0625 0.3125,0 0.67187,0.10938 0.375,0.10937 0.57813,0.29687 0.82812,0.76563 1.98437,0.76563 0.5,0 1.46875,-0.21875 1.25,-0.39063 1.96875,-0.95313 0.71875,-0.57812 1.70313,-1.79687 0.60937,-0.79688 0.90625,-1.60938 0.3125,-0.82812 0.3125,-1.5 0,-0.73437 -0.34375,-1.15625 -0.64063,-0.85937 -1.4375,-1.28125 -0.78125,-0.4375 -1.82813,-0.4375 -0.9375,0 -1.82812,0.25 l -1.85938,0.39063 q -0.125,0.0312 -0.34375,0.0469 -0.20312,0.0156 -0.59375,0.95313 -0.375,0.92187 -1.29687,3.51562 -0.67188,1.875 -1.125,2.70313 -0.45313,0.8125 -0.79688,0.8125 -0.39062,0 -0.82812,-0.70313 -0.0937,-0.15625 -0.40625,-0.64062 -0.29688,-0.48438 -0.29688,-0.9375 0,-0.21875 0.15625,-0.53125 0.15625,-0.29688 0.26563,-0.57813 0.10937,-0.29687 0.25,-0.79687 0.40625,-1.25 2.40625,-6.20313 2,-4.96875 3.125,-7.5 0.79687,-1.82812 0.48437,-1.92187 0,-0.0937 -0.10937,-0.28125 -0.0937,-0.20313 -0.0625,-0.29688 0,-0.1875 1.14062,-0.5625 1.14063,-0.39062 1.54688,-0.32812 h 0.20312 q 0.40625,0 0.40625,-0.15625 0.0625,0 0.59375,-0.0625 0.53125,-0.0625 1.14063,0 2.26562,0.1875 3.57812,1.29687 1.3125,1.09375 1.28125,3.23438 -0.0625,0.84375 -0.92187,2.28125 -0.85938,1.4375 -1.89063,2.35937 -0.51562,0.45313 -1.17187,0.95313 -0.65625,0.5 -0.84375,0.625 l 0.79687,0.28125 q 1.5,0.57812 2.32813,1.42187 0.84375,0.82813 1.1875,2.26563 0.0781,0.3125 0.0781,0.57812 0,0.70313 -0.25,1.59375 -0.23437,0.89063 -0.53125,1.5625 -0.1875,0.20313 -0.3125,0.42188 -0.125,0.21875 -0.15625,0.28125 -0.0312,0.14062 -0.57812,0.98437 -0.54688,0.84375 -0.96875,1.35938 -2.10938,1.79687 -3.48438,2.29687 -1.34375,0.60938 -3.20312,0.60938 -1.65625,0 -2.45313,-0.51563 z m 2.29688,-13.26562 q 0.32812,-0.0781 1.28125,-0.28125 0.95312,-0.20313 1.375,-0.35938 0.0625,-0.0312 0.39062,-0.20312 0.34375,-0.1875 0.70313,-0.54688 2.01562,-1.875 2.8125,-3.39062 0.79687,-1.3125 0.79687,-1.84375 0,-0.39063 -0.34375,-0.75 -0.60937,-0.70313 -2.01562,-0.70313 -0.60938,0 -1.03125,0.14063 -0.45313,0.125 -0.65625,0.29687 -0.20313,0.17188 -0.20313,0.53125 0,0.25 -0.17187,0.5625 -0.15625,0.29688 -0.34375,0.48438 -0.0312,0.20312 -0.17188,0.39062 -0.14062,0.1875 -0.21875,0.25 0,0.0781 -0.4375,0.875 -0.42187,0.79688 -0.79687,1.78125 -0.96875,2.28125 -0.96875,2.76563 z m 11.90643,10.29687 q -0.0937,-0.1875 -0.54688,-0.5 -0.45312,-0.32812 -0.45312,-0.39062 0,-0.14063 -0.14063,-0.375 -0.14062,-0.23438 -0.26562,-0.4375 -0.26563,-0.25 0.34375,-1.65625 0.60937,-1.40625 1.64062,-3.32813 0.1875,-0.1875 0.48438,-0.67187 0.3125,-0.48438 0.40625,-0.54688 0,-0.125 0.0937,-0.23437 0.0937,-0.10938 0.23437,-0.10938 l 0.25,-0.51562 q 0.125,-0.29688 1.25,-1.375 1.125,-1.09375 1.76563,-1.46875 0.59375,-0.42188 1.3125,-0.42188 0.76562,0 1.29687,0.45313 l 0.875,0.73437 q 0.95313,0.79688 1.07813,0.92188 l 1.1875,1.0625 -0.54688,1.59375 q -0.28125,0.85937 -0.28125,1.64062 0,1.20313 0.67188,2.26563 0.15625,0.34375 0.42187,0.54687 0.28125,0.1875 0.73438,0.25 0.70312,0.125 0.70312,0.70313 0,0.70312 -0.26562,1.0625 -0.25,0.34375 -0.60938,0.4375 -0.5,0.0781 -0.70312,0.0781 -0.92188,0 -1.6875,-0.48437 -0.76563,-0.48438 -1.34375,-1.64063 -0.125,-0.1875 -0.34375,-0.6875 -0.20313,-0.51562 -0.23438,-0.71875 -0.21875,0.32813 -0.73437,0.84375 -3.29688,3.32813 -5.34375,3.32813 -0.70313,0 -1.25,-0.35938 z m 1.76562,-2.65625 q 0.5,0.0625 1.98438,-1.15625 0.15625,-0.125 0.34375,-0.29687 0.1875,-0.17188 0.42187,-0.375 0.89063,-0.73438 1.39063,-1.21875 0.5,-0.5 1,-1.20313 l 0.70312,-1.15625 q -0.3125,-0.70312 -0.57812,-0.9375 -0.25,-0.25 -0.5,-0.25 -0.64063,0 -1.60938,1.14063 -0.95312,1.14062 -2.29687,3.4375 -0.35938,0.57812 -0.57813,1.20312 -0.21875,0.625 -0.28125,0.8125 z m 14.94391,3.8125 q -0.78125,0 -1.59375,-0.32812 -0.8125,-0.3125 -1.375,-0.78125 -0.5625,-0.46875 -0.5625,-0.84375 0,-0.39063 0.35938,-0.48438 l 0.15625,-0.125 q 0.0625,-0.0312 0.25,-0.0312 0.14062,-0.0312 0.32812,0.0156 0.1875,0.0469 0.3125,0.0469 0.23438,0.0625 0.67188,0.125 l 0.84375,0.0937 q 0.64062,0 1.1875,-0.125 0.5625,-0.125 0.5625,-0.28125 0,-0.10937 -0.25,-0.32812 -0.25,-0.21875 -0.54688,-0.21875 0,-0.125 -0.98437,-0.85938 -0.54688,-0.42187 -1.04688,-0.82812 -0.5,-0.42188 -0.5,-0.625 l 0.0312,-0.0625 v -0.15625 q -0.21875,0 -0.42187,-0.60938 -0.1875,-0.60937 -0.1875,-1.1875 0,-1.07812 0.95312,-2.39062 0.96875,-1.3125 2.39063,-2.35938 1.4375,-1.0625 2.71875,-1.45312 0.46875,-0.21875 1.40625,-0.21875 0.60937,0 0.82812,0.125 l 0.25,-0.0625 q 0.39063,0 1,0.64062 0.60938,0.64063 0.76563,1.01563 0,1.0625 -0.0625,1.70312 -0.0625,0.64063 -0.32813,0.64063 -0.0312,0 -0.0937,0.0312 -0.0625,0.0312 -0.0625,0.15625 0,0.45312 -0.78125,1 -0.78125,0.53125 -1.42187,0.53125 l -0.125,-0.0312 q -0.20313,-0.0312 -0.57813,-0.375 -0.35937,-0.35938 -0.35937,-0.70313 0,-0.0937 0.375,-0.51562 0.39062,-0.42188 0.65625,-0.8125 0.28125,-0.40625 0.28125,-0.85938 0,-0.15625 -0.10938,-0.20312 -0.10937,-0.0469 -0.4375,-0.0469 -1.75,0.79688 -2.51562,1.5 -0.10938,0.20313 -0.1875,0.29688 -0.0781,0.0937 -0.20313,0.0937 0,0 0,-0.0156 0,-0.0156 -0.0312,-0.0156 -0.0625,0 -0.42187,0.375 -0.34375,0.35937 -0.65625,0.84375 -0.29688,0.46875 -0.35938,0.89062 l -0.0625,0.0937 v 0.15625 q -0.0312,0.0625 -0.0312,0.26563 0,0.28125 0.25,0.59375 0.25,0.29687 1.03125,0.84375 1.4375,1.09375 1.95313,1.625 0.53125,0.51562 0.67187,0.9375 0.28125,0.57812 0.28125,1.01562 0,1.09375 -1.14062,1.6875 -1.125,0.59375 -3.07813,0.59375 z m 13.21564,-13.79687 q -0.10937,0.0937 -0.20312,0.0937 -0.125,0 -0.40625,-0.1875 -0.32813,-0.0312 -0.85938,-0.54688 -0.51562,-0.51562 -0.51562,-0.67187 -0.0312,-0.39063 0.0625,-1.10938 0.0937,-0.71875 0.35937,-1.28125 0.28125,-0.57812 0.71875,-0.60937 0.23438,0.0312 0.60938,0.14062 0.39062,0.10938 0.875,0.29688 0.59375,1.21875 0.59375,2.07812 0,1.28125 -1.23438,1.79688 z m -2.95312,13.28125 q -0.76563,0 -1.32813,-0.79688 -0.5625,-0.79687 -0.6875,-1.98437 0.125,-1.1875 0.1875,-1.73438 0.35938,-1.5625 0.89063,-3.6875 0.53125,-2.14062 0.625,-2.35937 0.15625,-0.28125 0.5,-0.28125 0.35937,0 0.78125,0.26562 0.4375,0.26563 0.73437,0.75 0.25,0.35938 0.25,0.79688 0,0.67187 -0.64062,2.82812 -0.4375,1.59375 -0.57813,2.26563 -0.125,0.67187 -0.0937,1.4375 -0.125,1.4375 -0.25,1.98437 -0.125,0.54688 -0.39062,0.51563 z m 6.8189,-0.21875 q -0.95312,0 -1.25,-0.10938 -0.28125,-0.125 -0.70312,-0.5 -0.0937,-0.23437 -0.40625,-0.60937 -0.29688,-0.39063 -0.48438,-0.57813 -0.0937,-0.35937 -0.0937,-0.82812 0,-0.84375 0.23438,-1.95313 0.23437,-1.125 0.59375,-2.04687 0.125,-0.26563 0.20312,-0.48438 0.0781,-0.21875 0.10938,-0.28125 0,-0.23437 0.8125,-1.39062 0.82812,-1.17188 1.3125,-1.625 0.21875,-0.25 0.67187,-0.53125 0.46875,-0.29688 0.82813,-0.42188 0.0937,-0.0312 0.78125,-0.28125 0.6875,-0.26562 1.26562,-0.26562 0.51563,0 1,0.20312 0.5,0.20313 0.82813,0.65625 0.82812,0.82813 0.82812,2.17188 0,0.26562 -0.0625,0.57812 -0.17187,0.64063 -0.67187,1.04688 -0.48438,0.39062 -1.03125,0.39062 -0.54688,0 -0.92188,-0.40625 -0.125,-0.32812 -0.125,-0.42187 0,-0.0937 0.125,-0.54688 0.125,-0.57812 0.125,-0.73437 0,-0.40625 -0.28125,-0.40625 -0.39062,0 -0.625,0.1875 -0.21875,0.1875 -0.82812,0.82812 -0.40625,0.45313 -0.96875,1.29688 -0.5625,0.84375 -0.875,1.60937 -0.29688,0.875 -0.40625,1.25 -0.10938,0.35938 -0.10938,0.42188 0,0.125 -0.0312,0.23437 -0.0312,0.10938 -0.0312,0.25 0,0.21875 0.0937,0.34375 0.0937,0.125 0.34375,0.26563 0.0781,0.0312 0.23438,0.125 0.15625,0.0937 0.3125,0.0937 0.20312,0 0.54687,-0.15625 0.35938,-0.17188 0.45313,-0.20313 0.40625,-0.1875 1.17187,-0.82812 l 0.78125,-0.57813 q 0.21875,-0.15625 0.39063,-0.26562 0.1875,-0.10938 0.28125,-0.15625 0.0937,-0.0469 0.0937,0.0156 0,0.125 0.0937,0.125 0.0937,0 0.21875,-0.125 0.125,-0.20312 0.45312,-0.20312 0.25,0 0.51563,0.15625 0.28125,0.14062 0.28125,0.23437 l -0.0312,0.15625 q 0,0.15625 0.15625,0.32813 0,0.125 -0.0781,0.32812 -0.0781,0.20313 -0.20312,0.34375 -0.32813,0.46875 -0.73438,0.98438 -0.39062,0.51562 -0.54687,0.70312 h -0.0312 q -0.15625,0 -0.45313,0.29688 -0.46875,0.4375 -0.82812,0.4375 -0.125,0.0625 -0.29688,0.15625 -0.15625,0.0781 -0.15625,0.17187 -0.53125,0.54688 -2.875,0.54688 z m 10.59815,-0.32813 q -0.54688,0.125 -1.3125,-0.57812 -0.76563,-0.70313 -0.90625,-1.3125 -0.0625,-0.3125 -0.0625,-0.54688 0,-1.23437 0.79687,-3.01562 0.8125,-1.78125 2.17188,-3.34375 1.35937,-1.5625 2.98437,-2.26563 0.67188,-0.35937 0.85938,-0.35937 0.20312,0 0.875,0.35937 0.64062,0.15625 1.01562,0.34375 0.39063,0.1875 0.70313,0.60938 0.42187,0.82812 0.60937,2.04687 0.20313,0.67188 0.20313,0.95313 0,0.26562 -0.20313,0.45312 -0.0312,0.0312 -0.10937,0.125 -0.0781,0.0937 -0.0781,0.15625 0,0.0469 0.0625,0.0469 0,0.0312 -0.0312,0.14062 -0.0312,0.10938 -0.125,0.29688 -0.39062,1.21875 -0.39062,1.89062 0,0.21875 0.0625,0.54688 0.32812,0.82812 0.8125,0.82812 0.4375,0 1.01562,-0.57812 0.48438,-0.3125 0.96875,-0.3125 0.28125,0 0.28125,0.15625 0.0312,0.0312 0.125,0.0937 0.0937,0.0625 0.1875,0.0625 0.14063,0 0.14063,0.29687 0,0.25 -0.125,0.67188 -0.10938,0.40625 -0.26563,0.67187 -0.21875,0.21875 -0.35937,0.39063 -0.125,0.15625 -0.125,0.25 l -0.46875,0.4375 q -0.60938,0.35937 -1.45313,0.35937 -0.85937,0 -1.375,-0.35937 -0.5,-0.34375 -0.95312,-0.875 -0.45313,-0.53125 -0.57813,-0.9375 l -0.15625,-0.32813 -0.9375,0.60938 -0.79687,0.82812 q -0.25,0.32813 -0.4375,0.32813 h -0.32813 q 0,0.21875 -0.90625,0.54687 -0.89062,0.3125 -1.39062,0.3125 z m 0.67187,-2.6875 q 0.34375,0 1.4375,-0.8125 1.10938,-0.8125 1.5,-1.29687 0.125,-0.15625 0.32813,-0.45313 0.21875,-0.3125 0.375,-0.46875 0.51562,-0.60937 0.75,-0.96875 0.25,-0.375 0.25,-0.73437 0,-0.25 -0.23438,-0.70313 -0.125,-0.39062 -0.375,-0.39062 -0.14062,0 -0.73437,0.23437 -0.60938,0.34375 -1.4375,1.4375 -0.8125,1.07813 -1.42188,2.20313 -0.60937,1.125 -0.60937,1.64062 0,0.0937 0.0625,0.1875 0.0781,0.0937 0.10937,0.125 z m 11.06427,4.21875 q -0.48437,-0.15625 -0.71875,-0.57812 -0.23437,-0.40625 -0.23437,-1.20313 0,-1.3125 0.25,-2.5625 l 0.0312,-0.25 q 0.0312,-0.20312 0.0937,-0.625 0.0625,-0.4375 0.0937,-0.25 l 0.20312,-0.85937 q 0.21875,-0.89063 0.42188,-1.65625 0.21875,-0.78125 0.34375,-1.125 l 0.34375,-1.125 q 0.96875,-3.10938 1.79687,-4.1875 0.0312,-0.125 0.0937,-0.23438 0.0625,-0.125 0.0937,-0.25 0,-0.51562 0.20313,-0.79687 0.28125,-0.48438 0.46875,-0.85938 l 0.14062,-0.35937 0.89063,-1.82813 q 0.0937,-0.3125 0.39062,-0.6875 0.3125,-0.39062 0.625,-0.53125 l -0.0312,-0.0312 q 0.17188,-0.25 0.59375,-0.4375 0.4375,-0.20312 0.75,-0.20312 0.42188,0 0.67188,0.26562 0.26562,0.25 0.26562,0.70313 v 0.3125 q 0,0.29687 -0.0469,0.45312 -0.0469,0.15625 -0.21875,0.39063 h 0.0312 q 0.0469,0.0312 -0.0469,0.20312 -0.0781,0.17188 -0.14062,0.32813 -0.21875,0.57812 -0.3125,0.64062 -0.23438,0.29688 -1.48438,2.71875 -1.23437,2.40625 -1.59375,3.32813 l -0.125,0.48437 -0.0625,0.20313 q -0.35937,0.76562 -0.39062,0.82812 l -0.51563,1.98438 -0.125,0.67187 -0.28125,0.73438 -0.1875,0.60937 q -0.0781,0.15625 -0.125,0.42188 -0.0469,0.25 -0.10937,0.3125 l -0.0312,0.15625 q -0.0312,0.125 -0.21875,0.73437 -0.29688,0.90625 -0.42188,1.5 -0.125,0.57813 -0.15625,1.25 l -0.0312,0.14063 q -0.0937,0.70312 -0.40625,1.04687 -0.29687,0.35938 -0.78125,0.21875 z m 6.82099,-0.59375 q -0.0937,0.0312 -0.26563,0.0312 -0.67187,0 -1.125,-0.76562 -0.4375,-0.78125 -0.4375,-1.96875 0,-0.625 0.125,-1.23438 0.35938,-0.9375 0.35938,-1.34375 0.0937,-0.60937 0.65625,-2.1875 0.5625,-1.59375 0.71875,-1.92187 0.125,0 0.21875,-0.20313 0.0937,-0.20312 0.0937,-0.46875 0,-0.28125 0.10937,-0.57812 0.125,-0.3125 0.25,-0.4375 0,-0.0937 0.0781,-0.34375 0.0781,-0.26563 0.0781,-0.35938 0,-0.0625 0.0781,-0.23437 0.0781,-0.1875 0.0781,-0.28125 0.51563,-0.98438 1.79688,-4.45313 1.28125,-3.48437 1.28125,-3.85937 0,-0.26563 0.1875,-0.48438 0.20312,-0.21875 0.45312,-0.21875 0.0937,-0.10937 0.40625,-0.15625 0.32813,-0.0469 0.51563,-0.0469 l 0.51562,0.51562 q 0.35938,0.32813 0.54688,0.57813 0.1875,0.25 0.1875,0.60937 0,0.57813 -0.20313,1.14063 -0.20312,0.54687 -1.20312,2.625 -0.0625,0.17187 -0.51563,1.15625 -0.3125,0.64062 -0.3125,0.73437 0,0.0937 -0.14062,0.40625 -0.125,0.29688 -0.21875,0.625 -0.25,0.51563 -0.8125,1.8125 -0.5625,1.29688 -0.5625,1.42188 0,0.34375 -0.15625,0.5 -0.0937,0 -0.125,0.15625 -0.0312,0.14062 -0.0312,0.20312 0,0.3125 -0.51563,1.70313 l -0.28125,0.92187 q -0.60937,2.07813 -0.85937,3.04688 -0.23438,0.95312 -0.23438,1.625 0,0.70312 -0.23437,1.15625 -0.21875,0.4375 -0.5,0.57812 z m 13.97729,-12.23437 q 0.29688,0.57812 0.29688,1.3125 0,0.9375 -0.42188,1.53125 -0.3125,0.45312 -0.5625,1.35937 -0.23437,0.89063 -0.45312,2.39063 -0.29688,1.21875 -0.71875,3.75 -0.5,1.34375 -0.625,2.01562 -0.0469,0.1875 -0.3125,0.79688 -0.26563,0.60937 -0.4375,0.95312 -1.4375,2.92188 -3.40625,4.39063 -1.96875,1.46875 -4.01563,1.59375 -0.51562,0.0625 -0.60937,0.0625 -0.28125,0 -0.53125,-0.0937 -0.23438,-0.0937 -0.51563,-0.25 -0.64062,-0.45313 -0.73437,-0.64063 -0.17188,-0.32812 -0.17188,-0.64062 0,-0.48438 0.45313,-0.65625 0.45312,-0.17188 1.03125,-0.17188 0.21875,0.125 0.76562,0.125 0.54688,0 0.64063,-0.125 0.0625,-0.0625 0.375,-0.21875 0.32812,-0.14062 0.73437,-0.23437 1.0625,-0.51563 1.625,-1.03125 0.5625,-0.5 1.23438,-1.625 0.79687,-1.21875 1.04687,-2.20313 0.42188,-1.0625 1,-3.14062 l -0.0312,0.0312 q -0.17187,0.1875 -0.8125,0.5625 -0.625,0.375 -0.76562,0.375 -0.0312,0 -0.0937,0.0312 -0.0625,0.0312 -0.0625,0.125 0,0.125 -0.82813,0.40625 -0.82812,0.26562 -1.0625,0.26562 -0.3125,0 -0.6875,-0.34375 -0.35937,-0.35937 -0.53125,-0.65625 -0.125,-0.28125 -0.46875,-0.57812 -0.39062,-0.375 -0.57812,-0.67188 -0.1875,-0.3125 -0.1875,-0.73437 0,-0.21875 0.0312,-0.34375 0.375,-1.4375 1.40625,-4.42188 1.04687,-3 1.53125,-3.73437 0.125,-0.20313 0.46875,-0.32813 0.35937,-0.125 0.71875,-0.125 0.64062,0 0.85937,0.375 0.39063,0.54688 0.39063,0.9375 0,0.4375 -0.45313,1.3125 -0.125,0.34375 -0.32812,0.76563 -0.1875,0.42187 -0.1875,0.51562 -0.0312,0.125 -0.1875,0.78125 -0.15625,0.65625 -0.28125,0.65625 -0.20313,0.1875 -0.4375,1.32813 -0.23438,1.14062 -0.23438,1.8125 0,0.375 0.0937,0.375 0.85938,0 1.65625,-0.45313 0.79688,-0.46875 1.67188,-1.5 0.125,-0.79687 0.35937,-1.71875 0.25,-0.9375 0.34375,-1.25 0.57813,-1.98437 0.76563,-3.20312 0.1875,-0.45313 0.28125,-0.53125 0.0937,-0.0781 0.42187,-0.0781 0.39063,0 0.84375,0.23438 0.46875,0.21875 0.6875,0.53125 z m 11.99976,13.79687 q -1.25,-0.0312 -1.78125,-0.59375 -0.51563,-0.5625 -0.51563,-1.57812 0,-0.26563 0.0625,-0.84375 0.125,-0.4375 0.1875,-0.89063 0.15625,-0.70312 0.32813,-0.82812 0.0312,-0.0312 0.0625,-0.125 0.0312,-0.10938 -0.0625,-0.17188 0,-0.40625 0.28125,-1.14062 0.0625,-0.23438 0.15625,-0.45313 0.0937,-0.21875 0.14062,-0.28125 -0.0781,-0.0781 -0.0781,-0.20312 0,-0.15625 0.17187,-0.3125 0.1875,-0.15625 0.26563,-0.28125 0.0781,-0.14063 -0.0156,-0.26563 -0.0625,-0.15625 0.60938,-1.4375 0.42187,-0.82812 0.45312,-0.9375 -0.0312,-0.0937 0.45313,-1.17187 0.5,-1.09375 0.79687,-1.64063 0.28125,-0.15625 -0.23437,-0.21875 -0.51563,-0.0625 -1.25,-0.0312 -1.04688,-0.0312 -1.625,-0.10937 -0.57813,-0.0937 -0.9375,-0.28125 -0.21875,-0.21875 -0.21875,-0.57813 0,-0.40625 0.29687,-0.75 0.3125,-0.35937 0.76563,-0.39062 0.82812,-0.0312 2.5625,-0.23438 l 1.625,-0.125 0.0937,-0.54687 q 0.15625,-0.28125 0.29687,-0.48438 0.15625,-0.21875 0.20313,-0.375 0.0469,-0.15625 0.0781,-0.48437 0.0937,-0.125 0.32813,-0.625 0.25,-0.51563 0.3125,-1 0.35937,-0.60938 0.65625,-1.17188 0.29687,-0.5625 0.375,-0.65625 0.5625,-0.85937 1.26562,-0.85937 0.39063,0 0.8125,0.25 0.40625,0.29687 0.625,0.51562 0.23438,0.21875 0.23438,0.60938 0,0.28125 -0.29688,0.82812 -0.125,0.25 -0.375,1.03125 -0.21875,0.28125 -0.48437,0.78125 -0.25,0.5 -0.34375,0.9375 -0.0625,0.15625 -0.20313,0.4375 -0.125,0.26563 -0.17187,0.3125 -0.0469,0.0469 -0.14063,0.0469 0.0312,0.20312 0.125,0.26562 0.0937,0.0625 0.32813,0.0625 0.15625,0 0.39062,-0.0312 0.25,-0.0312 0.59375,-0.0312 0.32813,-0.0312 0.5625,-0.0937 0.23438,-0.0625 0.46875,-0.0625 0.125,0 0.375,0.0937 0.15625,0.1875 0.15625,0.73438 0,0.40625 -0.125,0.78125 -0.125,0.375 -0.3125,0.46875 -0.15625,0.0312 -0.70312,0.1875 -0.54688,0.15625 -1.1875,0.1875 l -1.70313,0.15625 -0.53125,1.09375 q -0.23437,0.39062 -0.39062,0.75 -0.15625,0.34375 -0.21875,0.40625 0,0.15625 -0.26563,0.64062 -0.25,0.48438 -0.375,0.76563 -0.17187,0.45312 -0.39062,0.89062 -0.21875,0.42188 -0.28125,0.54688 -0.0937,0 -0.125,0.0469 -0.0156,0.0469 -0.0156,0.14063 l 0.0781,0.26562 q 0,0.15625 -0.17188,0.3125 -0.125,0.0625 -0.23437,0.23438 -0.10938,0.15625 -0.14063,0.21875 0.0625,0.0625 0.0625,0.15625 0,0.23437 -0.21875,0.57812 0,0.54688 -0.45312,1.625 0.0312,0.57813 0.0625,0.75 0.0312,0.15625 0.15625,0.125 h 0.15625 l 0.70312,-0.0312 q 0.23438,-0.0937 0.10938,0.48438 -0.10938,0.5625 -0.54688,1.75 -0.48437,0.73437 -1.01562,1.29687 -0.51563,0.5625 -0.6875,0.5625 z m 8.84204,-3.14062 q -0.57813,-0.0937 -1.0625,-0.64063 -0.48438,-0.54687 -0.48438,-1.25 0,-0.76562 0.96875,-3.9375 0.15625,-0.82812 0.70313,-2.14062 0.1875,-0.54688 0.21875,-0.64063 0.57812,-1.92187 1.57812,-4.70312 l 0.28125,-0.76563 q 1.125,-3.04687 1.34375,-3.71875 l 0.51563,-1.40625 q 0,-0.25 0.35937,-0.48437 0.375,-0.25 0.625,-0.25 0.51563,0 0.98438,0.67187 0.48437,0.67188 0.51562,1.40625 0,0.39063 -0.15625,0.70313 -0.73437,1.60937 -1.82812,4.96875 -0.46875,1.40625 -0.95313,2.67187 -0.48437,1.26563 -0.76562,1.8125 l -0.35938,0.70313 -0.4375,1.6875 q 0,0.0937 -0.0312,0.17187 -0.0312,0.0625 -0.0312,0.0937 0,0.0937 0.0312,0.0937 l 0.53125,-0.54687 q 0.9375,-0.92188 2.21875,-1.89063 1.04688,-0.73437 1.07813,-0.82812 0.0312,-0.0937 0.26562,-0.29688 l 0.3125,-0.25 q 1.03125,-0.73437 2.21875,-0.73437 0.34375,0 0.82813,0.35937 0.48437,0.34375 0.8125,0.96875 0.34375,0.625 0.34375,1.32813 v 0.25 0.23437 q 0,0.79688 -0.3125,2.09375 -0.29688,1.29688 -0.32813,1.48438 0.15625,0 0.46875,-0.0625 0.48438,0 0.6875,0.0937 0.21875,0.0937 0.375,0.26562 0.15625,0.15625 0.28125,0.1875 0.0625,0.0625 0.0625,0.1875 0,0.73438 -0.79687,1.28125 -0.79688,0.54688 -1.53125,0.60938 -0.89063,0 -1.4375,-0.67188 -0.54688,-0.67187 -0.54688,-1.64062 0,-0.375 0.0312,-0.75 0.0312,-0.39063 0.0625,-0.6875 0.0937,-0.64063 0.0937,-1.01563 0,-0.42187 -0.0781,-0.5625 -0.0781,-0.14062 -0.26563,-0.14062 -0.64062,0 -2.23437,1.17187 -1.57813,1.15625 -3.20313,3.01563 -1.03125,1.5 -1.95312,1.5 z m 17.8548,1.21875 q -1.82813,0 -2.95313,-1.01563 -1.10937,-1.03125 -1.10937,-2.92187 0,-0.73438 0.0937,-1.625 0.0625,-0.78125 0.625,-2.125 0.5625,-1.34375 1.53125,-2.60938 0.96875,-1.28125 2.28125,-1.89062 0.60937,-0.29688 1.03125,-0.40625 0.42187,-0.10938 0.95312,-0.10938 1.3125,0 2.125,0.48438 0.82813,0.46875 1.15625,1.07812 0.34375,0.60938 0.34375,1.0625 v 0.15625 0.0937 q 0,0.39063 -0.45312,1.42188 -0.45313,1.01562 -1.42188,1.92187 -0.96875,0.89063 -2.51562,0.98438 -0.76563,0 -1.48438,-0.14063 -0.71875,-0.15625 -0.90625,-0.15625 -0.48437,0 -0.60937,0.35938 -0.125,0.34375 -0.125,1.1875 0.0937,0.89062 0.48437,1.25 0.40625,0.34375 1.17188,0.34375 0.39062,0 0.89062,-0.21875 0.51563,-0.26563 1.21875,-0.8125 0.39063,-0.3125 0.60938,-0.4375 0.21875,-0.125 0.54687,-0.125 0.28125,0 0.67188,0.39062 0.39062,0.375 0.42187,0.71875 -0.0781,0.39063 -0.71875,1.15625 -0.625,0.76563 -1.65625,1.375 -1.03125,0.60938 -2.20312,0.60938 z m 1.10937,-7.71875 q 0.60938,0 0.84375,-0.125 0.3125,-0.15625 0.96875,-0.8125 0.65625,-0.65625 0.6875,-0.90625 0,-0.32813 -0.28125,-0.53125 -0.26562,-0.20313 -0.73437,-0.28125 h -0.35938 q -0.42187,0 -1.20312,0.70312 -0.78125,0.6875 -1.35938,1.57813 0.0312,0.15625 0.48438,0.26562 0.45312,0.10938 0.95312,0.10938 z m 16.38513,7.90625 q -0.98437,0 -1.85937,-0.4375 -0.85938,-0.45313 -1.375,-1.14063 -0.5,-0.6875 -0.5,-1.35937 0,-0.54688 0.4375,-0.54688 0.29687,0.0625 0.67187,0.35938 0.48438,0.28125 0.8125,0.4375 0.34375,0.15625 0.70313,0.15625 0.53125,0 0.9375,-0.17188 0.40625,-0.17187 0.46875,-0.53125 0,-0.125 -0.14063,-0.54687 -0.125,-0.4375 -0.3125,-0.79688 -0.70312,-1.04687 -0.95312,-2.04687 -0.23438,-1.01563 -0.29688,-2.35938 0,-0.73437 1.0625,-1.75 1.07813,-1.03125 2.5,-1.82812 1.4375,-0.8125 2.32813,-0.90625 1.76562,0 2.10937,0.29687 0.45313,0.34375 0.82813,1.14063 0.39062,0.79687 0.39062,1.34375 0,0.23437 -0.21875,0.57812 -0.20312,0.32813 -0.26562,0.42188 l -0.28125,0.4375 -0.21875,0.48437 q -0.0781,0.15625 -0.21875,0.48438 -0.14063,0.3125 -0.25,0.39062 -0.10938,0.0625 -0.34375,0.0156 -0.53125,0 -0.98438,-0.46875 -0.45312,-0.48438 -0.45312,-1.0625 0,-0.40625 0.28125,-0.82813 0.29687,-0.42187 0.29687,-0.48437 0,-0.125 -0.28125,-0.125 -0.42187,0 -1.29687,0.42187 -0.85938,0.40625 -1.53125,1.09375 -0.67188,0.6875 -0.67188,1.53125 0,0.4375 0.17188,1.03125 0.1875,0.59375 0.40625,0.98438 0.26562,0.25 0.75,1.51562 0.5,1.26563 0.5,2 0,1.125 -0.84375,1.70313 -0.84375,0.5625 -2.35938,0.5625 z m 11.04377,-0.64063 q -0.3125,0.0625 -0.57813,0.0625 -0.40625,0 -0.76562,-0.0781 -0.34375,-0.0781 -0.34375,-0.20313 0,-0.0937 -0.0469,-0.125 -0.0312,-0.0312 -0.125,-0.0312 -0.0937,-0.0937 -0.4375,-0.32812 -0.32812,-0.25 -0.54687,-0.34375 -0.39063,-0.54688 -0.39063,-1.1875 0,-0.60938 0.39063,-1.95313 0.0312,-1.25 0.90625,-2.79687 0.875,-1.54688 2.125,-2.875 1.25,-1.32813 2.25,-1.90625 0.46875,-0.48438 1.34375,-0.48438 0.67187,0 1.1875,0.28125 0.40625,0.26563 1.03125,1.09375 0.625,0.82813 0.9375,1.5 0.0781,0.23438 0.1875,0.625 0.10937,0.375 0.10937,0.92188 l -0.0625,0.89062 q -0.39062,1.3125 -0.39062,2.35938 -0.0625,0.45312 0.0625,0.73437 0.14062,0.26563 0.35937,0.26563 0.15625,0 0.34375,-0.0937 0.20313,-0.0937 0.45313,-0.0937 0.42187,0 0.70312,0.25 0.29688,0.25 0.29688,0.64062 0,0.3125 -0.17188,0.60938 -0.28125,0.64062 -1.0625,1.15625 -0.78125,0.5 -1.45312,0.5 -0.54688,0 -1.04688,-0.5625 -0.5,-0.57813 -0.78125,-1.51563 l -0.1875,-0.73437 -1.15625,1.0625 -0.73437,0.73437 q -0.51563,0.51563 -0.57813,0.60938 -0.15625,0.25 -0.5625,0.48437 -0.40625,0.21875 -1.26562,0.53125 z m 0.85937,-3.0625 q 0.57813,-0.32812 0.96875,-0.79687 0.40625,-0.48438 1.01563,-1.34375 0.14062,-0.20313 0.26562,-0.40625 0.125,-0.20313 0.3125,-0.4375 0.54688,-0.73438 0.78125,-1.17188 0.25,-0.45312 0.375,-0.96875 -0.125,-0.42187 -0.375,-0.78125 -0.23437,-0.375 -0.42187,-0.40625 -0.90625,0 -2.4375,2.10938 -0.15625,0.23437 -0.5,0.8125 -0.32813,0.5625 -0.42188,0.76562 -0.17187,0.21875 -0.4375,1.07813 -0.26562,0.84375 -0.26562,1.09375 0,0.70312 0.40625,0.70312 0.32812,0 0.73437,-0.25 z m 25.32318,0.92188 q 0.21875,0.32812 0.21875,0.60937 0,0.32813 -0.23437,0.57813 -0.21875,0.25 -0.375,0.25 -0.76563,0 -1.71875,-0.35938 -0.9375,-0.375 -0.9375,-1.07812 v -1.98438 q 0,-1.28125 -0.21875,-1.46875 -0.48438,-0.25 -1.28125,0.71875 -0.79688,0.96875 -1.95313,2.89063 -0.45312,0.70312 -0.78125,1.01562 -0.3125,0.29688 -0.5625,0.29688 -0.21875,0 -0.39062,-0.125 -0.98438,-0.28125 -0.98438,-1.125 0,-0.0937 0.0625,-0.40625 0.28125,-1 0.60938,-2.20313 0.34375,-1.21875 0.54687,-2.09375 -0.0937,-0.1875 -0.32812,-0.1875 -0.3125,0 -0.73438,0.32813 -0.125,0.0937 -0.46875,0.29687 -0.32812,0.20313 -0.67187,0.54688 -0.32813,0.32812 -0.6875,0.90625 -0.125,0.23437 -0.4375,0.67187 -1.60938,2.4375 -1.95313,2.75 -0.17187,0.45313 -0.3125,0.64063 -0.14062,0.17187 -0.32812,0.17187 -0.125,0 -0.32813,-0.10937 -0.60937,-0.28125 -0.98437,-0.8125 -0.35938,-0.53125 -0.35938,-1.26563 0,-0.46875 0.28125,-1.90625 0.29688,-1.45312 0.65625,-2.70312 0.25,-0.5 0.39063,-1.0625 0.14062,-0.5625 0.14062,-0.78125 0.10938,-0.60938 0.39063,-1.64063 0.28125,-1.04687 0.42187,-1.15625 0.1875,-0.1875 0.70313,-0.1875 0.34375,0 0.65625,0.14063 0.32812,0.125 0.45312,0.375 0.42188,0.48437 0.42188,1.3125 0,0.76562 -0.3125,1.625 -0.32813,0.73437 -0.8125,2.15625 0,0 0.0625,-0.0781 0.78125,-1.07812 1.98437,-2.42187 0.57813,-0.64063 1.29688,-0.98438 0.71875,-0.35937 1.17187,-0.35937 0.76563,0.125 1.29688,0.75 0.53125,0.625 0.65625,1.70312 0,0.39063 -0.0625,0.95313 -0.0625,0.5625 -0.0937,0.71875 0.125,-0.15625 0.65625,-0.78125 0.53125,-0.625 1.29687,-1.29688 0.70313,-0.54687 1.3125,-0.54687 0.5,0 0.89063,0.42187 0.70312,0.40625 1.09375,1.34375 0.40625,0.92188 0.40625,2.10938 0,0.40625 -0.0312,0.64062 -0.0312,0.21875 -0.0312,0.64063 0,0.4375 0.0937,0.84375 0.10937,0.40625 0.20312,0.71875 z m 4.44238,2.3125 q -1.0625,-0.42188 -1.5625,-1.29688 -0.48437,-0.89062 -0.48437,-2.29687 0,-0.3125 0.0625,-1.15625 0.0312,-0.25 0.0312,-0.59375 0,-0.54688 -0.125,-0.8125 -0.125,-0.28125 -0.45313,-0.57813 -0.0937,-0.0937 -0.45312,-0.40625 -0.34375,-0.32812 -0.3125,-0.48437 v -0.0937 q 0,-0.21875 0.10937,-0.48438 0.10938,-0.28125 0.21875,-0.34375 0.0937,-0.0312 0.375,-0.1875 0.29688,-0.17187 0.45313,-0.10937 0.125,0 0.39062,0.17187 0.28125,0.15625 0.28125,0.28125 0.1875,0.39063 0.98438,-1.25 0.45312,-0.67187 1.21875,-1.32812 0.76562,-0.65625 1.40625,-1 0.23437,-0.10938 1.01562,-0.28125 0.78125,-0.17188 1.23438,-0.17188 0.5,0 1.1875,0.42188 0.6875,0.40625 1.14062,0.89062 0.67188,0.48438 0.67188,1.82813 0,0.79687 -0.25,1.375 -0.125,0.46875 -1.26563,1.45312 -1.14062,0.96875 -1.84375,1.26563 -0.70312,0.15625 -1.5625,0.15625 -0.875,0 -1.1875,-0.15625 -0.21875,-0.125 -0.375,0.10937 -0.14062,0.23438 -0.23437,1.07813 l -0.0312,0.3125 q 0,0.35937 0.15625,0.67187 0.23437,0.39063 0.42187,0.54688 0.1875,0.15625 0.57813,0.15625 l 0.28125,-0.0312 q 0.35937,-0.0312 0.75,-0.34375 0.40625,-0.32813 0.95312,-0.875 0.57813,-0.54688 0.67188,-0.64063 0.15625,-0.3125 0.85937,-0.3125 0.28125,0 0.73438,0.0937 0.32812,0.25 0.32812,0.64063 0,0.54687 -0.51562,1.46875 -0.15625,0.32812 -0.98438,1.09375 -0.8125,0.76562 -1.29687,1.04687 -0.85938,0.48438 -1.90625,0.48438 -0.8125,0 -1.67188,-0.3125 z m 2.375,-7.20313 q 0.76563,-0.20312 1.1875,-0.46875 0.4375,-0.26562 1.01563,-1.07812 0.25,-0.76563 0.0937,-0.92188 -0.1875,-0.1875 -0.45313,-0.1875 -0.3125,0 -0.76562,0.34375 -0.3125,0.0937 -0.875,0.59375 -0.5625,0.5 -0.95313,0.96875 -0.375,0.45313 -0.21875,0.54688 -0.0625,0.10937 0.3125,0.15625 0.39063,0.0469 0.65625,0.0469 z" id="path4094" inkscape:connector-curvature="0" style="fill:#000000;fill-rule:nonzero" /> <path d="M 427.809,138.3307 H 553.08723 V 253.13125 H 427.809 Z" id="path4100" inkscape:connector-curvature="0" style="fill:#000000;fill-opacity:0;fill-rule:evenodd" /> <path d="M 379.751,138.43701 H 505.04235 V 253.24016 H 379.751 Z" id="path4109" inkscape:connector-curvature="0" style="fill:#00fdc8;fill-rule:evenodd" /> <path d="M 379.751,138.43701 H 505.04235 V 253.24016 H 379.751 Z" id="path4111" inkscape:connector-curvature="0" style="fill-rule:evenodd;stroke:#666666;stroke-width:1;stroke-linecap:butt;stroke-linejoin:round" /> <path d="m 299.38583,404.37924 h 70.36065 v -6.68504 l 8.40525,13.37008 -8.40525,13.37007 v -6.68502 h -70.36065 z" id="path4071-3" inkscape:connector-curvature="0" style="fill:#7c7ce0;fill-rule:evenodd;stroke-width:0.79288208" /> <path d="m 506.71874,190.09462 h 70.36065 v -6.68504 l 8.40525,13.37008 -8.40525,13.37007 v -6.68502 h -70.36065 z" id="path4071-6" inkscape:connector-curvature="0" style="fill:#7c7ce0;fill-rule:evenodd;stroke-width:0.79288208" /> <path d="m 506.71875,404.41666 h 70.36065 v -6.68504 l 8.40525,13.37008 -8.40525,13.37007 v -6.68502 h -70.36065 z" id="path4071-6-7" inkscape:connector-curvature="0" style="fill:#7c7ce0;fill-rule:evenodd;stroke-width:0.79288208" /> <flowRoot xml:space="preserve" id="flowRoot4272" style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:26.66666603px;line-height:1.25;font-family:'Comic Sans MS';-inkscape-font-specification:'Comic Sans MS';text-align:center;letter-spacing:0px;word-spacing:0px;text-anchor:middle;fill:#000000;fill-opacity:1;stroke:none"><flowRegion id="flowRegion4274" style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:26.66666603px;font-family:'Comic Sans MS';-inkscape-font-specification:'Comic Sans MS';text-align:center;text-anchor:middle"><rect id="rect4276" width="125.97456" height="90.782555" x="379.06781" y="162.45763" style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:26.66666603px;font-family:'Comic Sans MS';-inkscape-font-specification:'Comic Sans MS';text-align:center;text-anchor:middle" /></flowRegion><flowPara id="flowPara4278">Some process</flowPara></flowRoot> <g transform="matrix(0.45555722,0,0,0.45555774,379.751,138.43701)" id="g4082-1" inkscape:transform-center-x="-34.322034" inkscape:transform-center-y="102.9661"> <clipPath id="clipPath4292"> <path d="M 0,0 H 275 V 252 H 0 Z" id="path4290" inkscape:connector-curvature="0" style="clip-rule:evenodd" /> </clipPath> <image clip-path="url(#p.3-4)" width="275" height="252" x="0" y="0" preserveAspectRatio="none" xlink:href="https://desfontain.es/privacy/images/magic.gif" id="image4080-3" style="fill:#000000" /> </g> <rect style="fill:#ffffff" id="rect148" width="28.983051" height="45.762711" x="212.03391" y="388.22034" /> </g> </svg> </center> Let's now translate that into a formal definition. <a name="definition"></a> A process $A$ is $\varepsilon$-differentially private if for all databases $D_1$ and $D_2$ which differ in only one individual: <div class="math">$$ \mathbb{P}\left[A(D_1)=O\right] \le e^\varepsilon\cdot\mathbb{P}\left[A(D_2)=O\right] $$</div> … and this must be true for all possible outputs $O$. Let's unpack this. $\mathbb{P}\left[A(D_1)=O\right]$ is the probability that when you run the process $A$ on the database $D_1$, the output is $O$. This process is probabilistic: if you run it several times, it might give you different answers. A typical process might be: "count the people with blue eyes, add some random number to this count, and return this sum". Since the random number changes every time you run the process, the results will vary. $e^\varepsilon$ is the <a href="https://en.wikipedia.org/wiki/Exponential_function">exponential function</a> applied to the parameter $\varepsilon>0$. If $\varepsilon$ is very close to 0, then $e^\varepsilon$ is very close to 1, so the probabilities are very similar. The bigger $\varepsilon$ is, the more the probabilities can differ. Of course, the definition is symmetrical: you can replace $D_1$ by $D_2$ and vice-versa, and the two databases will still differ in only one individual. So we could replace it by: <div class="math">$$ e^{-\varepsilon}\cdot\mathbb{P}\left[A(D_2)=O\right] \le \mathbb{P}\left[A(D_1)=O\right] \le e^\varepsilon\cdot\mathbb{P}\left[A(D_2)=O\right] $$</div> Thus, this formula means that the output of the process is similar if you change or remove the data of one person. The degree of similarity depends on $\varepsilon$: the smaller it is, the more similar the outputs are. What does this similarity have to do with privacy? First, I'll explain this with an intuitive example. Then, I'll formalize this idea with a more generic interpretation. <h1 id="a-simple-example-randomized-response">A simple example: randomized response <a name="rr"></a></h1> Suppose you want to do a survey to know how many people are illegal drug users. If you naively go out and ask people whether they're using illegal drugs, many will lie to you. So you devise the following mechanism. The participants no longer directly answer the question "have you consumed illegal drugs in the past week?". Instead, each of them will flip a coin, without showing it to you. <ul> <li>On heads, the participant tells the truth (Yes or No).</li> <li>On tails, they flip a second coin. If the second coin lands on heads, they answer Yes. Otherwise, they answer No.</li> </ul> How is this better for survey respondents? They can now answer Yes without revealing that they're doing something illegal. When someone answers Yes, you can't know their true answer for sure. They could be actually doing drugs, but they might also have answered at random. Let's compute the probabilities of each answer for a drug user. <ul> <li>With probability 50%, they will say the truth and answer Yes.</li> <li>With probability 50%, they will answer at random.<ul> <li>They then have another 50% chance to answer Yes, so 25% chance in total.</li> <li>Similarly, in total, they have a 25% chance to answer No.</li> </ul> </li> </ul> All in all, we get a 75% chance to answer Yes and a 25% chance to answer No. For someone who is not doing drugs, the probabilities are reversed: 25% chance to answer Yes and 75% to answer No. Using the notations from earlier: <ul> <li>$\mathbb{P}\left[A(Yes)=Yes\right] = 0.75$, $\mathbb{P}\left[A(Yes)=No\right] = 0.25$</li> <li>$\mathbb{P}\left[A(No)=Yes\right] = 0.25$, $\mathbb{P}\left[A(No)=No\right] = 0.75$</li> </ul> Now, $0.75$ is three times bigger than $0.25$. So if we choose $\varepsilon$ such as $e^\varepsilon=3$ (that's $\varepsilon\simeq1.1$), this process is $\varepsilon$-differentially private. So this plausible deniability translates nicely in the language of differential privacy. Of course, with a differentially private process like this one, you're getting some noise into your data. But if you have enough answers, with high probability, the noise will cancel itself out. Suppose you have 1000 answers in total: 400 of them are Yes and 600 are No. About 50% of all 1000 answers are random, so you can remove 250 answers from each count. In total, you get 150 Yes answers out of 500 non-random answers, so about 30% of Yes overall. What if you want more privacy? Instead of having the participants say the truth with probability 50%, you can have them tell the truth 25% of the time. What if you want less noise instead, at the cost of less protection? Have them tell the truth 75% of the time. Finding out $\varepsilon$ and quantifying the noise for each option is left as an exercise for the reader =) <h1 id="a-generalization-quantifying-the-attackers-knowledge">A generalization: quantifying the attacker's knowledge</h1> <a name="quantifying"></a> Let's forget about the previous example and consider a more generic scenario. In line with the <a href="differential-privacy-awesomeness.html">previous article</a>, we will describe this scenario from the attacker's perspective. We have a mechanism $A$ which is $\varepsilon$-differentially private. We run it on some database $D$, and release the output $A(D)$ to an attacker. Then, the attacker tries to figure out whether someone (their target) is in $D$. Under differential privacy, the attacker can't gain a lot of information about their target. And this is true even if this attacker has a lot of knowledge about the dataset. Let's take the stronger attacker we can think of: they know all the database, except their target. This attacker has to determine which database is the real one, between two options: one with their target in it (let's call it $D_{in}$), the other without ($D_{out}$)<a class="footnote-ref" href="#fn:dbs">1</a>. So, in the attacker's model of the world, the actual database $D$ can be either $D_{in}$ or $D_{out}$. They might have an initial suspicion that their target is in the database. This suspicion is represented by a probability, $\mathbb{P}\left[D=D_{in}\right]$. This probability can be anything between $0$ and $1$. Say, $0.9$ if the attacker's suspicion is strong, $0.01$ if they think it's very unlikely, $0.5$ if they have no idea… Similarly, their suspicion that their target is not in the dataset is also a probability, $\mathbb{P}\left[D=D_{out}\right]$. Since there are only two options, $\mathbb{P}\left[D=D_{out}\right]=1-\mathbb{P}\left[D=D_{in}\right]$. Now, suppose the attacker sees that the mechanism returns output $O$. How much information did the attacker gain? This is captured by looking at how much their suspicion changed after seeing this output. In mathematical terms, we have to compare $\mathbb{P}\left[D=D_{in}\right]$ with the updated suspicion $\mathbb{P}\left[D=D_{in}\mid A(D)=O\right]$. This updated suspicion is the attacker's model of the world after seeing $O$. With differential privacy, the updated probability is never too far from the initial suspicion. And we can quantify this phenomenon exactly. For example, with $\varepsilon=1.1$, here is what the upper and lower bounds look like. <center> <img alt="Graph showing the bounds on the posterior as a function of the prior" src="https://desfontain.es/blog/images/dp-bounds-graph.svg"> </center> The black line is what happens if the attacker didn't get their suspicion updated at all. The blue lines are the lower and upper bounds on the updated suspicion: it can be anywhere between the two. We can visualize the example mentioned in the <a href="differential-privacy-awesomeness.html">previous article</a>: for an initial suspicion of 50%, the updated suspicion is approximately between 25% and 75%. How do we prove that these bounds hold? We'll need a result from probability theory, and some basic arithmetic manipulation. I reproduced the proof as simply as I could, but you still don't have to read it. If you want to, click here: <button id="toggleProof"></button> <div id="proof" style="display: none; border-left: double; padding-left: 10px"> The proof is based on a theorem called Bayes' rule. Explaining the full intuition behind this theorem is a bit out of scope for this post. If you want to understand what it says and why it works, I recommend you read <a href="https://arbital.com/p/bayes_rule/?l=1zq">this guide</a>. If you don't, just trust me: this theorem allows us to rephrase the updated suspicion in other terms. <div class="math">$$ \mathbb{P}\left[D=D_{in}\mid A(D)=O\right]=\frac{\mathbb{P}\left[D=D_{in}\right]\cdot\mathbb{P}\left[A(D)=O\mid D=D_{in}\right]}{\mathbb{P}\left[A(D)=O\right]} $$</div> Let's interpret each of these terms. We recognize $\mathbb{P}\left[D=D_{in}\right]$, that's the initial suspicion of the attacker. $\mathbb{P}\left[A(D)=O\mid D=D_{in}\right]$ is the probability of getting output $O$ from database $D_{in}$, which we can simplify into $\mathbb{P}\left[A\left(D_{in}\right)=O\right]$. Finally, $\mathbb{P}\left[A(D)=O\right]$ is the probability that we get the output $O$, in the attacker's model of the world. This last term is icky. We don't know its value. So let's make it disappear by considering the ratio between the two updated probabilities $\mathbb{P}\left[D=D_{in}\mid A(D)=O\right]$ and $\mathbb{P}\left[D=D_{out}\mid A(D)=O\right]$. The icky term will nicely go away: <div class="math">$$ \frac{\mathbb{P}\left[D=D_{in}\mid A(D)=O\right]}{\mathbb{P}\left[D=D_{out}\mid A(D)=O\right]} = \frac{\mathbb{P}\left[D=D_{in}\right]}{\mathbb{P}\left[D=D_{out}\right]} \cdot\frac{\mathbb{P}\left[A\left(D_{in}\right)=O\right]}{\mathbb{P}\left[A\left(D_{out}\right)=O\right]} $$</div> Note that this isn't a meaningless math trick. This ratio of probabilities actually has a simple interpretation: it's what gamblers call betting odds. For example, on betting websites, the odds for the France v. Croatia game in the 2018 World Cup were 2:1. This means that according to bookies, the probability for France to win was twice as much as for Croatia. This corresponds to probabilities of about 67% and 33%, respectively. Anyway, look! The two terms $\mathbb{P}\left[A\left(D_{in}\right)=O\right]$ and $\mathbb{P}\left[A\left(D_{out}\right)=O\right]$ are the ones from the differential privacy definition. So we know that their ratio is bounded: <div class="math">$$ e^{-\varepsilon} \le \frac{\mathbb{P}\left[A\left(D_{in}\right)=O\right]}{\mathbb{P}\left[A\left(D_{out}\right)=O\right]} \le e^\varepsilon $$</div> If we plug this into the previous formula, we get a nice relation: <div class="math">$$ e^{-\varepsilon}\cdot\frac{\mathbb{P}\left[D=D_{in}\right]}{\mathbb{P}\left[D=D_{out}\right]} \le \frac{\mathbb{P}\left[D=D_{in}\mid A(D)=O\right]}{\mathbb{P}\left[D=D_{out}\mid A(D)=O\right]} \le e^\varepsilon\cdot\frac{\mathbb{P}\left[D=D_{in}\right]}{\mathbb{P}\left[D=D_{out}\right]} $$</div> This relation is a reformulation of differential privacy. The original definition said that the probability distributions of outputs are similar. This relation says that the odds don't change too much after looking at the output. And the two formulations are equivalent: you could write the same proof in the other direction. But back to our proof. All we need to do now is replace $\mathbb{P}\left[D=D_{out}\right]$ with $1-\mathbb{P}\left[D=D_{in}\right]$, do the same for $\mathbb{P}\left[D=D_{out}\mid A\left(D\right)=O\right]$, and solve for $\mathbb{P}\left[D=D_{in}\mid A\left(D\right)=O\right]$. You end up with the following bounds: <div class="math">$$ \frac{\mathbb{P}\left[D=D_{in}\right]}{e^{\varepsilon}+\left(1-e^{\varepsilon}\right)\cdot\mathbb{P}\left[D=D_{in}\right]} \leq \mathbb{P}\left[D=D_{in}\mid A\left(D\right)=O\right] \leq \frac{e^{\varepsilon}\cdot\mathbb{P}\left[D=D_{in}\right]}{1+\left(e^{\varepsilon}-1\right)\cdot\mathbb{P}\left[D=D_{in}\right]} $$</div> which you can plot using your favorite software. </div> What does this look like for various values of $\varepsilon$? We can draw a generalization of this graph with pretty colors: <center> <img alt="Graph showing the bounds on the posterior as a function of the prior for many values of ε" src="https://desfontain.es/blog/images/dp-contour-graph.png"> </center> For larger values of $\varepsilon$, this gets scary quite fast. Let's say you're using $\varepsilon=5$. Then, an attacker can go from a small suspicion (say, 10%) to a very high degree of certainty (94%). <h1 id="what-about-composition">What about composition? <a name="composition"></a></h1> In the previous section, I formalized two claims I made in my <a href="differential-privacy-awesomeness.html">last article</a>. First, I explained what it means to quantify information gain. Furthermore, I picked an attacker with full background knowledge. If the attacker knows less information in the first place, the bounds we showed still hold. What about the third claim? I said that differential privacy was composable. Suppose that two algorithms $A$ and $B$ are $\varepsilon$-differentially private. We want to prove that publishing the result of both is $2\varepsilon$-differentially private. Let's call $C$ the algorithm which combines $A$ and $B$: $C(D)=\left(A(D),B(D)\right)$. The output of this algorithm will be a pair of outputs: $O=\left(O_A,O_B\right)$. The insight is that the two algorithms are independent. They each have their own randomness, so the result of one does not impact the result of the other. This allows us to simply write: <div class="math">$$ \begin{align*} \mathbb{P}\left[C\left(D_{1}\right)=O\right] & =\mathbb{P}\left[A\left(D_{1}\right)=O_{A}\right]\cdot\mathbb{P}\left[B\left(D_{1}\right)=O_{B}\right]\\ & \leq e^{2\varepsilon}\cdot\mathbb{P}\left[A\left(D_{2}\right)=O_{A}\right]\cdot\mathbb{P}\left[B\left(D_{2}\right)=O_{B}\right]\\ & \leq e^{2\varepsilon}\cdot\mathbb{P}\left[C\left(D_{2}\right)=O\right] \end{align*} $$</div> so $C$ is $2\varepsilon$-differentially private. <h1 id="future-steps">Future steps</h1> I hope that I convinced you that differential privacy can be an excellent way to protect your data (if your $\varepsilon$ is low). Now, if everything is going according to my master plan, you should be like… "This is awesome! I want to use it everywhere! How do I do that?" I have good news for you: this blog post has sequels. Head over to the <a href="friendly-intro-to-differential-privacy.html">table of contents</a> of this series to decide which one you want to read next! <hr> Thanks to Chao Li for introducing me to the Bayesian interpretation of differential privacy, and to <a href="http://a3nm.net/">a3nm</a>, <a href="https://virgile.anbuco.fr/">Armavica</a>, <a href="http://www.normalesup.org/~bouya/">immae</a> and <a href="https://pablo.rauzy.name/">p4bl0</a> for their helpful comments on drafts of this article (as well as previous ones). <script type="text/javascript"> var button = document.getElementById('toggleProof'); var defaultButton = 'Show me the proof'; button.innerHTML = defaultButton button.addEventListener('click', function (event) { button.innerHTML = button.innerHTML == defaultButton ? 'Hide the proof' : defaultButton; proof = document.getElementById('proof'); proof.style.display = proof.style.display == 'none' ? 'block' : 'none'; }); </script> <div class="footnote"> <hr> <ol> <li id="fn:dbs"> This can mean that $D_{out}$ is the same as $D_{in}$ with one fewer user. This can also mean that $D_{out}$ is the same as $D_{in}$, except one user has been changed to some arbitrary other user. This distinction doesn't change anything to the reasoning, so we can simply forget about it. <a class="footnote-backref" href="#fnref:dbs" title="Jump back to footnote 1 in the text">↩</a> </li> </ol> </div> <script type="text/javascript">if (!document.getElementById('mathjaxscript_pelican_#%@#$@#')) { var align = "center", indent = "0em", linebreak = "false"; if (false) { align = (screen.width < 768) ? "left" : align; indent = (screen.width < 768) ? "0em" : indent; linebreak = (screen.width < 768) ? 'true' : linebreak; } var mathjaxscript = document.createElement('script'); mathjaxscript.id = 'mathjaxscript_pelican_#%@#$@#'; mathjaxscript.type = 'text/javascript'; mathjaxscript.src = 'https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.3/latest.js?config=TeX-AMS-MML_HTMLorMML'; var configscript = document.createElement('script'); configscript.type = 'text/x-mathjax-config'; configscript[(window.opera ? "innerHTML" : "text")] = "MathJax.Hub.Config({" + " config: ['MMLorHTML.js']," + " TeX: { extensions: ['AMSmath.js','AMSsymbols.js','noErrors.js','noUndefined.js'], equationNumbers: { autoNumber: 'none' } }," + " jax: ['input/TeX','input/MathML','output/HTML-CSS']," + " extensions: ['tex2jax.js','mml2jax.js','MathMenu.js','MathZoom.js']," + " displayAlign: '"+ align +"'," + " displayIndent: '"+ indent +"'," + " showMathMenu: true," + " messageStyle: 'normal'," + " tex2jax: { " + " inlineMath: [ ['\\\$','\\\$'] ], " + " displayMath: [ ['$$','$$'] ]," + " processEscapes: true," + " preview: 'TeX'," + " }, " + " 'HTML-CSS': { " + " availableFonts: ['STIX', 'TeX']," + " preferredFont: 'STIX'," + " styles: { '.MathJax_Display, .MathJax .mo, .MathJax .mi, .MathJax .mn': {color: 'inherit ! important'} }," + " linebreaks: { automatic: "+ linebreak +", width: '90% container' }," + " }, " + "}); " + "if ('default' !== 'default') {" + "MathJax.Hub.Register.StartupHook('HTML-CSS Jax Ready',function () {" + "var VARIANT = MathJax.OutputJax['HTML-CSS'].FONTDATA.VARIANT;" + "VARIANT['normal'].fonts.unshift('MathJax_default');" + "VARIANT['bold'].fonts.unshift('MathJax_default-bold');" + "VARIANT['italic'].fonts.unshift('MathJax_default-italic');" + "VARIANT['-tex-mathit'].fonts.unshift('MathJax_default-italic');" + "});" + "MathJax.Hub.Register.StartupHook('SVG Jax Ready',function () {" + "var VARIANT = MathJax.OutputJax.SVG.FONTDATA.VARIANT;" + "VARIANT['normal'].fonts.unshift('MathJax_default');" + "VARIANT['bold'].fonts.unshift('MathJax_default-bold');" + "VARIANT['italic'].fonts.unshift('MathJax_default-italic');" + "VARIANT['-tex-mathit'].fonts.unshift('MathJax_default-italic');" + "});" + "}"; (document.body || document.getElementsByTagName('head')[0]).appendChild(configscript); (document.body || document.getElementsByTagName('head')[0]).appendChild(mathjaxscript); } </script> </div> </article> <center><button id="showBibtex">Cite this blog post!</button></center> <div id="bibtex" style="display: none"> The BibTeX entry was copied to your clipboard. <textarea id="bibtexcode" readonly></textarea> </div> <script type="text/javascript"> var bibtexdetails = `@misc{desfontainesblog20180816, title = {Differential privacy in (a bit) more detail}, author = {Damien Desfontaines}, howpublished = {\\url{https://desfontain.es/blog/differential-privacy-in-more-detail.html}}, note = {Ted is writing things (personal blog)}, year = {2018}, month = {08} }` // We need to use textarea for the tag containing code so we can select it to // copy it (<pre> wouldn't work), but inputs can't be dynamically resized to fit // the content, so we compute its size manually. Isn't web development great? var lines = bibtexdetails.split("\n"); var heigth = lines.length; var width = Math.max(...(lines.map(line => line.length))); var button = document.getElementById('showBibtex'); button.addEventListener('click', function (event) { bibtex = document.getElementById('bibtex'); bibtex.style.display = 'block'; var bibtexcode = document.getElementById('bibtexcode'); bibtexcode.innerHTML = bibtexdetails; bibtexcode.rows = heigth; bibtexcode.cols = width; bibtexcode.select(); document.execCommand('copy'); document.getSelection().removeAllRanges(); }); </script> <nav> <ul class="nav"> <li> <a href="differential-privacy-awesomeness.html">← previous</a> </li> <li> <a href="part-time-phd.html">next →</a> </li> </ul> <ul> <li><a href="#menuGlobal">back to top</a></li> <li><a href="index.html">home</a></li> <li><a href="posts.html">archives</a></li> </ul> </nav> <div class="feedback"> All opinions here are my own, not my employer's.   |   Feedback on these posts is very welcome! Please reach out via e-mail (se.niatnofsed@neimad) or <a href="https://desfontain.es/mastodon">Mastodon</a> for comments and suggestions.   |   Interested in deploying formal anonymization methods? My colleagues and I at <a href="https://tmlt.io">Tumult Labs</a> can help. Contact me at oi.tlmt@neimad, and let's chat! </div> <footer> by <a rel="dct:publisher" href="http://desfontain.es"> Damien Desfontaines </a> — <a rel="license" href="http://creativecommons.org/publicdomain/zero/1.0/"> <img src="../cc0.png" style="border-style: none;" alt="CC0" title="I don't think intellectual property makes any sense. The contents of this blog are under public domain."/> </a> — propulsed by <a href="https://getpelican.com">Pelican</a> </footer> </div> </body> </html>