DUO's approach to fraud found to be discriminatory and illegal

<!DOCTYPE html> <html lang="en" dir="ltr" prefix="og: https://ogp.me/ns#"> <head> <meta charset="utf-8" /> <meta name="description" content="DUO used an algorithm to check whether students were abusing grants. This was done in a discriminatory and therefore unlawful way, concludes the Dutch DPA." /> <meta property="og:site_name" content="Autoriteit Persoonsgegevens" /> <meta property="og:title" content="DUO's approach to fraud found to be discriminatory and illegal" /> <meta property="og:image" content="https://autoriteitpersoonsgegevens.nl/uploads/styles/og/public/2024-11/GettyImages-988152768%20studenten%20klaslokaal.jpg?itok=v8LvBCnR" /> <meta name="Generator" content="Drupal 10 (https://www.drupal.org)" /> <meta name="MobileOptimized" content="width" /> <meta name="HandheldFriendly" content="true" /> <meta name="viewport" content="width=device-width, initial-scale=1.0" /> <link rel="alternate" hreflang="nl" href="https://autoriteitpersoonsgegevens.nl/actueel/fraudeaanpak-duo-discriminerend-en-illegaal" /> <link rel="alternate" hreflang="en" href="https://autoriteitpersoonsgegevens.nl/en/current/duos-approach-to-fraud-found-to-be-discriminatory-and-illegal" /> <link rel="canonical" href="https://autoriteitpersoonsgegevens.nl/en/current/duos-approach-to-fraud-found-to-be-discriminatory-and-illegal" /> <link rel="shortlink" href="https://autoriteitpersoonsgegevens.nl/en/node/5189" /> <title>DUO's approach to fraud found to be discriminatory and illegal | Autoriteit Persoonsgegevens</title> <link rel="stylesheet" media="all" href="/uploads/css/css_DzG0_SeWHD8vpNH4XcMcMDzgvxW0gFHIKk171LSmg18.css?delta=0&language=en&theme=frontend&include=eJx1y0EKgDAMBdELKT1SSO1XC7EpSRSPr4jgyuU8mIwIGOHs6ig0V7nT04IGYxlm0xZoJS2i-W6Dd21eD1BwFrxD-nx8fHCwTStxr8R76KRbFwTSj19lAzZC" /> <link rel="stylesheet" media="all" href="/uploads/css/css_jkrh_07mRmkD4qBqx70NUSj5hYKMvNYe6yxJAzZxnzM.css?delta=1&language=en&theme=frontend&include=eJx1y0EKgDAMBdELKT1SSO1XC7EpSRSPr4jgyuU8mIwIGOHs6ig0V7nT04IGYxlm0xZoJS2i-W6Dd21eD1BwFrxD-nx8fHCwTStxr8R76KRbFwTSj19lAzZC" /> <script src="/uploads/js/js_E_EgzpvJvWBNX-OhJY85ENYVfwO4q34ivwkDYpIVwKE.js?scope=header&delta=0&language=en&theme=frontend&include=eJx1yEsOgCAMBcALYThSU-DxSSolpSYeX_fG5UyCO4xwL90oVIe83LFhwlhCNZ2OWWITTV8fHVzCBlvuxGsQX65ZzyVwxJ9_AFZ3LbU"></script> </head> <body> <a href="#main-content" class="visually-hidden focusable skip-link">Skip to main content</a> <div class="dialog-off-canvas-main-canvas" data-off-canvas-main-canvas> <div id="state-navbar" hidden></div> <div id="state-search" hidden></div> <div class="page page--en"> <header class="page__header page-header"> <div class="page-header__topbar region-topbar"> <div id="block-sitebranding" class="region-topbar__sitebranding block-sitebranding"> <a href="/en" rel="home"> <img src="/logo.svg" alt="the Dutch DPA" fetchpriority="high" /> </a> </div> <nav class="region-topbar__states block-states"> <a class="block-states__link block-states__link--navbar" href="#state-navbar" title="Menu"> <span class="block-states__link-label"> Menu </span> </a> </nav> </div> <div class="page-header__navbar region-navbar"> <div class="region-navbar__navbar-close block-navbar-close"> <a class="block-navbar-close__link" href="#" title="Close menu"> <span class="block-navbar-close__link-label">Close menu</span> </a> </div> <nav aria-labelledby="block-mainnavigation-title" id="block-mainnavigation" class="region-navbar__mainnavigation block-mainnavigation"> <h2 id="block-mainnavigation-title" class="visually-hidden block-mainnavigation__title">Hoofdnavigatie</h2> <ul class="block-mainnavigation__menu"> <li class="block-mainnavigation__menu-item"> <a href="/en" class="block-mainnavigation__menu-item-link" data-drupal-link-system-path="<front>">Home</a> </li> <li class="block-mainnavigation__menu-item"> <a href="/en/themes" class="block-mainnavigation__menu-item-link" data-drupal-link-system-path="node/3">Themes</a> </li> <li class="block-mainnavigation__menu-item block-mainnavigation__menu-item--collapsed block-mainnavigation__menu-item--active-trail"> <a href="/en/current" class="block-mainnavigation__menu-item-link" data-drupal-link-system-path="node/4">Current</a> </li> <li class="block-mainnavigation__menu-item"> <a href="/en/documents" class="block-mainnavigation__menu-item-link" data-drupal-link-system-path="node/5">Documents</a> </li> <li class="block-mainnavigation__menu-item"> <a href="/en/contact" class="block-mainnavigation__menu-item-link" data-drupal-link-system-path="node/6">Contact</a> </li> <li class="block-mainnavigation__menu-item"> <a href="/en/dpo-information" class="block-mainnavigation__menu-item-link" data-drupal-link-system-path="node/4221">DPO</a> </li> </ul> </nav> <nav aria-labelledby="block-actionbuttons-title" id="block-actionbuttons" class="region-navbar__actionbuttons block-actionbuttons"> <h2 id="block-actionbuttons-title" class="visually-hidden block-actionbuttons__title">Actieknoppen</h2> <ul class="block-actionbuttons__menu"> <li class="block-actionbuttons__menu-item"> <a href="/en/report-data-breach" class="block-actionbuttons__menu-item-link" data-drupal-link-system-path="node/8">Report data breach</a> </li> <li class="block-actionbuttons__menu-item"> <a href="/en/submitting-a-tip-off-or-a-complaint-to-the-dutch-dpa" class="block-actionbuttons__menu-item-link" data-drupal-link-system-path="node/9">File a complaint</a> </li> </ul> </nav> <div id="block-topnavigationgroup" class="region-navbar__topnavigationgroup block-topnavigationgroup"> <nav aria-labelledby="block-topnavigation-title" id="block-topnavigation" class="region-top-navigation-group__topnavigation block-topnavigation"> <h2 id="block-topnavigation-title" class="visually-hidden block-topnavigation__title">Topnavigatie</h2> <ul class="block-topnavigation__menu"> <li class="block-topnavigation__menu-item block-topnavigation__menu-item--collapsed"> <a href="/en/about-the-dutch-dpa" class="block-topnavigation__menu-item-link" data-drupal-link-system-path="node/11">About the Dutch DPA</a> </li> <li class="block-topnavigation__menu-item"> <a href="/en/themes/international/international-cooperation/european-data-protection-board-edpb" class="block-topnavigation__menu-item-link" data-drupal-link-system-path="node/264">EDPB</a> </li> <li class="block-topnavigation__menu-item"> <a href="/en/node/12" class="block-topnavigation__menu-item-link" data-drupal-link-system-path="node/12">Career</a> </li> <li class="block-topnavigation__menu-item"> <a href="/en/node/4157" class="block-topnavigation__menu-item-link" data-drupal-link-system-path="node/4157">Press</a> </li> <li class="block-topnavigation__menu-item"> <a href="/en/node/14" class="block-topnavigation__menu-item-link" data-drupal-link-system-path="node/14">Dictionary</a> </li> </ul> </nav> <div class="language-switcher-language-url region-top-navigation-group__taalkeuze block-taalkeuze" id="block-taalkeuze" role="navigation"> <ul class="links"><li hreflang="nl" data-drupal-link-system-path="node/5189"><a href="/actueel/fraudeaanpak-duo-discriminerend-en-illegaal" class="language-link" hreflang="nl" data-drupal-link-system-path="node/5189">NL</a></li><li hreflang="en" data-drupal-link-system-path="node/5189" class="is-active" aria-current="page"><a href="/en/current/duos-approach-to-fraud-found-to-be-discriminatory-and-illegal" class="language-link is-active" hreflang="en" data-drupal-link-system-path="node/5189">EN</a></li></ul> </div> </div> <div class="views-exposed-form bef-exposed-form region-navbar__search block-search" itemscope="" itemtype="http://schema.org/SearchAction" data-drupal-selector="views-exposed-form-search-search-bar" id="block-search"> <form action="/en/search" method="get" id="views-exposed-form-search-search-bar" accept-charset="UTF-8" class="form-views-exposed-form"> <div class="form-views-exposed-form__content"> <h2 class="form-views-exposed-form__title">Filter search results</h2> <div class="form-views-exposed-form__container form-views-exposed-form__filters"> <div class="form-views-exposed-form__keys js-form-item js-form-type-search-api-autocomplete js-form-item-keys form-item form-item-keys form-type-search-api-autocomplete"> <label for="edit-keys" class="form-item__label">Search within website</label> <input placeholder="What are you looking for?" itemprop="http://schema.org/query" data-drupal-selector="edit-keys" data-search-api-autocomplete-search="search" class="form-autocomplete form-text" data-autocomplete-path="/en/search_api_autocomplete/search?display=search_bar&&filter=keys" type="text" id="edit-keys" name="keys" value="" size="30" maxlength="128" /> </div> <meta itemprop="target" content="/en/search?keys={query}"/> <div data-drupal-selector="edit-actions" class="form-actions js-form-wrapper form-wrapper" id="edit-actions"> <button data-drupal-selector="edit-submit-search" type="submit" id="edit-submit-search" value="Search" class="button js-form-submit form-submit">Search</button> </div> </div> </div> </form> </div> </div> </header> <main class="page__body page-body"> <a id="main-content" tabindex="-1"></a> <div class="page-body__content region-content"> <div data-drupal-messages-fallback class="hidden"></div><div id="block-campaign-notifcation" class="region-content__campaign-notifcation block-campaign-notifcation"> </div> <div id="block-breadcrumb" class="region-content__breadcrumb block-breadcrumb"> <nav class="breadcrumb" aria-labelledby="breadcrumb"> <h2 id="breadcrumb" class="breadcrumb__label">Breadcrumb</h2> <ol class="breadcrumb__list" itemscope itemtype="http://schema.org/BreadcrumbList"> <li class="breadcrumb__item" itemprop="itemListElement" itemscope itemtype="http://schema.org/ListItem"> <a class="breadcrumb__item-link" title="Back to Home" href="/en" itemprop="item"><span itemprop="name">Home</span></a> <meta itemprop="position" content="1"/> </li> <li class="breadcrumb__item" itemprop="itemListElement" itemscope itemtype="http://schema.org/ListItem"> <a class="breadcrumb__item-link" title="Back to Current" href="/en/current" itemprop="item"><span itemprop="name">Current</span></a> <meta itemprop="position" content="2"/> </li> </ol> </nav> </div> <div id="block-primary-content" class="region-content__primary-content block-primary-content"> <article class="node-article-full node-article-full--with-sidebar"> <div class="node-article-full__content node-article-full__type--news"> <h1 class="node-article-full__title"> <span>DUO's approach to fraud found to be discriminatory and illegal</span> </h1> <div class="node-article-full__meta"> <div class="node-article-full__meta-submitted"> 11 November 2024 </div> <div class="node-article-full__meta-topics"> Themes: <div class="node-article-full__topics"> <div class="node-article-full__topics-item"> <a href="/en/themes/algorithms-ai/ai-algorithmic-risks-developments-in-the-netherlands" hreflang="en">AI & algorithmic risks: developments in the Netherlands</a> </div> </div> </div> </div> <div class="node-article-full__intro"> <div class="node-article-full__intro-item text-intro-html"><p>The Dutch Education Executive Agency DUO used an algorithm to check whether students were abusing the grant for students living away from home. The way in which DUO did this was discriminatory and therefore unlawful, concludes the Dutch Data Protection Authority (Dutch DPA) after an investigation.</p></div> </div> <div class="node-article-full__primary-content"> <div class="node-article-full__primary-content-item paragraph-text-partial"> <div class="paragraph-text-partial__content"> <div class="paragraph-text-partial__body"> <div class="paragraph-text-partial__body-item text-basic-html"><p>DUO gave students a ‘risk score’ by looking at type of education, distance between addresses, and age. There was no objective justification for those criteria. That made this way of working discriminatory and therefore unlawful. Furthermore, the Minister of Education, Culture and Science (OCW) – the party responsible for DUO – says that this algorithm has indirectly discriminated students with a migration background.</p><h2>No substantiation</h2><p>One student was given a higher risk score than another, without any proper substantiation. In addition, DUO has never evaluated the functioning of the algorithm.</p><p>Aleid Wolfsen, chairman of the Dutch DPA: ‘When you use an algorithm with selection criteria, you make a distinction, by definition, between groups of individuals. You always have to substantiate that distinction very well and precisely. You also have to check whether the outcome of the algorithm does not, unintentionally, make an unfair distinction. If you do not do this and start working with an algorithm without a sound substantiation, then there will very soon be a discriminatory algorithm and therefore a discriminatory, unlawful personal data processing.’</p><h2>Selection criteria</h2><p>DUO used <a href="/en/themes/algorithms-ai" data-entity-type="node" data-entity-uuid="d9fc827e-ff54-489e-93eb-07a6616d0e1e" data-entity-substitution="canonical">an algorithm</a> to calculate the risk of students committing fraud with the grant for students living away from home. Based on the algorithm, DUO selected students for a house call. The following criteria were used in the process:</p><ul><li><strong>Type of education: </strong>a senior secondary vocational education resulted in a higher risk score than a higher professional or university education.</li><li><strong>Distance: </strong>a shorter distance between the home address of the student and that of the parent(s) resulted in a higher risk score.</li><li><strong>Age: </strong>a younger age of the student resulted in a higher risk score.</li></ul><p>If, as a student, you had a higher risk score, you had a higher chance of (after a manual selection by DUO) being faced with a check and a house call by DUO. The Dutch DPA estimates that between 2013 and 2022, DUO selected 21,500 students and checked them for fraud, partly on the basis of the calculations made by the algorithm.</p><h2>Direct discrimination</h2><p>Making a distinction between people always requires a substantiated objective justification. For example, a legal or scientific substantiation or a statistic analysis with robust significant outcomes that are stable over time. But the Dutch DPA finds that these were missing for the selection criteria applied by DUO. As a result, DUO has discriminated directly, based on type of education, distance, and age. This makes the processing of personal data by DUO unlawful.</p><h2>Indirect discrimination</h2><p>What's more, according to the minister, the selection process resulted in indirect discrimination. The Minister of OCW concluded this himself, based on research. This indirect discrimination arose because students with a non-European migration background score higher on the selection criteria. As a result, they were selected for a check or a house call more often than other students.</p><h2>Any form of discrimination is always prohibited</h2><p>Discrimination is always prohibited, regardless of whether it concerns direct or indirect discrimination. This prohibition can be found in Article 21 of the Charter of Fundamental Rights of the European Union (CFREU), Article 14 of the European Convention on Human Rights (ECHR), and Article 1 of the Dutch Constitution. DUO discriminated indirectly based on type of education, distance and age, and indirectly based on background of the student.</p><h2>Algorithms at the government</h2><p>The government increasingly uses algorithms. For example, for the implementation of automated selection processes for which large volumes of data have to be processed. For processes such as these, the use of algorithms may be appropriate.</p><p>If a governmental organisation wants to use an algorithm with selection criteria, the organisation will always have to justify these criteria by an advance objective substantiation. The organisation will also have to test the outcome of an algorithm in advance, because criteria in algorithms always make a distinction between groups of individuals. Including criteria that at first sight seem neutral, such as type of education, distance, and age in this case. There must always be an objective justification for this distinction, and it may not result in unjustified indirect discrimination.</p><p>Aleid Wolfsen: ‘Unfortunately, discrimination is by no means always recognised for what it is. Because it is often thought that discrimination is about origin, sexual preference or religion, for example. But do you make an unjustified distinction between people in any other way? Then this is just as well discrimination, and therefore prohibited. DUO also overlooked this at first. Although DUO did not register the migration background of students, it did register other data to make a distinction. And there was no objective justification for that either.’</p><h2>Measures</h2><p>Following alarming media reports, the Dutch DPA started <a href="/documenten/onderzoeksrapport-fraudeaanpak-duo" data-entity-type="node" data-entity-uuid="b8f37b30-05a3-449e-80d2-c979fc33cabc" data-entity-substitution="canonical">an investigation</a> into DUO. The investigation showed that there was no objective justification for the selection criteria. And that as a result, DUO's approach to fraud was discriminatory and unlawful.</p><p>Aleid Wolfsen: ‘The violation was so evident that we did not wait for the completion of the report, but informed the minister about the preliminary results immediately. We have drawn his attention to the importance of rectifying this as soon as possible for the affected students. The package of measures that the minister comes up with today is in keeping with this.’</p></div> </div> </div> </div> </div> </div> <div class="node-article-full__content-secondary"> <div class="node-article-full__highlighted-image"> <div class="media-image-figure-decorative"> <div class="media-image-figure-decorative__field-media-image"> <img loading="lazy" src="/uploads/styles/auto_768/public/2024-11/GettyImages-988152768%20studenten%20klaslokaal.jpg?itok=eNWJwlXp" width="768" height="513" alt="" /> </div> </div> </div> <div class="node-article-full__secondary-content"> <div class="node-article-full__secondary-content-item paragraph-publications-partial"> <div class="paragraph-publications-partial__content"> </div> </div> </div> </div> <div class="node-article-full__lanes"> </div> <div class="node-article-full__related"> <h2 class="node-article-full__related-title">Also read</h2> <div class="node-article-full__related-articles"> <div class="node-article-full__related-articles-item"> <article class="node-article-teaser node-article-teaser--nieuwsbericht"> <a class="node-article-teaser__action" href="/en/current/dutch-dpa-still-identifies-privacy-risks-within-the-government" title="View Dutch DPA still identifies privacy risks within the government"></a> <div class="node-article-teaser__content"> <h1 class="node-article-teaser__title"> <span>Dutch DPA still identifies privacy risks within the government</span> </h1> <div class="node-article-teaser__submitted"> 09 October 2024 </div> <a class="node-article-teaser__link" href="/en/current/dutch-dpa-still-identifies-privacy-risks-within-the-government" title="Read news item"> Read this news item </a> </div> <div class="node-article-teaser__highlighted-image"> <div class="media-image-figure-decorative"> <div class="media-image-figure-decorative__field-media-image"> <img loading="lazy" src="/uploads/styles/auto_768/public/2024-10/GettyImages-1599001070_vrouw%20tussen%20fietsen.jpg?itok=eqQBiVnt" width="768" height="513" alt="" /> </div> </div> </div> </article> </div> </div> <a class="node-article-full__related-link" href="/actueel">View all current affairs</a> </div> </article> </div> </div> </main> <footer class="page__footer page-footer"> <div class="page-footer__footer region-footer"> <nav aria-labelledby="block-footer-actionbuttons-title" id="block-footer-actionbuttons" class="region-footer__footer-actionbuttons block-footer-actionbuttons"> <h2 id="block-footer-actionbuttons-title" class="visually-hidden block-footer-actionbuttons__title">Actieknoppen</h2> <ul class="block-footer-actionbuttons__menu"> <li class="block-footer-actionbuttons__menu-item"> <a href="/en/report-data-breach" class="block-footer-actionbuttons__menu-item-link" data-drupal-link-system-path="node/8">Report data breach</a> </li> <li class="block-footer-actionbuttons__menu-item"> <a href="/en/submitting-a-tip-off-or-a-complaint-to-the-dutch-dpa" class="block-footer-actionbuttons__menu-item-link" data-drupal-link-system-path="node/9">File a complaint</a> </li> </ul> </nav> <nav aria-labelledby="block-footer-mainnavigation-title" id="block-footer-mainnavigation" class="region-footer__footer-mainnavigation block-footer-mainnavigation"> <h2 id="block-footer-mainnavigation-title" class="visually-hidden block-footer-mainnavigation__title">Hoofdnavigatie</h2> <ul class="block-footer-mainnavigation__menu"> <li class="block-footer-mainnavigation__menu-item"> <a href="/en" class="block-footer-mainnavigation__menu-item-link" data-drupal-link-system-path="<front>">Home</a> </li> <li class="block-footer-mainnavigation__menu-item"> <a href="/en/themes" class="block-footer-mainnavigation__menu-item-link" data-drupal-link-system-path="node/3">Themes</a> </li> <li class="block-footer-mainnavigation__menu-item block-footer-mainnavigation__menu-item--collapsed block-footer-mainnavigation__menu-item--active-trail"> <a href="/en/current" class="block-footer-mainnavigation__menu-item-link" data-drupal-link-system-path="node/4">Current</a> </li> <li class="block-footer-mainnavigation__menu-item"> <a href="/en/documents" class="block-footer-mainnavigation__menu-item-link" data-drupal-link-system-path="node/5">Documents</a> </li> <li class="block-footer-mainnavigation__menu-item"> <a href="/en/contact" class="block-footer-mainnavigation__menu-item-link" data-drupal-link-system-path="node/6">Contact</a> </li> <li class="block-footer-mainnavigation__menu-item"> <a href="/en/dpo-information" class="block-footer-mainnavigation__menu-item-link" data-drupal-link-system-path="node/4221">DPO</a> </li> </ul> </nav> <nav aria-labelledby="block-footer-footernavigation-title" id="block-footer-footernavigation" class="region-footer__footer-footernavigation block-footer-footernavigation"> <h2 id="block-footer-footernavigation-title" class="visually-hidden block-footer-footernavigation__title">Footernavigatie</h2> <ul class="block-footer-footernavigation__menu"> <li class="block-footer-footernavigation__menu-item block-footer-footernavigation__menu-item--collapsed"> <a href="/en/contact" class="block-footer-footernavigation__menu-item-link" data-drupal-link-system-path="node/6">Contact</a> </li> <li class="block-footer-footernavigation__menu-item"> <a href="/en/node/19" class="block-footer-footernavigation__menu-item-link" data-drupal-link-system-path="node/19">About this website</a> </li> <li class="block-footer-footernavigation__menu-item block-footer-footernavigation__menu-item--collapsed"> <a href="/en/node/4079" class="block-footer-footernavigation__menu-item-link" data-drupal-link-system-path="node/4079">The AP and privacy</a> </li> <li class="block-footer-footernavigation__menu-item"> <a href="/en/node/12" class="block-footer-footernavigation__menu-item-link" data-drupal-link-system-path="node/12">Career</a> </li> </ul> </nav> <div id="block-footer-sitebranding" class="region-footer__footer-sitebranding block-footer-sitebranding"> <a href="/en" rel="home"> <img src="/logo.svg" alt="the Dutch DPA" fetchpriority="high" /> </a> </div> </div> </footer> </div> </div> <script type="application/json" data-drupal-selector="drupal-settings-json">{"path":{"baseUrl":"\/","pathPrefix":"en\/","currentPath":"node\/5189","currentPathIsAdmin":false,"isFront":false,"currentLanguage":"en"},"pluralDelimiter":"\u0003","suppressDeprecationErrors":true,"ajaxPageState":{"libraries":"eJx1zUEKwzAMBdELOfGRjBz_xAbFFpJc2tu3lEIWoct5m8lwhyY8ZRhK2ht_0uKBDiUOu47u6CUePPK9lwoqQUjpUJJqsegU4vWSdXaZmZtVlKAwGd3aA8kpM367ePny9WAg3WoiaYmmj22cwnDEPx7sZY4zZjK8AfRSUvw","theme":"frontend","theme_token":null},"ajaxTrustedUrl":{"\/en\/search":true},"search_api_autocomplete":{"search":{"auto_submit":true,"min_length":2}},"csp":{"nonce":"LhCciszZ1MJR_W-SUz2rdA"},"user":{"uid":0,"permissionsHash":"1b22ba74c9f94929ab2025337ef49118c66aa437eefe3b6e0872c0d3c881682a"}}</script> <script src="/uploads/js/js_zOjX7KMekAhQM5M8Zr3HMzH2TsUP3CUiuj9p334kUVM.js?scope=footer&delta=0&language=en&theme=frontend&include=eJx1yEsOgCAMBcALYThSU-DxSSolpSYeX_fG5UyCO4xwL90oVIe83LFhwlhCNZ2OWWITTV8fHVzCBlvuxGsQX65ZzyVwxJ9_AFZ3LbU"></script> </body> </html>

CINXE.COM

DUO's approach to fraud found to be discriminatory and illegal | Autoriteit Persoonsgegevens