CINXE.COM
<!doctype html> <html lang="en" dir="ltr" class="docs-wrapper docs-doc-page docs-version-6.1.1 plugin-docs plugin-id-default docs-doc-id-graph-production-workflow/cleaning"> <head> <meta charset="UTF-8"> <meta name="generator" content="Docusaurus v2.2.0"> <title data-rh="true">Cleaning | OpenAIRE Graph Documentation</title><meta data-rh="true" name="viewport" content="width=device-width,initial-scale=1"><meta data-rh="true" name="twitter:card" content="summary_large_image"><meta data-rh="true" property="og:url" content="https://graph.openaire.eu/docs/6.1.1/graph-production-workflow/cleaning"><meta data-rh="true" name="docusaurus_locale" content="en"><meta data-rh="true" name="docsearch:language" content="en"><meta data-rh="true" name="docusaurus_version" content="6.1.1"><meta data-rh="true" name="docusaurus_tag" content="docs-default-6.1.1"><meta data-rh="true" name="docsearch:version" content="6.1.1"><meta data-rh="true" name="docsearch:docusaurus_tag" content="docs-default-6.1.1"><meta data-rh="true" property="og:title" content="Cleaning | OpenAIRE Graph Documentation"><meta data-rh="true" name="description" content="The aggregation processes run independently one from another and continuously. Each aggregation process, depending on the characteristics of the records exposed by the data source, makes use of one or more vocabularies to harmonise the values available in a given field."><meta data-rh="true" property="og:description" content="The aggregation processes run independently one from another and continuously. Each aggregation process, depending on the characteristics of the records exposed by the data source, makes use of one or more vocabularies to harmonise the values available in a given field."><link data-rh="true" rel="icon" href="/docs/img/favicon.ico"><link data-rh="true" rel="canonical" href="https://graph.openaire.eu/docs/6.1.1/graph-production-workflow/cleaning"><link data-rh="true" rel="alternate" href="https://graph.openaire.eu/docs/6.1.1/graph-production-workflow/cleaning" hreflang="en"><link data-rh="true" rel="alternate" href="https://graph.openaire.eu/docs/6.1.1/graph-production-workflow/cleaning" hreflang="x-default"><link rel="preconnect" href="https://analytics.openaire.eu/"> <script>var _paq=window._paq=window._paq||[];_paq.push(["setRequestMethod","POST"]),_paq.push(["trackPageView"]),_paq.push(["enableLinkTracking"]),_paq.push(["enableHeartBeatTimer"]),function(){var e="https://analytics.openaire.eu/";_paq.push(["setRequestMethod","POST"]),_paq.push(["setTrackerUrl",e+"piwik.php"]),_paq.push(["setSiteId","373"]);var a=document,t=a.createElement("script"),p=a.getElementsByTagName("script")[0];t.type="text/javascript",t.async=!0,t.src=e+"matomo.js",p.parentNode.insertBefore(t,p)}()</script> <link rel="stylesheet" href="https://cdn.jsdelivr.net/npm/katex@0.13.24/dist/katex.min.css" integrity="sha384-odtC+0UGzzFL/6PNoE8rX/SPcQDXBJ+uRepguP4QkPCm2LBxH3FA3y+fKSiJ+AmM" crossorigin="anonymous"><link rel="stylesheet" href="/docs/assets/css/styles.fa5879b5.css"> <link rel="preload" href="/docs/assets/js/runtime~main.17d936df.js" as="script"> <link rel="preload" href="/docs/assets/js/main.42758c1f.js" as="script"> </head> <body class="navigation-with-keyboard"> <script>!function(){function t(t){document.documentElement.setAttribute("data-theme",t)}var e=function(){var t=null;try{t=localStorage.getItem("theme")}catch(t){}return t}();t(null!==e?e:"light")}()</script><div id="__docusaurus"> <div role="region" aria-label="Skip to main content"><a class="skipToContent_fXgn" href="#docusaurus_skipToContent_fallback">Skip to main content</a></div><nav class="navbar navbar--fixed-top"><div class="navbar__inner"><div class="navbar__items"><button aria-label="Toggle navigation bar" aria-expanded="false" class="navbar__toggle clean-btn" type="button"><svg width="30" height="30" viewBox="0 0 30 30" aria-hidden="true"><path stroke="currentColor" stroke-linecap="round" stroke-miterlimit="10" stroke-width="2" d="M4 7h22M4 15h22M4 23h22"></path></svg></button><a class="navbar__brand" href="/docs/"><div class="navbar__logo"><img src="/docs/img/logo.png" alt="OpenAIRE" class="themedImage_ToTc themedImage--light_HNdA"><img src="/docs/img/logo.png" alt="OpenAIRE" class="themedImage_ToTc themedImage--dark_i4oU"></div><b class="navbar__title text--truncate">documentation</b></a></div><div class="navbar__items navbar__items--right"><div class="navbar__item dropdown dropdown--hoverable dropdown--right"><a aria-current="page" class="navbar__link active" aria-haspopup="true" aria-expanded="false" role="button" href="/docs/6.1.1/">6.1.1</a><ul class="dropdown__menu"><li><a class="dropdown__link" href="/docs/next/graph-production-workflow/cleaning">Next</a></li><li><a class="dropdown__link" href="/docs/graph-production-workflow/cleaning">9.0.0</a></li><li><a class="dropdown__link" href="/docs/8.0.1/graph-production-workflow/cleaning">8.0.1</a></li><li><a class="dropdown__link" href="/docs/8.0.0/graph-production-workflow/cleaning">8.0.0</a></li><li><a class="dropdown__link" href="/docs/7.2.0/graph-production-workflow/cleaning">7.2.0</a></li><li><a class="dropdown__link" href="/docs/7.1.3/graph-production-workflow/cleaning">7.1.3</a></li><li><a class="dropdown__link" href="/docs/7.1.2/graph-production-workflow/cleaning">7.1.2</a></li><li><a class="dropdown__link" href="/docs/7.1.1/graph-production-workflow/cleaning">7.1.1</a></li><li><a class="dropdown__link" href="/docs/7.1.0/graph-production-workflow/cleaning">7.1.0</a></li><li><a class="dropdown__link" href="/docs/7.0.0/graph-production-workflow/cleaning">7.0.0</a></li><li><a class="dropdown__link" href="/docs/6.2.2/graph-production-workflow/cleaning">6.2.2</a></li><li><a aria-current="page" class="dropdown__link dropdown__link--active" href="/docs/6.1.1/graph-production-workflow/cleaning">6.1.1</a></li><li><a class="dropdown__link" href="/docs/6.0.0/graph-production-workflow/cleaning">6.0.0</a></li><li><a class="dropdown__link" href="/docs/5.2.0/graph-production-workflow/cleaning">5.2.0</a></li><li><a class="dropdown__link" href="/docs/5.1.3/graph-production-workflow/cleaning">5.1.3</a></li><li><a class="dropdown__link" href="/docs/5.1.2/graph-production-workflow/cleaning">5.1.2</a></li><li><a class="dropdown__link" href="/docs/5.1.1/graph-production-workflow/cleaning">5.1.1</a></li><li><a class="dropdown__link" href="/docs/5.1.0/">5.1.0</a></li><li><a class="dropdown__link" href="/docs/5.0.0/">5.0.0</a></li></ul></div><div class="searchBox_ZlJk"><div class="navbar__search searchBarContainer_NW3z"><input placeholder="Search" aria-label="Search" class="navbar__search-input"><div class="loadingRing_RJI3 searchBarLoadingRing_YnHq"><div></div><div></div><div></div><div></div></div></div></div></div></div><div role="presentation" class="navbar-sidebar__backdrop"></div></nav><div id="docusaurus_skipToContent_fallback" class="main-wrapper mainWrapper_z2l0 docsWrapper_BCFX"><button aria-label="Scroll back to top" class="clean-btn theme-back-to-top-button backToTopButton_sjWU" type="button"></button><div class="docPage__5DB"><aside class="theme-doc-sidebar-container docSidebarContainer_b6E3"><div class="sidebar_njMd"><nav class="menu thin-scrollbar menu_SIkG"><ul class="theme-doc-sidebar-menu menu__list"><li class="theme-doc-sidebar-item-link theme-doc-sidebar-item-link-level-1 menu__list-item"><a class="menu__link" href="/docs/6.1.1/">Overview</a></li><li class="theme-doc-sidebar-item-category theme-doc-sidebar-item-category-level-1 menu__list-item menu__list-item--collapsed"><div class="menu__list-item-collapsible"><a class="menu__link menu__link--sublist" aria-expanded="false" href="/docs/6.1.1/data-model/">Data model</a><button aria-label="Toggle the collapsible sidebar category 'Data model'" type="button" class="clean-btn menu__caret"></button></div></li><li class="theme-doc-sidebar-item-link theme-doc-sidebar-item-link-level-1 menu__list-item"><a href="https://graph.openaire.eu/develop/overview.html" target="_blank" rel="noopener noreferrer" class="menu__link menuExternalLink_NmtK">Public API<svg width="13.5" height="13.5" aria-hidden="true" viewBox="0 0 24 24" class="iconExternalLink_nPIU"><path fill="currentColor" d="M21 13v10h-21v-19h12v2h-10v15h17v-8h2zm3-12h-10.988l4.035 4-6.977 7.07 2.828 2.828 6.977-7.07 4.125 4.172v-11z"></path></svg></a></li><li class="theme-doc-sidebar-item-category theme-doc-sidebar-item-category-level-1 menu__list-item menu__list-item--collapsed"><div class="menu__list-item-collapsible"><a class="menu__link menu__link--sublist" aria-expanded="false" href="/docs/6.1.1/category/downloads">Downloads</a><button aria-label="Toggle the collapsible sidebar category 'Downloads'" type="button" class="clean-btn menu__caret"></button></div></li><li class="theme-doc-sidebar-item-category theme-doc-sidebar-item-category-level-1 menu__list-item"><div class="menu__list-item-collapsible"><a class="menu__link menu__link--sublist menu__link--active" aria-expanded="true" href="/docs/6.1.1/graph-production-workflow/">Graph production workflow</a><button aria-label="Toggle the collapsible sidebar category 'Graph production workflow'" type="button" class="clean-btn menu__caret"></button></div><ul style="display:block;overflow:visible;height:auto" class="menu__list"><li class="theme-doc-sidebar-item-category theme-doc-sidebar-item-category-level-2 menu__list-item menu__list-item--collapsed"><div class="menu__list-item-collapsible"><a class="menu__link menu__link--sublist" aria-expanded="false" tabindex="0" href="/docs/6.1.1/graph-production-workflow/aggregation/">Aggregation</a><button aria-label="Toggle the collapsible sidebar category 'Aggregation'" type="button" class="clean-btn menu__caret"></button></div></li><li class="theme-doc-sidebar-item-link theme-doc-sidebar-item-link-level-2 menu__list-item"><a class="menu__link" tabindex="0" href="/docs/6.1.1/graph-production-workflow/merge-by-id">Merge by id</a></li><li class="theme-doc-sidebar-item-category theme-doc-sidebar-item-category-level-2 menu__list-item menu__list-item--collapsed"><div class="menu__list-item-collapsible"><a class="menu__link menu__link--sublist" aria-expanded="false" tabindex="0" href="/docs/6.1.1/graph-production-workflow/enrichment-by-mining/">Enrichment by mining</a><button aria-label="Toggle the collapsible sidebar category 'Enrichment by mining'" type="button" class="clean-btn menu__caret"></button></div></li><li class="theme-doc-sidebar-item-link theme-doc-sidebar-item-link-level-2 menu__list-item"><a class="menu__link menu__link--active" aria-current="page" tabindex="0" href="/docs/6.1.1/graph-production-workflow/cleaning">Cleaning</a></li><li class="theme-doc-sidebar-item-category theme-doc-sidebar-item-category-level-2 menu__list-item menu__list-item--collapsed"><div class="menu__list-item-collapsible"><a class="menu__link menu__link--sublist" aria-expanded="false" tabindex="0" href="/docs/6.1.1/graph-production-workflow/deduplication/">Deduplication</a><button aria-label="Toggle the collapsible sidebar category 'Deduplication'" type="button" class="clean-btn menu__caret"></button></div></li><li class="theme-doc-sidebar-item-category theme-doc-sidebar-item-category-level-2 menu__list-item menu__list-item--collapsed"><div class="menu__list-item-collapsible"><a class="menu__link menu__link--sublist" aria-expanded="false" tabindex="0" href="/docs/6.1.1/category/deduction--propagation">Deduction & propagation</a><button aria-label="Toggle the collapsible sidebar category 'Deduction & propagation'" type="button" class="clean-btn menu__caret"></button></div></li><li class="theme-doc-sidebar-item-category theme-doc-sidebar-item-category-level-2 menu__list-item menu__list-item--collapsed"><div class="menu__list-item-collapsible"><a class="menu__link menu__link--sublist" aria-expanded="false" tabindex="0" href="/docs/6.1.1/graph-production-workflow/indicators-ingestion/">Indicators ingestion</a><button aria-label="Toggle the collapsible sidebar category 'Indicators ingestion'" type="button" class="clean-btn menu__caret"></button></div></li><li class="theme-doc-sidebar-item-link theme-doc-sidebar-item-link-level-2 menu__list-item"><a class="menu__link" tabindex="0" href="/docs/6.1.1/graph-production-workflow/finalisation">Finalisation</a></li><li class="theme-doc-sidebar-item-link theme-doc-sidebar-item-link-level-2 menu__list-item"><a class="menu__link" tabindex="0" href="/docs/6.1.1/graph-production-workflow/indexing">Indexing</a></li><li class="theme-doc-sidebar-item-link theme-doc-sidebar-item-link-level-2 menu__list-item"><a class="menu__link" tabindex="0" href="/docs/6.1.1/graph-production-workflow/stats">Stats analysis</a></li></ul></li><li class="theme-doc-sidebar-item-link theme-doc-sidebar-item-link-level-1 menu__list-item"><a class="menu__link" href="/docs/6.1.1/publications">Relevant publications</a></li><li class="theme-doc-sidebar-item-link theme-doc-sidebar-item-link-level-1 menu__list-item"><a class="menu__link" href="/docs/6.1.1/license">License</a></li><li class="theme-doc-sidebar-item-link theme-doc-sidebar-item-link-level-1 menu__list-item"><a class="menu__link" href="/docs/6.1.1/changelog">Versions & changelog</a></li><li class="theme-doc-sidebar-item-link theme-doc-sidebar-item-link-level-1 menu__list-item"><a href="https://graph.openaire.eu/support" target="_blank" rel="noopener noreferrer" class="menu__link menuExternalLink_NmtK">Helpdesk<svg width="13.5" height="13.5" aria-hidden="true" viewBox="0 0 24 24" class="iconExternalLink_nPIU"><path fill="currentColor" d="M21 13v10h-21v-19h12v2h-10v15h17v-8h2zm3-12h-10.988l4.035 4-6.977 7.07 2.828 2.828 6.977-7.07 4.125 4.172v-11z"></path></svg></a></li></ul></nav></div></aside><main class="docMainContainer_gTbr"><div class="container padding-top--md padding-bottom--lg"><div class="row"><div class="col docItemCol_VOVn"><div class="theme-doc-version-banner alert alert--warning margin-bottom--md" role="alert"><div>This is documentation for <!-- -->OpenAIRE Graph Documentation<!-- --> <b>6.1.1</b>, which is no longer actively maintained.</div><div class="margin-top--md">For up-to-date documentation, see the <b><a href="/docs/graph-production-workflow/cleaning">latest version</a></b> (<!-- -->9.0.0<!-- -->).</div></div><div class="docItemContainer_Djhp"><article><nav class="theme-doc-breadcrumbs breadcrumbsContainer_Z_bl" aria-label="Breadcrumbs"><ul class="breadcrumbs" itemscope="" itemtype="https://schema.org/BreadcrumbList"><li class="breadcrumbs__item"><a aria-label="Home page" class="breadcrumbs__link" href="/docs/"><svg viewBox="0 0 24 24" class="breadcrumbHomeIcon_OVgt"><path d="M10 19v-5h4v5c0 .55.45 1 1 1h3c.55 0 1-.45 1-1v-7h1.7c.46 0 .68-.57.33-.87L12.67 3.6c-.38-.34-.96-.34-1.34 0l-8.36 7.53c-.34.3-.13.87.33.87H5v7c0 .55.45 1 1 1h3c.55 0 1-.45 1-1z" fill="currentColor"></path></svg></a></li><li itemscope="" itemprop="itemListElement" itemtype="https://schema.org/ListItem" class="breadcrumbs__item"><a class="breadcrumbs__link" itemprop="item" href="/docs/6.1.1/graph-production-workflow/"><span itemprop="name">Graph production workflow</span></a><meta itemprop="position" content="1"></li><li itemscope="" itemprop="itemListElement" itemtype="https://schema.org/ListItem" class="breadcrumbs__item breadcrumbs__item--active"><span class="breadcrumbs__link" itemprop="name">Cleaning</span><meta itemprop="position" content="2"></li></ul></nav><span class="theme-doc-version-badge badge badge--secondary">Version: 6.1.1</span><div class="theme-doc-markdown markdown"><h1>Cleaning</h1><p>The aggregation processes run independently one from another and continuously. Each aggregation process, depending on the characteristics of the records exposed by the data source, makes use of one or more vocabularies to harmonise the values available in a given field. In this page, we describe the <em>vocabulary-based cleaning</em> operation performed to harmonise the data of the different data sources. A vocabulary is a data structure that defines a list of terms, and for each term defines a list of synonyms:</p><div class="language-xml codeBlockContainer_Ckt0 theme-code-block" style="--prism-color:#393A34;--prism-background-color:#f6f8fa"><div class="codeBlockContent_biex"><pre tabindex="0" class="prism-code language-xml codeBlock_bY9V thin-scrollbar"><code class="codeBlockLines_e6Vv"><span class="token-line" style="color:#393A34"><span class="token tag punctuation" style="color:#393A34"><</span><span class="token tag" style="color:#00009f">TERMS</span><span class="token tag punctuation" style="color:#393A34">></span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain"> </span><span class="token tag punctuation" style="color:#393A34"><</span><span class="token tag" style="color:#00009f">TERM</span><span class="token tag" style="color:#00009f"> </span><span class="token tag attr-name" style="color:#00a4db">native_name</span><span class="token tag attr-value punctuation attr-equals" style="color:#393A34">=</span><span class="token tag attr-value punctuation" style="color:#393A34">"</span><span class="token tag attr-value" style="color:#e3116c">Annotation</span><span class="token tag attr-value punctuation" style="color:#393A34">"</span><span class="token tag" style="color:#00009f"> </span><span class="token tag attr-name" style="color:#00a4db">code</span><span class="token tag attr-value punctuation attr-equals" style="color:#393A34">=</span><span class="token tag attr-value punctuation" style="color:#393A34">"</span><span class="token tag attr-value" style="color:#e3116c">0018</span><span class="token tag attr-value punctuation" style="color:#393A34">"</span><span class="token tag" style="color:#00009f"> </span><span class="token tag attr-name" style="color:#00a4db">english_name</span><span class="token tag attr-value punctuation attr-equals" style="color:#393A34">=</span><span class="token tag attr-value punctuation" style="color:#393A34">"</span><span class="token tag attr-value" style="color:#e3116c">Annotation</span><span class="token tag attr-value punctuation" style="color:#393A34">"</span><span class="token tag" style="color:#00009f"> </span><span class="token tag attr-name" style="color:#00a4db">encoding</span><span class="token tag attr-value punctuation attr-equals" style="color:#393A34">=</span><span class="token tag attr-value punctuation" style="color:#393A34">"</span><span class="token tag attr-value" style="color:#e3116c">OPENAIRE</span><span class="token tag attr-value punctuation" style="color:#393A34">"</span><span class="token tag punctuation" style="color:#393A34">></span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain"> </span><span class="token tag punctuation" style="color:#393A34"><</span><span class="token tag" style="color:#00009f">SYNONYMS</span><span class="token tag punctuation" style="color:#393A34">></span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain"> </span><span class="token tag punctuation" style="color:#393A34"><</span><span class="token tag" style="color:#00009f">SYNONYM</span><span class="token tag" style="color:#00009f"> </span><span class="token tag attr-name" style="color:#00a4db">term</span><span class="token tag attr-value punctuation attr-equals" style="color:#393A34">=</span><span class="token tag attr-value punctuation" style="color:#393A34">"</span><span class="token tag attr-value" style="color:#e3116c">Comentario</span><span class="token tag attr-value punctuation" style="color:#393A34">"</span><span class="token tag" style="color:#00009f"> </span><span class="token tag attr-name" style="color:#00a4db">encoding</span><span class="token tag attr-value punctuation attr-equals" style="color:#393A34">=</span><span class="token tag attr-value punctuation" style="color:#393A34">"</span><span class="token tag attr-value" style="color:#e3116c">CSIC</span><span class="token tag attr-value punctuation" style="color:#393A34">"</span><span class="token tag punctuation" style="color:#393A34">/></span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain"> </span><span class="token tag punctuation" style="color:#393A34"><</span><span class="token tag" style="color:#00009f">SYNONYM</span><span class="token tag" style="color:#00009f"> </span><span class="token tag attr-name" style="color:#00a4db">term</span><span class="token tag attr-value punctuation attr-equals" style="color:#393A34">=</span><span class="token tag attr-value punctuation" style="color:#393A34">"</span><span class="token tag attr-value" style="color:#e3116c">Comment/debate</span><span class="token tag attr-value punctuation" style="color:#393A34">"</span><span class="token tag" style="color:#00009f"> </span><span class="token tag attr-name" style="color:#00a4db">encoding</span><span class="token tag attr-value punctuation attr-equals" style="color:#393A34">=</span><span class="token tag attr-value punctuation" style="color:#393A34">"</span><span class="token tag attr-value" style="color:#e3116c">Aaltodoc Publication Archive</span><span class="token tag attr-value punctuation" style="color:#393A34">"</span><span class="token tag punctuation" style="color:#393A34">/></span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain"> </span><span class="token tag punctuation" style="color:#393A34"><</span><span class="token tag" style="color:#00009f">SYNONYM</span><span class="token tag" style="color:#00009f"> </span><span class="token tag attr-name" style="color:#00a4db">term</span><span class="token tag attr-value punctuation attr-equals" style="color:#393A34">=</span><span class="token tag attr-value punctuation" style="color:#393A34">"</span><span class="token tag attr-value" style="color:#e3116c">annotation</span><span class="token tag attr-value punctuation" style="color:#393A34">"</span><span class="token tag" style="color:#00009f"> </span><span class="token tag attr-name" style="color:#00a4db">encoding</span><span class="token tag attr-value punctuation attr-equals" style="color:#393A34">=</span><span class="token tag attr-value punctuation" style="color:#393A34">"</span><span class="token tag attr-value" style="color:#e3116c">OPENAIRE-PR202112</span><span class="token tag attr-value punctuation" style="color:#393A34">"</span><span class="token tag punctuation" style="color:#393A34">/></span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain"> [...]</span><br></span><span class="token-line" style="color:#393A34"><span class="token plain"> </span><span class="token tag punctuation" style="color:#393A34"></</span><span class="token tag" style="color:#00009f">SYNONYMS</span><span class="token tag punctuation" style="color:#393A34">></span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain"> </span><span class="token tag punctuation" style="color:#393A34"><</span><span class="token tag" style="color:#00009f">RELATIONS</span><span class="token tag punctuation" style="color:#393A34">/></span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain"> </span><span class="token tag punctuation" style="color:#393A34"></</span><span class="token tag" style="color:#00009f">TERM</span><span class="token tag punctuation" style="color:#393A34">></span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain"> </span><span class="token tag punctuation" style="color:#393A34"><</span><span class="token tag" style="color:#00009f">TERM</span><span class="token tag" style="color:#00009f"> </span><span class="token tag attr-name" style="color:#00a4db">native_name</span><span class="token tag attr-value punctuation attr-equals" style="color:#393A34">=</span><span class="token tag attr-value punctuation" style="color:#393A34">"</span><span class="token tag attr-value" style="color:#e3116c">Article</span><span class="token tag attr-value punctuation" style="color:#393A34">"</span><span class="token tag" style="color:#00009f"> </span><span class="token tag attr-name" style="color:#00a4db">code</span><span class="token tag attr-value punctuation attr-equals" style="color:#393A34">=</span><span class="token tag attr-value punctuation" style="color:#393A34">"</span><span class="token tag attr-value" style="color:#e3116c">0001</span><span class="token tag attr-value punctuation" style="color:#393A34">"</span><span class="token tag" style="color:#00009f"> </span><span class="token tag attr-name" style="color:#00a4db">english_name</span><span class="token tag attr-value punctuation attr-equals" style="color:#393A34">=</span><span class="token tag attr-value punctuation" style="color:#393A34">"</span><span class="token tag attr-value" style="color:#e3116c">Article</span><span class="token tag attr-value punctuation" style="color:#393A34">"</span><span class="token tag" style="color:#00009f"> </span><span class="token tag attr-name" style="color:#00a4db">encoding</span><span class="token tag attr-value punctuation attr-equals" style="color:#393A34">=</span><span class="token tag attr-value punctuation" style="color:#393A34">"</span><span class="token tag attr-value" style="color:#e3116c">OPENAIRE</span><span class="token tag attr-value punctuation" style="color:#393A34">"</span><span class="token tag punctuation" style="color:#393A34">></span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain"> </span><span class="token tag punctuation" style="color:#393A34"><</span><span class="token tag" style="color:#00009f">SYNONYMS</span><span class="token tag punctuation" style="color:#393A34">></span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain"> </span><span class="token tag punctuation" style="color:#393A34"><</span><span class="token tag" style="color:#00009f">SYNONYM</span><span class="token tag" style="color:#00009f"> </span><span class="token tag attr-name" style="color:#00a4db">term</span><span class="token tag attr-value punctuation attr-equals" style="color:#393A34">=</span><span class="token tag attr-value punctuation" style="color:#393A34">"</span><span class="token tag attr-value" style="color:#e3116c">A1 Alkuper盲isartikkeli tieteellisess盲 aikakauslehdess盲</span><span class="token tag attr-value punctuation" style="color:#393A34">"</span><span class="token tag" style="color:#00009f"> </span><span class="token tag attr-name" style="color:#00a4db">encoding</span><span class="token tag attr-value punctuation attr-equals" style="color:#393A34">=</span><span class="token tag attr-value punctuation" style="color:#393A34">"</span><span class="token tag attr-value" style="color:#e3116c">Aaltodoc Publication Archive</span><span class="token tag attr-value punctuation" style="color:#393A34">"</span><span class="token tag punctuation" style="color:#393A34">/></span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain"> </span><span class="token tag punctuation" style="color:#393A34"><</span><span class="token tag" style="color:#00009f">SYNONYM</span><span class="token tag" style="color:#00009f"> </span><span class="token tag attr-name" style="color:#00a4db">term</span><span class="token tag attr-value punctuation attr-equals" style="color:#393A34">=</span><span class="token tag attr-value punctuation" style="color:#393A34">"</span><span class="token tag attr-value" style="color:#e3116c">A4 Artikkeli konferenssijulkaisussa</span><span class="token tag attr-value punctuation" style="color:#393A34">"</span><span class="token tag" style="color:#00009f"> </span><span class="token tag attr-name" style="color:#00a4db">encoding</span><span class="token tag attr-value punctuation attr-equals" style="color:#393A34">=</span><span class="token tag attr-value punctuation" style="color:#393A34">"</span><span class="token tag attr-value" style="color:#e3116c">Aaltodoc Publication Archive</span><span class="token tag attr-value punctuation" style="color:#393A34">"</span><span class="token tag punctuation" style="color:#393A34">/></span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain"> </span><span class="token tag punctuation" style="color:#393A34"><</span><span class="token tag" style="color:#00009f">SYNONYM</span><span class="token tag" style="color:#00009f"> </span><span class="token tag attr-name" style="color:#00a4db">term</span><span class="token tag attr-value punctuation attr-equals" style="color:#393A34">=</span><span class="token tag attr-value punctuation" style="color:#393A34">"</span><span class="token tag attr-value" style="color:#e3116c">Article</span><span class="token tag attr-value punctuation" style="color:#393A34">"</span><span class="token tag" style="color:#00009f"> </span><span class="token tag attr-name" style="color:#00a4db">encoding</span><span class="token tag attr-value punctuation attr-equals" style="color:#393A34">=</span><span class="token tag attr-value punctuation" style="color:#393A34">"</span><span class="token tag attr-value" style="color:#e3116c">OTHER</span><span class="token tag attr-value punctuation" style="color:#393A34">"</span><span class="token tag punctuation" style="color:#393A34">/></span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain"> </span><span class="token tag punctuation" style="color:#393A34"><</span><span class="token tag" style="color:#00009f">SYNONYM</span><span class="token tag" style="color:#00009f"> </span><span class="token tag attr-name" style="color:#00a4db">term</span><span class="token tag attr-value punctuation attr-equals" style="color:#393A34">=</span><span class="token tag attr-value punctuation" style="color:#393A34">"</span><span class="token tag attr-value" style="color:#e3116c">Article (author)</span><span class="token tag attr-value punctuation" style="color:#393A34">"</span><span class="token tag" style="color:#00009f"> </span><span class="token tag attr-name" style="color:#00a4db">encoding</span><span class="token tag attr-value punctuation attr-equals" style="color:#393A34">=</span><span class="token tag attr-value punctuation" style="color:#393A34">"</span><span class="token tag attr-value" style="color:#e3116c">OTHER</span><span class="token tag attr-value punctuation" style="color:#393A34">"</span><span class="token tag punctuation" style="color:#393A34">/></span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain"> [...]</span><br></span></code></pre><div class="buttonGroup__atx"><button type="button" aria-label="Copy code to clipboard" title="Copy" class="clean-btn"><span class="copyButtonIcons_eSgA" aria-hidden="true"><svg class="copyButtonIcon_y97N" viewBox="0 0 24 24"><path d="M19,21H8V7H19M19,5H8A2,2 0 0,0 6,7V21A2,2 0 0,0 8,23H19A2,2 0 0,0 21,21V7A2,2 0 0,0 19,5M16,1H4A2,2 0 0,0 2,3V17H4V3H16V1Z"></path></svg><svg class="copyButtonSuccessIcon_LjdS" viewBox="0 0 24 24"><path d="M21,7L9,19L3.5,13.5L4.91,12.09L9,16.17L19.59,5.59L21,7Z"></path></svg></span></button></div></div></div><p>Each vocabulary is typically used to control and harmonise the values available in a specific field characterising the bibliographic records. The example above provides a preview of the vocabulary used to clean the <a href="/docs/6.1.1/data-model/entities/result#instance">result's instance typology</a>.</p><p>The content of the vocabularies can be accessed on <a href="https://api.openaire.eu/vocabularies/" target="_blank" rel="noopener noreferrer">api.openaire.eu/vocabularies</a>.</p><p>Given a value provided in the original records, the cleaning process looks for a synonym and, when found, resolves the corresponding term which is used in turn to build the cleaned record. Each aggregation process applies vocabularies according to their definitions in a given moment of time, however, it could be the case that a vocabulary changes after the aggregation of one data source has finished, thus the aggregated content does not reflect the current status of the controlled vocabularies.</p><p>In addition, the integration of ScholeXplorer and DOIBoost and some enrichment processes applied on the raw and on the de-duplicated graph may introduce values that do not comply with the current status of the OpenAIRE controlled vocabularies. For these reasons, we included a final step of cleansing at the end of the workflow materialisation.</p></div></article><nav class="pagination-nav docusaurus-mt-lg" aria-label="Docs pages navigation"><a class="pagination-nav__link pagination-nav__link--prev" href="/docs/6.1.1/graph-production-workflow/enrichment-by-mining/metadata_extraction"><div class="pagination-nav__sublabel">Previous</div><div class="pagination-nav__label">Metadata extraction</div></a><a class="pagination-nav__link pagination-nav__link--next" href="/docs/6.1.1/graph-production-workflow/deduplication/"><div class="pagination-nav__sublabel">Next</div><div class="pagination-nav__label">Deduplication</div></a></nav></div></div></div></div></main></div></div><footer class="footer"><div class="container container-fluid"><div class="footer__bottom text--center"><div class="footer__copyright">Copyright 漏 2024 OpenAIRE</div></div></div></footer></div> <script src="/docs/assets/js/runtime~main.17d936df.js"></script> <script src="/docs/assets/js/main.42758c1f.js"></script> </body> </html>