CINXE.COM

<!DOCTYPE html> <html xmlns="http://www.w3.org/1999/xhtml" xmlns:dc="http://purl.org/dc/terms/" xmlns:doi="http://dx.doi.org/" lang="en" xml:lang="en" itemscope itemtype="http://schema.org/Article" class="no-js"> <head prefix="og: http://ogp.me/ns#"> <link rel="stylesheet" href="/resource/css/screen.css?79f248ebefa43b7800a14562e5049ab4"/>   <link rel="stylesheet" type="text/css" href="https://fonts.googleapis.com/css?family=Open+Sans:400,400i,600"> <link media="print" rel="stylesheet" type="text/css" href="/resource/css/print.css"/> <script type="text/javascript"> var siteUrlPrefix = "/digitalhealth/"; </script> <script src="/resource/js/vendor/modernizr-v2.7.1.js" type="text/javascript"></script> <script src="/resource/js/vendor/detectizr.min.js" type="text/javascript"></script> <link rel="shortcut icon" href="/resource/img/favicon.ico" type="image/x-icon"/> <meta http-equiv="Content-Type" content="text/html; charset=utf-8"/> <link rel="canonical" href="https://journals.plos.org/digitalhealth/article?id=10.1371/journal.pdig.0000115" /> <meta name="description" content="Author Summary The methodological aim of this study was to take advantage of missing data chronology in the imputation process in order to handle missing time series data. The practical goal of COVID application was to study the link between the numbers of chronological COVID confirmed cases and death. To achieve these goals we proposed 3 imputation methods of missing time series data each of them associated with an uncertainty model. For the COVID number of death prediction task, we set up a non-linear regression modeling for the number of COVID deaths prediction from past deaths and confirmed cases data. This led us to extend the Evidential K-Nearest Neighbor method to regression problems and to assess the impact of uncertainty modeling within imputation process in regards to predictive task. Finally, we showed the superiority of the time-EKNN (TEKNN) in terms of predictive performances compared to the Last Observation Carried Forward (LOCF) and Centered Moving Average (CMA) methods. More globally, we showed the interest of modeling the uncertainty in the imputation process to better control the prediction error, especially during relative stable periods." /> <meta name="citation_abstract" content="The object of this study is to put forward uncertainty modeling associated with missing time series data imputation in a predictive context. We propose three imputation methods associated with uncertainty modeling. These methods are evaluated on a COVID-19 dataset out of which some values have been randomly removed. The dataset contains the numbers of daily COVID-19 confirmed diagnoses (“new cases”) and daily deaths (“new deaths”) recorded since the start of the pandemic up to July 2021. The considered task is to predict the number of new deaths 7 days in advance. The more values are missing, the higher the imputation impact is on the predictive performances. The Evidential K-Nearest Neighbors (EKNN) algorithm is used for its ability to take into account labels uncertainty. Experiments are provided to measure the benefits of the label uncertainty models. Results show the positive impact of uncertainty models on imputation performances, especially in a noisy context where the number of missing values is high."> <meta name="keywords" content="Forecasting,COVID 19,Machine learning,Machine learning algorithms,Decision making,Pandemics,Probability distribution,Time measurement" /> <meta name="citation_doi" content="10.1371/journal.pdig.0000115"/> <meta name="citation_author" content="Rayane Elimam"/> <meta name="citation_author_institution" content="EuroMov Digital Health in Motion, Univ Montpellier, IMT Mines Ales, Ales, France"/> <meta name="citation_author" content="Nicolas Sutton-Charani"/> <meta name="citation_author_institution" content="EuroMov Digital Health in Motion, Univ Montpellier, IMT Mines Ales, Ales, France"/> <meta name="citation_author" content="Stéphane Perrey"/> <meta name="citation_author_institution" content="EuroMov Digital Health in Motion, Univ Montpellier, IMT Mines Ales, Ales, France"/> <meta name="citation_author" content="Jacky Montmain"/> <meta name="citation_author_institution" content="EuroMov Digital Health in Motion, Univ Montpellier, IMT Mines Ales, Ales, France"/> <meta name="citation_title" content="Uncertain imputation for time-series forecasting: Application to COVID-19 daily mortality prediction"/> <meta itemprop="name" content="Uncertain imputation for time-series forecasting: Application to COVID-19 daily mortality prediction"/> <meta name="citation_journal_title" content="PLOS Digital Health"/> <meta name="citation_journal_abbrev" content="PLOS Digital Health"/> <meta name="citation_date" content="Oct 25, 2022"/> <meta name="citation_firstpage" content="e0000115"/> <meta name="citation_issue" content="10"/> <meta name="citation_volume" content="1"/> <meta name="citation_issn" content="2767-3170"/> <meta name="citation_publisher" content="Public Library of Science"/> <meta name="citation_pdf_url" content="https://journals.plos.org/digitalhealth/article/file?id=10.1371/journal.pdig.0000115&type=printable"> <meta name="citation_article_type" content="Research Article"> <meta name="dc.identifier" content="10.1371/journal.pdig.0000115" /> <meta name="twitter:card" content="summary" /> <meta name="twitter:site" content="PLOSDigiHealth"/> <meta name="twitter:title" content="Uncertain imputation for time-series forecasting: Application to COVID-19 daily mortality prediction" /> <meta property="twitter:description" content="Author Summary The methodological aim of this study was to take advantage of missing data chronology in the imputation process in order to handle missing time series data. The practical goal of COVID application was to study the link between the numbers of chronological COVID confirmed cases and death. To achieve these goals we proposed 3 imputation methods of missing time series data each of them associated with an uncertainty model. For the COVID number of death prediction task, we set up a non-linear regression modeling for the number of COVID deaths prediction from past deaths and confirmed cases data. This led us to extend the Evidential K-Nearest Neighbor method to regression problems and to assess the impact of uncertainty modeling within imputation process in regards to predictive task. Finally, we showed the superiority of the time-EKNN (TEKNN) in terms of predictive performances compared to the Last Observation Carried Forward (LOCF) and Centered Moving Average (CMA) methods. More globally, we showed the interest of modeling the uncertainty in the imputation process to better control the prediction error, especially during relative stable periods." /> <meta property="twitter:image" content="https://journals.plos.org/digitalhealth/article/figure/image?id=10.1371/journal.pdig.0000115.g013&size=inline" /> <meta property="og:type" content="article" /> <meta property="og:url" content="https://journals.plos.org/digitalhealth/article?id=10.1371/journal.pdig.0000115"/> <meta property="og:title" content="Uncertain imputation for time-series forecasting: Application to COVID-19 daily mortality prediction"/> <meta property="og:description" content="Author Summary The methodological aim of this study was to take advantage of missing data chronology in the imputation process in order to handle missing time series data. The practical goal of COVID application was to study the link between the numbers of chronological COVID confirmed cases and death. To achieve these goals we proposed 3 imputation methods of missing time series data each of them associated with an uncertainty model. For the COVID number of death prediction task, we set up a non-linear regression modeling for the number of COVID deaths prediction from past deaths and confirmed cases data. This led us to extend the Evidential K-Nearest Neighbor method to regression problems and to assess the impact of uncertainty modeling within imputation process in regards to predictive task. Finally, we showed the superiority of the time-EKNN (TEKNN) in terms of predictive performances compared to the Last Observation Carried Forward (LOCF) and Centered Moving Average (CMA) methods. More globally, we showed the interest of modeling the uncertainty in the imputation process to better control the prediction error, especially during relative stable periods."/> <meta property="og:image" content="https://journals.plos.org/digitalhealth/article/figure/image?id=10.1371/journal.pdig.0000115.g013&size=inline"/> <meta name="citation_reference" content="citation_title=Missing data imputation using statistical and machine learning methods in a real breast cancer problem;citation_author=J. Jerez;citation_author=I. Molina;citation_author=P. García-Laencina;citation_author=E. Alba;citation_author=N. Ribelles;citation_author=L Martín, M., Franco;citation_journal_title=Artificial Intelligence In Medicine;citation_volume=50;citation_number=50;citation_issue=2;citation_first_page=105;citation_last_page=115;citation_publication_date=2010;"/> <meta name="citation_reference" content="citation_title=Imputation of missing data in industrial databases;citation_author=K. Lakshminaryan;citation_author=S. Harp;citation_author=T Samed;citation_journal_title=Applied Intelligence;citation_volume=11;citation_number=11;citation_issue=3;citation_first_page=259;citation_last_page=275;citation_publication_date=1999;"/> <meta name="citation_reference" content="citation_title=Inference and missing data;citation_author=D Rubin;citation_journal_title=Biometrika;citation_volume=63;citation_number=63;citation_issue=3;citation_first_page=581;citation_last_page=592;citation_publication_date=1976;"/> <meta name="citation_reference" content="citation_title=Statistical analysis with missing data;citation_author=R. Little;citation_author=D. Rubin;citation_publication_date=2002;citation_publisher=Wiley"/> <meta name="citation_reference" content="citation_title=Impact of imputation of missing values on classification error for discrete data;citation_author=A. Farhangfar;citation_author=L. Kurgan;citation_author=J Dy;citation_journal_title=Pattern Recognition;citation_volume=41;citation_number=41;citation_issue=12;citation_first_page=3692;citation_last_page=3705;citation_publication_date=2008;"/> <meta name="citation_reference" content="citation_title=Evidential classification of incomplete data via imprecise relabelling: Application to plastic sorting;citation_author=L. Jacquin;citation_author=A. Imoussaten;citation_author=F. Trousset;citation_author=J. Montmain;citation_author=D Perrin;citation_journal_title=International Conference On Scalable Uncertainty Management;citation_first_page=122;citation_last_page=135;citation_publication_date=2019;"/> <meta name="citation_reference" content="citation_title=Handling of uncertainty in medical data using machine learning and probability theory techniques: A review of 30 years (1991–2020);citation_author=R. Alizadehsani;citation_author=M. Roshanzamir;citation_author=S. Hussain;citation_author=A. Khosravi;citation_author=A. Koohestani;citation_author=M. Zangooei;citation_author=M. Abdar;citation_author=A. Beykikhoshk;citation_author=A. Shoeibi;citation_author=A. Zare;citation_author=M. Panahiazar;citation_author=S. Nahavandi;citation_author=D. Srinivasan;citation_author=A. Atiya;citation_author=U Acharya;citation_journal_title=Annals Of Operations Research;citation_first_page=1;citation_last_page=42;citation_publication_date=2021;"/> <meta name="citation_reference" content="citation_title=Fuzzy logic, neural networks, and soft computing;citation_author=L Zadeh;citation_journal_title=Fuzzy Sets, Fuzzy Logic, And Fuzzy Systems: Selected Papers By Lotfi A Zadeh;citation_first_page=775;citation_last_page=782;citation_publication_date=1996;"/> <meta name="citation_reference" content="citation_title=Fuzzy Sets;citation_author=L. Zadeh;citation_journal_title=Information And Control;citation_volume=8;citation_number=8;citation_first_page=338;citation_last_page=353;citation_publication_date=1965;"/> <meta name="citation_reference" content="citation_title=Possibility theory: qualitative and quantitative aspects;citation_author=D. Dubois;citation_author=H Prade;citation_journal_title=Quantified Representation Of Uncertainty And Imprecision;citation_first_page=169;citation_last_page=226;citation_publication_date=1998;"/> <meta name="citation_reference" content="Walley, P. Statistical reasoning with imprecise probabilities. (Chapman,1991). ISBN: 0412286602 9780412286605."/> <meta name="citation_reference" content="citation_title=Upper and lower probabilities induced by a multivalued mapping;citation_author=A Dempster;citation_journal_title=Classic Works Of The Dempster-Shafer Theory Of Belief Functions;citation_first_page=57;citation_last_page=72;citation_publication_date=2008;"/> <meta name="citation_reference" content="Shafer, G. A mathematical theory of evidence. (Princeton university press,1976)"/> <meta name="citation_reference" content="citation_title=A k-nearest neighbor classification rule based on Dempster-Shafer theory;citation_author=T Denœux;citation_journal_title=IEEE Transactions On Systems, Man, And Cybernetics;citation_volume=25;citation_number=25;citation_issue=5;citation_first_page=804;citation_last_page=813;citation_publication_date=1995;"/> <meta name="citation_reference" content="citation_title=A neural network classifier based on Dempster-Shafer theory;citation_author=T Denœux;citation_journal_title=IEEE Transactions On Systems, Man, And Cybernetics-Part A: Systems And Humans;citation_volume=30;citation_number=30;citation_issue=2;citation_first_page=131;citation_last_page=150;citation_publication_date=2001;"/> <meta name="citation_reference" content="citation_title=Belief decision trees: theoretical foundations;citation_author=Z. Elouedi;citation_author=K. Mellouli;citation_author=P Smets;citation_journal_title=International Journal Of Approximate Reasoning;citation_volume=28;citation_number=28;citation_issue=2-3;citation_first_page=91;citation_last_page=124;citation_publication_date=2001;"/> <meta name="citation_reference" content="Sutton-Charani, N., Destercke, S., Denœux, T. Learning decision trees from uncertain data with an evidential EM approach. 2013 12th International Conference On Machine Learning And Applications. 2013;1:111-116."/> <meta name="citation_reference" content="Azim, S. &amp; Aggarwal, S. Hybrid model for data imputation: using fuzzy c means and multi layer perceptron. 2014 IEEE International Advance Computing Conference (IACC) . 2014; 1281-1285"/> <meta name="citation_reference" content="citation_title=Towards missing data imputation: a study of fuzzy k-means clustering method;citation_author=D. Li;citation_author=J. Deogun;citation_author=W. Spaulding;citation_author=B Shuart;citation_journal_title=International Conference On Rough Sets And Current Trends In Computing;citation_first_page=573;citation_last_page=579;citation_publication_date=2004;"/> <meta name="citation_reference" content="citation_title=Probabilistic neural network based categorical data imputation;citation_author=K. Nishanth;citation_author=V Ravi;citation_journal_title=Neurocomputing;citation_volume=218;citation_number=218;citation_first_page=17;citation_last_page=25;citation_publication_date=2016;"/> <meta name="citation_reference" content="citation_title=Enhancing data analysis: uncertainty-resistance method for handling incomplete data;citation_author=J. Hamizadeh;citation_author=M Moradi;citation_journal_title=Applied Intelligence;citation_volume=50;citation_number=50;citation_issue=1;citation_first_page=74;citation_last_page=86;citation_publication_date=2020;"/> <meta name="citation_reference" content="Liu, Z., Pan, Q., Mercier, G., Dezert, J. Pattern classification with missing data using belief functions. 17th International Conference On Information Fusion (FUSION 2014;1-8, https://hal-onera.archives-ouvertes.fr/hal-01070496 ."/> <meta name="citation_reference" content="citation_title=Comparison of uncertainty representations for missing data in information retrieval;citation_author=A. Jousselme;citation_author=P Maupin;citation_journal_title=Proceedings Of The 16th International Conference On Information Fusion;citation_first_page=1902;citation_last_page=1909;citation_publication_date=2013;"/> <meta name="citation_reference" content="citation_title=Decision tree classifiers for evidential attribute values and class labels;citation_author=A. Trabelsi;citation_author=Z. Elouedi;citation_author=E Lefevre;citation_journal_title=Fuzzy Sets And Systems;citation_volume=366;citation_number=366;citation_first_page=46;citation_last_page=62;citation_publication_date=2019;"/> <meta name="citation_reference" content="citation_title=Decision trees for uncertain data;citation_author=S. Tsang;citation_author=B. Kao;citation_author=K. Yip;citation_author=W. Ho;citation_author=S Lee;citation_journal_title=IEEE Transactions On Knowledge And Data Engineering;citation_volume=23;citation_number=23;citation_issue=1;citation_first_page=64;citation_last_page=78;citation_publication_date=2009;"/> <meta name="citation_reference" content="citation_title=An evidential k-nearest neighbor classification method with weighted attributes;citation_author=L. Jiao;citation_author=Q. Pan;citation_author=X. Feng;citation_author=F Yang;citation_journal_title=Proceedings Of The 16th International Conference On Information Fusion;citation_first_page=145;citation_last_page=150;citation_publication_date=2013;"/> <meta name="citation_reference" content="citation_title=Robust combination rules for evidence theory;citation_author=M. Florea;citation_author=A. Jousselme;citation_author=D Bossé, É., Grenier;citation_journal_title=Information Fusion;citation_volume=10;citation_number=10;citation_issue=2;citation_first_page=183;citation_last_page=197;citation_publication_date=2009;"/> <meta name="citation_reference" content="citation_title=Belief functions: the disjunctive rule of combination and the generalized Bayesian theorem;citation_author=P Smets;citation_journal_title=International Journal Of Approximate Reasoning;citation_volume=9;citation_number=9;citation_issue=1;citation_first_page=1;citation_last_page=35;citation_publication_date=1993;"/> <meta name="citation_reference" content="citation_title=The transferable belief model;citation_author=P. Smets;citation_author=R Kennes;citation_journal_title=Artificial Intelligence;citation_volume=66;citation_number=66;citation_issue=2;citation_first_page=191;citation_last_page=234;citation_publication_date=1994;"/> <meta name="citation_reference" content="citation_title=Constructing the Pignistic Probability Function in a Context of Uncertainty;citation_author=P Smets;citation_journal_title=UAI;citation_volume=89;citation_number=89;citation_first_page=29;citation_last_page=40;citation_publication_date=1989;"/> <meta name="citation_reference" content="citation_title=Decision making in the TBM: the necessity of the pignistic transformation;citation_author=P Smets;citation_journal_title=International Journal Of Approximate Reasoning;citation_volume=38;citation_number=38;citation_issue=2;citation_first_page=133;citation_last_page=147;citation_publication_date=2005;"/> <meta name="citation_reference" content="citation_title=Evidential deep neural networks for uncertain data classification;citation_author=B. Yuan;citation_author=X. Yue;citation_author=Y. Lv;citation_author=T Denœux;citation_journal_title=International Conference On Knowledge Science, Engineering And Management;citation_first_page=427;citation_last_page=437;citation_publication_date=2020;"/>  <script type='text/javascript'> var googletag = googletag || {}; googletag.cmd = googletag.cmd || []; (function() { var gads = document.createElement('script'); gads.async = true; gads.type = 'text/javascript'; var useSSL = 'https:' == document.location.protocol; gads.src = (useSSL ? 'https:' : 'http:') + '//www.googletagservices.com/tag/js/gpt.js'; var node = document.getElementsByTagName('script')[0]; node.parentNode.insertBefore(gads, node); })(); </script>  <script id="doubleClickSetupScript" type='text/javascript'> googletag.cmd.push(function() { googletag.defineSlot('/75507958/PDIG_728x90_ATF', [728, 90], 'div-gpt-ad-1458247671871-0').addService(googletag.pubads()); googletag.defineSlot('/75507958/PDIG_160x600_BTF', [160, 600], 'div-gpt-ad-1458247671871-1').addService(googletag.pubads()); var personalizedAds = window.plosCookieConsent && window.plosCookieConsent.hasConsented('advertising'); googletag.pubads().setRequestNonPersonalizedAds(personalizedAds ? 0 : 1); googletag.pubads().enableSingleRequest(); googletag.enableServices(); }); </script> <script type="text/javascript"> var WombatConfig = WombatConfig || {}; WombatConfig.journalKey = "PLOSDigitalHealth"; WombatConfig.journalName = "PLOS Digital Health"; WombatConfig.figurePath = "/digitalhealth/article/figure/image"; WombatConfig.figShareInstitutionString = "plos"; WombatConfig.doiResolverPrefix = "https://dx.plos.org/"; </script> <script type="text/javascript"> var WombatConfig = WombatConfig || {}; WombatConfig.metrics = WombatConfig.metrics || {}; WombatConfig.metrics.referenceUrl = "http://lagotto.io/plos"; WombatConfig.metrics.googleScholarUrl = "https://scholar.google.com/scholar"; WombatConfig.metrics.googleScholarCitationUrl = WombatConfig.metrics.googleScholarUrl + "?hl=en&lr=&q="; WombatConfig.metrics.crossrefUrl = "https://www.crossref.org"; </script> <script defer="defer" src="/resource/js/defer.js?13928eb59791c3cc61cf"></script><script src="/resource/js/sync.js?13928eb59791c3cc61cf"></script> <script src="/resource/js/vendor/jquery.min.js" type="text/javascript"></script> <script type="text/javascript" src="https://widgets.figshare.com/static/figshare.js"></script> <script src="/resource/js/vendor/fastclick/lib/fastclick.js" type="text/javascript"></script> <script src="/resource/js/vendor/foundation/foundation.js" type="text/javascript"></script> <script src="/resource/js/vendor/underscore-min.js" type="text/javascript"></script> <script src="/resource/js/vendor/underscore.string.min.js" type="text/javascript"></script> <script src="/resource/js/vendor/moment.js" type="text/javascript"></script> <script src="/resource/js/vendor/jquery-ui-effects.min.js" type="text/javascript"></script> <script src="/resource/js/vendor/foundation/foundation.tooltip.js" type="text/javascript"></script> <script src="/resource/js/vendor/foundation/foundation.dropdown.js" type="text/javascript"></script> <script src="/resource/js/vendor/foundation/foundation.tab.js" type="text/javascript"></script> <script src="/resource/js/vendor/foundation/foundation.reveal.js" type="text/javascript"></script> <script src="/resource/js/vendor/foundation/foundation.slider.js" type="text/javascript"></script> <script src="/resource/js/util/utils.js" type="text/javascript"></script> <script src="/resource/js/components/toggle.js" type="text/javascript"></script> <script src="/resource/js/components/truncate_elem.js" type="text/javascript"></script> <script src="/resource/js/components/tooltip_hover.js" type="text/javascript"></script> <script src="/resource/js/vendor/jquery.dotdotdot.js" type="text/javascript"></script>  <script> dataLayer = [{ 'mobileSite': 'false', 'desktopSite': 'true' }]; </script> <title>Uncertain imputation for time-series forecasting: Application to COVID-19 daily mortality prediction | PLOS Digital Health</title> </head> <body class="article plosdigitalhealth">  <noscript><iframe src="//www.googletagmanager.com/ns.html?id=GTM-TP26BH" height="0" width="0" style="display:none;visibility:hidden"></iframe></noscript> <script> (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start': new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0], j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src= '//www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f); })(window,document,'script','dataLayer','GTM-TP26BH'); </script> <noscript><iframe src="//www.googletagmanager.com/ns.html?id=GTM-MQQMGF" height="0" width="0" style="display:none;visibility:hidden"></iframe></noscript> <script>(function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start': new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0], j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src= '//www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f); })(window,document,'script','dataLayer','GTM-MQQMGF');</script>   <script type="text/javascript"> ;window.NREUM||(NREUM={});NREUM.init={distributed_tracing:{enabled:true},privacy:{cookies_enabled:true},ajax:{deny_list:["bam.nr-data.net"]}}; window.NREUM||(NREUM={}),__nr_require=function(t,e,n){function r(n){if(!e[n]){var o=e[n]={exports:{}};t[n][0].call(o.exports,function(e){var o=t[n][1][e];return r(o||e)},o,o.exports)}return e[n].exports}if("function"==typeof __nr_require)return __nr_require;for(var o=0;o<n.length;o++)r(n[o]);return r}({1:[function(t,e,n){function r(t){try{s.console&&console.log(t)}catch(e){}}var o,i=t("ee"),a=t(32),s={};try{o=localStorage.getItem("__nr_flags").split(","),console&&"function"==typeof console.log&&(s.console=!0,o.indexOf("dev")!==-1&&(s.dev=!0),o.indexOf("nr_dev")!==-1&&(s.nrDev=!0))}catch(c){}s.nrDev&&i.on("internal-error",function(t){r(t.stack)}),s.dev&&i.on("fn-err",function(t,e,n){r(n.stack)}),s.dev&&(r("NR AGENT IN DEVELOPMENT MODE"),r("flags: "+a(s,function(t,e){return t}).join(", ")))},{}],2:[function(t,e,n){function r(t,e,n,r,s){try{l?l-=1:o(s||new UncaughtException(t,e,n),!0)}catch(f){try{i("ierr",[f,c.now(),!0])}catch(d){}}return"function"==typeof u&&u.apply(this,a(arguments))}function UncaughtException(t,e,n){this.message=t||"Uncaught error with no additional information",this.sourceURL=e,this.line=n}function o(t,e){var n=e?null:c.now();i("err",[t,n])}var i=t("handle"),a=t(33),s=t("ee"),c=t("loader"),f=t("gos"),u=window.onerror,d=!1,p="nr@seenError";if(!c.disabled){var l=0;c.features.err=!0,t(1),window.onerror=r;try{throw new Error}catch(h){"stack"in h&&(t(14),t(13),"addEventListener"in window&&t(7),c.xhrWrappable&&t(15),d=!0)}s.on("fn-start",function(t,e,n){d&&(l+=1)}),s.on("fn-err",function(t,e,n){d&&!n[p]&&(f(n,p,function(){return!0}),this.thrown=!0,o(n))}),s.on("fn-end",function(){d&&!this.thrown&&l>0&&(l-=1)}),s.on("internal-error",function(t){i("ierr",[t,c.now(),!0])})}},{}],3:[function(t,e,n){var r=t("loader");r.disabled||(r.features.ins=!0)},{}],4:[function(t,e,n){function r(){U++,L=g.hash,this[u]=y.now()}function o(){U--,g.hash!==L&&i(0,!0);var t=y.now();this[h]=~~this[h]+t-this[u],this[d]=t}function i(t,e){E.emit("newURL",[""+g,e])}function a(t,e){t.on(e,function(){this[e]=y.now()})}var s="-start",c="-end",f="-body",u="fn"+s,d="fn"+c,p="cb"+s,l="cb"+c,h="jsTime",m="fetch",v="addEventListener",w=window,g=w.location,y=t("loader");if(w[v]&&y.xhrWrappable&&!y.disabled){var x=t(11),b=t(12),E=t(9),R=t(7),O=t(14),T=t(8),P=t(15),S=t(10),M=t("ee"),N=M.get("tracer"),C=t(23);t(17),y.features.spa=!0;var L,U=0;M.on(u,r),b.on(p,r),S.on(p,r),M.on(d,o),b.on(l,o),S.on(l,o),M.buffer([u,d,"xhr-resolved"]),R.buffer([u]),O.buffer(["setTimeout"+c,"clearTimeout"+s,u]),P.buffer([u,"new-xhr","send-xhr"+s]),T.buffer([m+s,m+"-done",m+f+s,m+f+c]),E.buffer(["newURL"]),x.buffer([u]),b.buffer(["propagate",p,l,"executor-err","resolve"+s]),N.buffer([u,"no-"+u]),S.buffer(["new-jsonp","cb-start","jsonp-error","jsonp-end"]),a(T,m+s),a(T,m+"-done"),a(S,"new-jsonp"),a(S,"jsonp-end"),a(S,"cb-start"),E.on("pushState-end",i),E.on("replaceState-end",i),w[v]("hashchange",i,C(!0)),w[v]("load",i,C(!0)),w[v]("popstate",function(){i(0,U>1)},C(!0))}},{}],5:[function(t,e,n){function r(){var t=new PerformanceObserver(function(t,e){var n=t.getEntries();s(v,[n])});try{t.observe({entryTypes:["resource"]})}catch(e){}}function o(t){if(s(v,[window.performance.getEntriesByType(w)]),window.performance["c"+p])try{window.performance[h](m,o,!1)}catch(t){}else try{window.performance[h]("webkit"+m,o,!1)}catch(t){}}function i(t){}if(window.performance&&window.performance.timing&&window.performance.getEntriesByType){var a=t("ee"),s=t("handle"),c=t(14),f=t(13),u=t(6),d=t(23),p="learResourceTimings",l="addEventListener",h="removeEventListener",m="resourcetimingbufferfull",v="bstResource",w="resource",g="-start",y="-end",x="fn"+g,b="fn"+y,E="bstTimer",R="pushState",O=t("loader");if(!O.disabled){O.features.stn=!0,t(9),"addEventListener"in window&&t(7);var T=NREUM.o.EV;a.on(x,function(t,e){var n=t[0];n instanceof T&&(this.bstStart=O.now())}),a.on(b,function(t,e){var n=t[0];n instanceof T&&s("bst",[n,e,this.bstStart,O.now()])}),c.on(x,function(t,e,n){this.bstStart=O.now(),this.bstType=n}),c.on(b,function(t,e){s(E,[e,this.bstStart,O.now(),this.bstType])}),f.on(x,function(){this.bstStart=O.now()}),f.on(b,function(t,e){s(E,[e,this.bstStart,O.now(),"requestAnimationFrame"])}),a.on(R+g,function(t){this.time=O.now(),this.startPath=location.pathname+location.hash}),a.on(R+y,function(t){s("bstHist",[location.pathname+location.hash,this.startPath,this.time])}),u()?(s(v,[window.performance.getEntriesByType("resource")]),r()):l in window.performance&&(window.performance["c"+p]?window.performance[l](m,o,d(!1)):window.performance[l]("webkit"+m,o,d(!1))),document[l]("scroll",i,d(!1)),document[l]("keypress",i,d(!1)),document[l]("click",i,d(!1))}}},{}],6:[function(t,e,n){e.exports=function(){return"PerformanceObserver"in window&&"function"==typeof window.PerformanceObserver}},{}],7:[function(t,e,n){function r(t){for(var e=t;e&&!e.hasOwnProperty(u);)e=Object.getPrototypeOf(e);e&&o(e)}function o(t){s.inPlace(t,[u,d],"-",i)}function i(t,e){return t[1]}var a=t("ee").get("events"),s=t("wrap-function")(a,!0),c=t("gos"),f=XMLHttpRequest,u="addEventListener",d="removeEventListener";e.exports=a,"getPrototypeOf"in Object?(r(document),r(window),r(f.prototype)):f.prototype.hasOwnProperty(u)&&(o(window),o(f.prototype)),a.on(u+"-start",function(t,e){var n=t[1];if(null!==n&&("function"==typeof n||"object"==typeof n)){var r=c(n,"nr@wrapped",function(){function t(){if("function"==typeof n.handleEvent)return n.handleEvent.apply(n,arguments)}var e={object:t,"function":n}[typeof n];return e?s(e,"fn-",null,e.name||"anonymous"):n});this.wrapped=t[1]=r}}),a.on(d+"-start",function(t){t[1]=this.wrapped||t[1]})},{}],8:[function(t,e,n){function r(t,e,n){var r=t[e];"function"==typeof r&&(t[e]=function(){var t=i(arguments),e={};o.emit(n+"before-start",[t],e);var a;e[m]&&e[m].dt&&(a=e[m].dt);var s=r.apply(this,t);return o.emit(n+"start",[t,a],s),s.then(function(t){return o.emit(n+"end",[null,t],s),t},function(t){throw o.emit(n+"end",[t],s),t})})}var o=t("ee").get("fetch"),i=t(33),a=t(32);e.exports=o;var s=window,c="fetch-",f=c+"body-",u=["arrayBuffer","blob","json","text","formData"],d=s.Request,p=s.Response,l=s.fetch,h="prototype",m="nr@context";d&&p&&l&&(a(u,function(t,e){r(d[h],e,f),r(p[h],e,f)}),r(s,"fetch",c),o.on(c+"end",function(t,e){var n=this;if(e){var r=e.headers.get("content-length");null!==r&&(n.rxSize=r),o.emit(c+"done",[null,e],n)}else o.emit(c+"done",[t],n)}))},{}],9:[function(t,e,n){var r=t("ee").get("history"),o=t("wrap-function")(r);e.exports=r;var i=window.history&&window.history.constructor&&window.history.constructor.prototype,a=window.history;i&&i.pushState&&i.replaceState&&(a=i),o.inPlace(a,["pushState","replaceState"],"-")},{}],10:[function(t,e,n){function r(t){function e(){f.emit("jsonp-end",[],l),t.removeEventListener("load",e,c(!1)),t.removeEventListener("error",n,c(!1))}function n(){f.emit("jsonp-error",[],l),f.emit("jsonp-end",[],l),t.removeEventListener("load",e,c(!1)),t.removeEventListener("error",n,c(!1))}var r=t&&"string"==typeof t.nodeName&&"script"===t.nodeName.toLowerCase();if(r){var o="function"==typeof t.addEventListener;if(o){var a=i(t.src);if(a){var d=s(a),p="function"==typeof d.parent[d.key];if(p){var l={};u.inPlace(d.parent,[d.key],"cb-",l),t.addEventListener("load",e,c(!1)),t.addEventListener("error",n,c(!1)),f.emit("new-jsonp",[t.src],l)}}}}}function o(){return"addEventListener"in window}function i(t){var e=t.match(d);return e?e[1]:null}function a(t,e){var n=t.match(l),r=n[1],o=n[3];return o?a(o,e[r]):e[r]}function s(t){var e=t.match(p);return e&&e.length>=3?{key:e[2],parent:a(e[1],window)}:{key:t,parent:window}}var c=t(23),f=t("ee").get("jsonp"),u=t("wrap-function")(f);if(e.exports=f,o()){var d=/[?&](?:callback|cb)=([^&#]+)/,p=/(.*)\.([^.]+)/,l=/^(\w+)(\.|$)(.*)$/,h=["appendChild","insertBefore","replaceChild"];Node&&Node.prototype&&Node.prototype.appendChild?u.inPlace(Node.prototype,h,"dom-"):(u.inPlace(HTMLElement.prototype,h,"dom-"),u.inPlace(HTMLHeadElement.prototype,h,"dom-"),u.inPlace(HTMLBodyElement.prototype,h,"dom-")),f.on("dom-start",function(t){r(t[0])})}},{}],11:[function(t,e,n){var r=t("ee").get("mutation"),o=t("wrap-function")(r),i=NREUM.o.MO;e.exports=r,i&&(window.MutationObserver=function(t){return this instanceof i?new i(o(t,"fn-")):i.apply(this,arguments)},MutationObserver.prototype=i.prototype)},{}],12:[function(t,e,n){function r(t){var e=i.context(),n=s(t,"executor-",e,null,!1),r=new f(n);return i.context(r).getCtx=function(){return e},r}var o=t("wrap-function"),i=t("ee").get("promise"),a=t("ee").getOrSetContext,s=o(i),c=t(32),f=NREUM.o.PR;e.exports=i,f&&(window.Promise=r,["all","race"].forEach(function(t){var e=f[t];f[t]=function(n){function r(t){return function(){i.emit("propagate",[null,!o],a,!1,!1),o=o||!t}}var o=!1;c(n,function(e,n){Promise.resolve(n).then(r("all"===t),r(!1))});var a=e.apply(f,arguments),s=f.resolve(a);return s}}),["resolve","reject"].forEach(function(t){var e=f[t];f[t]=function(t){var n=e.apply(f,arguments);return t!==n&&i.emit("propagate",[t,!0],n,!1,!1),n}}),f.prototype["catch"]=function(t){return this.then(null,t)},f.prototype=Object.create(f.prototype,{constructor:{value:r}}),c(Object.getOwnPropertyNames(f),function(t,e){try{r[e]=f[e]}catch(n){}}),o.wrapInPlace(f.prototype,"then",function(t){return function(){var e=this,n=o.argsToArray.apply(this,arguments),r=a(e);r.promise=e,n[0]=s(n[0],"cb-",r,null,!1),n[1]=s(n[1],"cb-",r,null,!1);var c=t.apply(this,n);return r.nextPromise=c,i.emit("propagate",[e,!0],c,!1,!1),c}}),i.on("executor-start",function(t){t[0]=s(t[0],"resolve-",this,null,!1),t[1]=s(t[1],"resolve-",this,null,!1)}),i.on("executor-err",function(t,e,n){t[1](n)}),i.on("cb-end",function(t,e,n){i.emit("propagate",[n,!0],this.nextPromise,!1,!1)}),i.on("propagate",function(t,e,n){this.getCtx&&!e||(this.getCtx=function(){if(t instanceof Promise)var e=i.context(t);return e&&e.getCtx?e.getCtx():this})}),r.toString=function(){return""+f})},{}],13:[function(t,e,n){var r=t("ee").get("raf"),o=t("wrap-function")(r),i="equestAnimationFrame";e.exports=r,o.inPlace(window,["r"+i,"mozR"+i,"webkitR"+i,"msR"+i],"raf-"),r.on("raf-start",function(t){t[0]=o(t[0],"fn-")})},{}],14:[function(t,e,n){function r(t,e,n){t[0]=a(t[0],"fn-",null,n)}function o(t,e,n){this.method=n,this.timerDuration=isNaN(t[1])?0:+t[1],t[0]=a(t[0],"fn-",this,n)}var i=t("ee").get("timer"),a=t("wrap-function")(i),s="setTimeout",c="setInterval",f="clearTimeout",u="-start",d="-";e.exports=i,a.inPlace(window,[s,"setImmediate"],s+d),a.inPlace(window,[c],c+d),a.inPlace(window,[f,"clearImmediate"],f+d),i.on(c+u,r),i.on(s+u,o)},{}],15:[function(t,e,n){function r(t,e){d.inPlace(e,["onreadystatechange"],"fn-",s)}function o(){var t=this,e=u.context(t);t.readyState>3&&!e.resolved&&(e.resolved=!0,u.emit("xhr-resolved",[],t)),d.inPlace(t,y,"fn-",s)}function i(t){x.push(t),m&&(E?E.then(a):w?w(a):(R=-R,O.data=R))}function a(){for(var t=0;t<x.length;t++)r([],x[t]);x.length&&(x=[])}function s(t,e){return e}function c(t,e){for(var n in t)e[n]=t[n];return e}t(7);var f=t("ee"),u=f.get("xhr"),d=t("wrap-function")(u),p=t(23),l=NREUM.o,h=l.XHR,m=l.MO,v=l.PR,w=l.SI,g="readystatechange",y=["onload","onerror","onabort","onloadstart","onloadend","onprogress","ontimeout"],x=[];e.exports=u;var b=window.XMLHttpRequest=function(t){var e=new h(t);try{u.emit("new-xhr",[e],e),e.addEventListener(g,o,p(!1))}catch(n){try{u.emit("internal-error",[n])}catch(r){}}return e};if(c(h,b),b.prototype=h.prototype,d.inPlace(b.prototype,["open","send"],"-xhr-",s),u.on("send-xhr-start",function(t,e){r(t,e),i(e)}),u.on("open-xhr-start",r),m){var E=v&&v.resolve();if(!w&&!v){var R=1,O=document.createTextNode(R);new m(a).observe(O,{characterData:!0})}}else f.on("fn-end",function(t){t[0]&&t[0].type===g||a()})},{}],16:[function(t,e,n){function r(t){if(!s(t))return null;var e=window.NREUM;if(!e.loader_config)return null;var n=(e.loader_config.accountID||"").toString()||null,r=(e.loader_config.agentID||"").toString()||null,f=(e.loader_config.trustKey||"").toString()||null;if(!n||!r)return null;var h=l.generateSpanId(),m=l.generateTraceId(),v=Date.now(),w={spanId:h,traceId:m,timestamp:v};return(t.sameOrigin||c(t)&&p())&&(w.traceContextParentHeader=o(h,m),w.traceContextStateHeader=i(h,v,n,r,f)),(t.sameOrigin&&!u()||!t.sameOrigin&&c(t)&&d())&&(w.newrelicHeader=a(h,m,v,n,r,f)),w}function o(t,e){return"00-"+e+"-"+t+"-01"}function i(t,e,n,r,o){var i=0,a="",s=1,c="",f="";return o+"@nr="+i+"-"+s+"-"+n+"-"+r+"-"+t+"-"+a+"-"+c+"-"+f+"-"+e}function a(t,e,n,r,o,i){var a="btoa"in window&&"function"==typeof window.btoa;if(!a)return null;var s={v:[0,1],d:{ty:"Browser",ac:r,ap:o,id:t,tr:e,ti:n}};return i&&r!==i&&(s.d.tk=i),btoa(JSON.stringify(s))}function s(t){return f()&&c(t)}function c(t){var e=!1,n={};if("init"in NREUM&&"distributed_tracing"in NREUM.init&&(n=NREUM.init.distributed_tracing),t.sameOrigin)e=!0;else if(n.allowed_origins instanceof Array)for(var r=0;r<n.allowed_origins.length;r++){var o=h(n.allowed_origins[r]);if(t.hostname===o.hostname&&t.protocol===o.protocol&&t.port===o.port){e=!0;break}}return e}function f(){return"init"in NREUM&&"distributed_tracing"in NREUM.init&&!!NREUM.init.distributed_tracing.enabled}function u(){return"init"in NREUM&&"distributed_tracing"in NREUM.init&&!!NREUM.init.distributed_tracing.exclude_newrelic_header}function d(){return"init"in NREUM&&"distributed_tracing"in NREUM.init&&NREUM.init.distributed_tracing.cors_use_newrelic_header!==!1}function p(){return"init"in NREUM&&"distributed_tracing"in NREUM.init&&!!NREUM.init.distributed_tracing.cors_use_tracecontext_headers}var l=t(29),h=t(18);e.exports={generateTracePayload:r,shouldGenerateTrace:s}},{}],17:[function(t,e,n){function r(t){var e=this.params,n=this.metrics;if(!this.ended){this.ended=!0;for(var r=0;r<p;r++)t.removeEventListener(d[r],this.listener,!1);e.aborted||(n.duration=a.now()-this.startTime,this.loadCaptureCalled||4!==t.readyState?null==e.status&&(e.status=0):i(this,t),n.cbTime=this.cbTime,s("xhr",[e,n,this.startTime,this.endTime,"xhr"],this))}}function o(t,e){var n=c(e),r=t.params;r.hostname=n.hostname,r.port=n.port,r.protocol=n.protocol,r.host=n.hostname+":"+n.port,r.pathname=n.pathname,t.parsedOrigin=n,t.sameOrigin=n.sameOrigin}function i(t,e){t.params.status=e.status;var n=v(e,t.lastSize);if(n&&(t.metrics.rxSize=n),t.sameOrigin){var r=e.getResponseHeader("X-NewRelic-App-Data");r&&(t.params.cat=r.split(", ").pop())}t.loadCaptureCalled=!0}var a=t("loader");if(a.xhrWrappable&&!a.disabled){var s=t("handle"),c=t(18),f=t(16).generateTracePayload,u=t("ee"),d=["load","error","abort","timeout"],p=d.length,l=t("id"),h=t(24),m=t(22),v=t(19),w=t(23),g=NREUM.o.REQ,y=window.XMLHttpRequest;a.features.xhr=!0,t(15),t(8),u.on("new-xhr",function(t){var e=this;e.totalCbs=0,e.called=0,e.cbTime=0,e.end=r,e.ended=!1,e.xhrGuids={},e.lastSize=null,e.loadCaptureCalled=!1,e.params=this.params||{},e.metrics=this.metrics||{},t.addEventListener("load",function(n){i(e,t)},w(!1)),h&&(h>34||h<10)||t.addEventListener("progress",function(t){e.lastSize=t.loaded},w(!1))}),u.on("open-xhr-start",function(t){this.params={method:t[0]},o(this,t[1]),this.metrics={}}),u.on("open-xhr-end",function(t,e){"loader_config"in NREUM&&"xpid"in NREUM.loader_config&&this.sameOrigin&&e.setRequestHeader("X-NewRelic-ID",NREUM.loader_config.xpid);var n=f(this.parsedOrigin);if(n){var r=!1;n.newrelicHeader&&(e.setRequestHeader("newrelic",n.newrelicHeader),r=!0),n.traceContextParentHeader&&(e.setRequestHeader("traceparent",n.traceContextParentHeader),n.traceContextStateHeader&&e.setRequestHeader("tracestate",n.traceContextStateHeader),r=!0),r&&(this.dt=n)}}),u.on("send-xhr-start",function(t,e){var n=this.metrics,r=t[0],o=this;if(n&&r){var i=m(r);i&&(n.txSize=i)}this.startTime=a.now(),this.listener=function(t){try{"abort"!==t.type||o.loadCaptureCalled||(o.params.aborted=!0),("load"!==t.type||o.called===o.totalCbs&&(o.onloadCalled||"function"!=typeof e.onload))&&o.end(e)}catch(n){try{u.emit("internal-error",[n])}catch(r){}}};for(var s=0;s<p;s++)e.addEventListener(d[s],this.listener,w(!1))}),u.on("xhr-cb-time",function(t,e,n){this.cbTime+=t,e?this.onloadCalled=!0:this.called+=1,this.called!==this.totalCbs||!this.onloadCalled&&"function"==typeof n.onload||this.end(n)}),u.on("xhr-load-added",function(t,e){var n=""+l(t)+!!e;this.xhrGuids&&!this.xhrGuids[n]&&(this.xhrGuids[n]=!0,this.totalCbs+=1)}),u.on("xhr-load-removed",function(t,e){var n=""+l(t)+!!e;this.xhrGuids&&this.xhrGuids[n]&&(delete this.xhrGuids[n],this.totalCbs-=1)}),u.on("xhr-resolved",function(){this.endTime=a.now()}),u.on("addEventListener-end",function(t,e){e instanceof y&&"load"===t[0]&&u.emit("xhr-load-added",[t[1],t[2]],e)}),u.on("removeEventListener-end",function(t,e){e instanceof y&&"load"===t[0]&&u.emit("xhr-load-removed",[t[1],t[2]],e)}),u.on("fn-start",function(t,e,n){e instanceof y&&("onload"===n&&(this.onload=!0),("load"===(t[0]&&t[0].type)||this.onload)&&(this.xhrCbStart=a.now()))}),u.on("fn-end",function(t,e){this.xhrCbStart&&u.emit("xhr-cb-time",[a.now()-this.xhrCbStart,this.onload,e],e)}),u.on("fetch-before-start",function(t){function e(t,e){var n=!1;return e.newrelicHeader&&(t.set("newrelic",e.newrelicHeader),n=!0),e.traceContextParentHeader&&(t.set("traceparent",e.traceContextParentHeader),e.traceContextStateHeader&&t.set("tracestate",e.traceContextStateHeader),n=!0),n}var n,r=t[1]||{};"string"==typeof t[0]?n=t[0]:t[0]&&t[0].url?n=t[0].url:window.URL&&t[0]&&t[0]instanceof URL&&(n=t[0].href),n&&(this.parsedOrigin=c(n),this.sameOrigin=this.parsedOrigin.sameOrigin);var o=f(this.parsedOrigin);if(o&&(o.newrelicHeader||o.traceContextParentHeader))if("string"==typeof t[0]||window.URL&&t[0]&&t[0]instanceof URL){var i={};for(var a in r)i[a]=r[a];i.headers=new Headers(r.headers||{}),e(i.headers,o)&&(this.dt=o),t.length>1?t[1]=i:t.push(i)}else t[0]&&t[0].headers&&e(t[0].headers,o)&&(this.dt=o)}),u.on("fetch-start",function(t,e){this.params={},this.metrics={},this.startTime=a.now(),this.dt=e,t.length>=1&&(this.target=t[0]),t.length>=2&&(this.opts=t[1]);var n,r=this.opts||{},i=this.target;"string"==typeof i?n=i:"object"==typeof i&&i instanceof g?n=i.url:window.URL&&"object"==typeof i&&i instanceof URL&&(n=i.href),o(this,n);var s=(""+(i&&i instanceof g&&i.method||r.method||"GET")).toUpperCase();this.params.method=s,this.txSize=m(r.body)||0}),u.on("fetch-done",function(t,e){this.endTime=a.now(),this.params||(this.params={}),this.params.status=e?e.status:0;var n;"string"==typeof this.rxSize&&this.rxSize.length>0&&(n=+this.rxSize);var r={txSize:this.txSize,rxSize:n,duration:a.now()-this.startTime};s("xhr",[this.params,r,this.startTime,this.endTime,"fetch"],this)})}},{}],18:[function(t,e,n){var r={};e.exports=function(t){if(t in r)return r[t];var e=document.createElement("a"),n=window.location,o={};e.href=t,o.port=e.port;var i=e.href.split("://");!o.port&&i[1]&&(o.port=i[1].split("/")[0].split("@").pop().split(":")[1]),o.port&&"0"!==o.port||(o.port="https"===i[0]?"443":"80"),o.hostname=e.hostname||n.hostname,o.pathname=e.pathname,o.protocol=i[0],"/"!==o.pathname.charAt(0)&&(o.pathname="/"+o.pathname);var a=!e.protocol||":"===e.protocol||e.protocol===n.protocol,s=e.hostname===document.domain&&e.port===n.port;return o.sameOrigin=a&&(!e.hostname||s),"/"===o.pathname&&(r[t]=o),o}},{}],19:[function(t,e,n){function r(t,e){var n=t.responseType;return"json"===n&&null!==e?e:"arraybuffer"===n||"blob"===n||"json"===n?o(t.response):"text"===n||""===n||void 0===n?o(t.responseText):void 0}var o=t(22);e.exports=r},{}],20:[function(t,e,n){function r(){}function o(t,e,n,r){return function(){return u.recordSupportability("API/"+e+"/called"),i(t+e,[f.now()].concat(s(arguments)),n?null:this,r),n?void 0:this}}var i=t("handle"),a=t(32),s=t(33),c=t("ee").get("tracer"),f=t("loader"),u=t(25),d=NREUM;"undefined"==typeof window.newrelic&&(newrelic=d);var p=["setPageViewName","setCustomAttribute","setErrorHandler","finished","addToTrace","inlineHit","addRelease"],l="api-",h=l+"ixn-";a(p,function(t,e){d[e]=o(l,e,!0,"api")}),d.addPageAction=o(l,"addPageAction",!0),d.setCurrentRouteName=o(l,"routeName",!0),e.exports=newrelic,d.interaction=function(){return(new r).get()};var m=r.prototype={createTracer:function(t,e){var n={},r=this,o="function"==typeof e;return i(h+"tracer",[f.now(),t,n],r),function(){if(c.emit((o?"":"no-")+"fn-start",[f.now(),r,o],n),o)try{return e.apply(this,arguments)}catch(t){throw c.emit("fn-err",[arguments,this,t],n),t}finally{c.emit("fn-end",[f.now()],n)}}}};a("actionText,setName,setAttribute,save,ignore,onEnd,getContext,end,get".split(","),function(t,e){m[e]=o(h,e)}),newrelic.noticeError=function(t,e){"string"==typeof t&&(t=new Error(t)),u.recordSupportability("API/noticeError/called"),i("err",[t,f.now(),!1,e])}},{}],21:[function(t,e,n){function r(t){if(NREUM.init){for(var e=NREUM.init,n=t.split("."),r=0;r<n.length-1;r++)if(e=e[n[r]],"object"!=typeof e)return;return e=e[n[n.length-1]]}}e.exports={getConfiguration:r}},{}],22:[function(t,e,n){e.exports=function(t){if("string"==typeof t&&t.length)return t.length;if("object"==typeof t){if("undefined"!=typeof ArrayBuffer&&t instanceof ArrayBuffer&&t.byteLength)return t.byteLength;if("undefined"!=typeof Blob&&t instanceof Blob&&t.size)return t.size;if(!("undefined"!=typeof FormData&&t instanceof FormData))try{return JSON.stringify(t).length}catch(e){return}}}},{}],23:[function(t,e,n){var r=!1;try{var o=Object.defineProperty({},"passive",{get:function(){r=!0}});window.addEventListener("testPassive",null,o),window.removeEventListener("testPassive",null,o)}catch(i){}e.exports=function(t){return r?{passive:!0,capture:!!t}:!!t}},{}],24:[function(t,e,n){var r=0,o=navigator.userAgent.match(/Firefox[\/\s](\d+\.\d+)/);o&&(r=+o[1]),e.exports=r},{}],25:[function(t,e,n){function r(t,e){var n=[a,t,{name:t},e];return i("storeMetric",n,null,"api"),n}function o(t,e){var n=[s,t,{name:t},e];return i("storeEventMetrics",n,null,"api"),n}var i=t("handle"),a="sm",s="cm";e.exports={constants:{SUPPORTABILITY_METRIC:a,CUSTOM_METRIC:s},recordSupportability:r,recordCustom:o}},{}],26:[function(t,e,n){function r(){return s.exists&&performance.now?Math.round(performance.now()):(i=Math.max((new Date).getTime(),i))-a}function o(){return i}var i=(new Date).getTime(),a=i,s=t(34);e.exports=r,e.exports.offset=a,e.exports.getLastTimestamp=o},{}],27:[function(t,e,n){function r(t){return!(!t||!t.protocol||"file:"===t.protocol)}e.exports=r},{}],28:[function(t,e,n){function r(t,e){var n=t.getEntries();n.forEach(function(t){"first-paint"===t.name?p("timing",["fp",Math.floor(t.startTime)]):"first-contentful-paint"===t.name&&p("timing",["fcp",Math.floor(t.startTime)])})}function o(t,e){var n=t.getEntries();if(n.length>0){var r=n[n.length-1];if(c&&c<r.startTime)return;p("lcp",[r])}}function i(t){t.getEntries().forEach(function(t){t.hadRecentInput||p("cls",[t])})}function a(t){if(t instanceof v&&!g){var e=Math.round(t.timeStamp),n={type:t.type};e<=l.now()?n.fid=l.now()-e:e>l.offset&&e<=Date.now()?(e-=l.offset,n.fid=l.now()-e):e=l.now(),g=!0,p("timing",["fi",e,n])}}function s(t){"hidden"===t&&(c=l.now(),p("pageHide",[c]))}if(!("init"in NREUM&&"page_view_timing"in NREUM.init&&"enabled"in NREUM.init.page_view_timing&&NREUM.init.page_view_timing.enabled===!1)){var c,f,u,d,p=t("handle"),l=t("loader"),h=t(31),m=t(23),v=NREUM.o.EV;if("PerformanceObserver"in window&&"function"==typeof window.PerformanceObserver){f=new PerformanceObserver(r);try{f.observe({entryTypes:["paint"]})}catch(w){}u=new PerformanceObserver(o);try{u.observe({entryTypes:["largest-contentful-paint"]})}catch(w){}d=new PerformanceObserver(i);try{d.observe({type:"layout-shift",buffered:!0})}catch(w){}}if("addEventListener"in document){var g=!1,y=["click","keydown","mousedown","pointerdown","touchstart"];y.forEach(function(t){document.addEventListener(t,a,m(!1))})}h(s)}},{}],29:[function(t,e,n){function r(){function t(){return e?15&e[n++]:16*Math.random()|0}var e=null,n=0,r=window.crypto||window.msCrypto;r&&r.getRandomValues&&(e=r.getRandomValues(new Uint8Array(31)));for(var o,i="xxxxxxxx-xxxx-4xxx-yxxx-xxxxxxxxxxxx",a="",s=0;s<i.length;s++)o=i[s],"x"===o?a+=t().toString(16):"y"===o?(o=3&t()|8,a+=o.toString(16)):a+=o;return a}function o(){return a(16)}function i(){return a(32)}function a(t){function e(){return n?15&n[r++]:16*Math.random()|0}var n=null,r=0,o=window.crypto||window.msCrypto;o&&o.getRandomValues&&Uint8Array&&(n=o.getRandomValues(new Uint8Array(31)));for(var i=[],a=0;a<t;a++)i.push(e().toString(16));return i.join("")}e.exports={generateUuid:r,generateSpanId:o,generateTraceId:i}},{}],30:[function(t,e,n){function r(t,e){if(!o)return!1;if(t!==o)return!1;if(!e)return!0;if(!i)return!1;for(var n=i.split("."),r=e.split("."),a=0;a<r.length;a++)if(r[a]!==n[a])return!1;return!0}var o=null,i=null,a=/Version\/(\S+)\s+Safari/;if(navigator.userAgent){var s=navigator.userAgent,c=s.match(a);c&&s.indexOf("Chrome")===-1&&s.indexOf("Chromium")===-1&&(o="Safari",i=c[1])}e.exports={agent:o,version:i,match:r}},{}],31:[function(t,e,n){function r(t){function e(){t(s&&document[s]?document[s]:document[i]?"hidden":"visible")}"addEventListener"in document&&a&&document.addEventListener(a,e,o(!1))}var o=t(23);e.exports=r;var i,a,s;"undefined"!=typeof document.hidden?(i="hidden",a="visibilitychange",s="visibilityState"):"undefined"!=typeof document.msHidden?(i="msHidden",a="msvisibilitychange"):"undefined"!=typeof document.webkitHidden&&(i="webkitHidden",a="webkitvisibilitychange",s="webkitVisibilityState")},{}],32:[function(t,e,n){function r(t,e){var n=[],r="",i=0;for(r in t)o.call(t,r)&&(n[i]=e(r,t[r]),i+=1);return n}var o=Object.prototype.hasOwnProperty;e.exports=r},{}],33:[function(t,e,n){function r(t,e,n){e||(e=0),"undefined"==typeof n&&(n=t?t.length:0);for(var r=-1,o=n-e||0,i=Array(o<0?0:o);++r<o;)i[r]=t[e+r];return i}e.exports=r},{}],34:[function(t,e,n){e.exports={exists:"undefined"!=typeof window.performance&&window.performance.timing&&"undefined"!=typeof window.performance.timing.navigationStart}},{}],ee:[function(t,e,n){function r(){}function o(t){function e(t){return t&&t instanceof r?t:t?f(t,c,a):a()}function n(n,r,o,i,a){if(a!==!1&&(a=!0),!l.aborted||i){t&&a&&t(n,r,o);for(var s=e(o),c=m(n),f=c.length,u=0;u<f;u++)c[u].apply(s,r);var p=d[y[n]];return p&&p.push([x,n,r,s]),s}}function i(t,e){g[t]=m(t).concat(e)}function h(t,e){var n=g[t];if(n)for(var r=0;r<n.length;r++)n[r]===e&&n.splice(r,1)}function m(t){return g[t]||[]}function v(t){return p[t]=p[t]||o(n)}function w(t,e){l.aborted||u(t,function(t,n){e=e||"feature",y[n]=e,e in d||(d[e]=[])})}var g={},y={},x={on:i,addEventListener:i,removeEventListener:h,emit:n,get:v,listeners:m,context:e,buffer:w,abort:s,aborted:!1};return x}function i(t){return f(t,c,a)}function a(){return new r}function s(){(d.api||d.feature)&&(l.aborted=!0,d=l.backlog={})}var c="nr@context",f=t("gos"),u=t(32),d={},p={},l=e.exports=o();e.exports.getOrSetContext=i,l.backlog=d},{}],gos:[function(t,e,n){function r(t,e,n){if(o.call(t,e))return t[e];var r=n();if(Object.defineProperty&&Object.keys)try{return Object.defineProperty(t,e,{value:r,writable:!0,enumerable:!1}),r}catch(i){}return t[e]=r,r}var o=Object.prototype.hasOwnProperty;e.exports=r},{}],handle:[function(t,e,n){function r(t,e,n,r){o.buffer([t],r),o.emit(t,e,n)}var o=t("ee").get("handle");e.exports=r,r.ee=o},{}],id:[function(t,e,n){function r(t){var e=typeof t;return!t||"object"!==e&&"function"!==e?-1:t===window?0:a(t,i,function(){return o++})}var o=1,i="nr@id",a=t("gos");e.exports=r},{}],loader:[function(t,e,n){function r(){if(!P++){var t=T.info=NREUM.info,e=v.getElementsByTagName("script")[0];if(setTimeout(f.abort,3e4),!(t&&t.licenseKey&&t.applicationID&&e))return f.abort();c(R,function(e,n){t[e]||(t[e]=n)});var n=a();s("mark",["onload",n+T.offset],null,"api"),s("timing",["load",n]);var r=v.createElement("script");0===t.agent.indexOf("http://")||0===t.agent.indexOf("https://")?r.src=t.agent:r.src=h+"://"+t.agent,e.parentNode.insertBefore(r,e)}}function o(){"complete"===v.readyState&&i()}function i(){s("mark",["domContent",a()+T.offset],null,"api")}var a=t(26),s=t("handle"),c=t(32),f=t("ee"),u=t(30),d=t(27),p=t(21),l=t(23),h=p.getConfiguration("ssl")===!1?"http":"https",m=window,v=m.document,w="addEventListener",g="attachEvent",y=m.XMLHttpRequest,x=y&&y.prototype,b=!d(m.location);NREUM.o={ST:setTimeout,SI:m.setImmediate,CT:clearTimeout,XHR:y,REQ:m.Request,EV:m.Event,PR:m.Promise,MO:m.MutationObserver};var E=""+location,R={beacon:"bam.nr-data.net",errorBeacon:"bam.nr-data.net",agent:"js-agent.newrelic.com/nr-spa-1212.min.js"},O=y&&x&&x[w]&&!/CriOS/.test(navigator.userAgent),T=e.exports={offset:a.getLastTimestamp(),now:a,origin:E,features:{},xhrWrappable:O,userAgent:u,disabled:b};if(!b){t(20),t(28),v[w]?(v[w]("DOMContentLoaded",i,l(!1)),m[w]("load",r,l(!1))):(v[g]("onreadystatechange",o),m[g]("onload",r)),s("mark",["firstbyte",a.getLastTimestamp()],null,"api");var P=0}},{}],"wrap-function":[function(t,e,n){function r(t,e){function n(e,n,r,c,f){function nrWrapper(){var i,a,u,p;try{a=this,i=d(arguments),u="function"==typeof r?r(i,a):r||{}}catch(l){o([l,"",[i,a,c],u],t)}s(n+"start",[i,a,c],u,f);try{return p=e.apply(a,i)}catch(h){throw s(n+"err",[i,a,h],u,f),h}finally{s(n+"end",[i,a,p],u,f)}}return a(e)?e:(n||(n=""),nrWrapper[p]=e,i(e,nrWrapper,t),nrWrapper)}function r(t,e,r,o,i){r||(r="");var s,c,f,u="-"===r.charAt(0);for(f=0;f<e.length;f++)c=e[f],s=t[c],a(s)||(t[c]=n(s,u?c+r:r,o,c,i))}function s(n,r,i,a){if(!h||e){var s=h;h=!0;try{t.emit(n,r,i,e,a)}catch(c){o([c,n,r,i],t)}h=s}}return t||(t=u),n.inPlace=r,n.flag=p,n}function o(t,e){e||(e=u);try{e.emit("internal-error",t)}catch(n){}}function i(t,e,n){if(Object.defineProperty&&Object.keys)try{var r=Object.keys(t);return r.forEach(function(n){Object.defineProperty(e,n,{get:function(){return t[n]},set:function(e){return t[n]=e,e}})}),e}catch(i){o([i],n)}for(var a in t)l.call(t,a)&&(e[a]=t[a]);return e}function a(t){return!(t&&t instanceof Function&&t.apply&&!t[p])}function s(t,e){var n=e(t);return n[p]=t,i(t,n,u),n}function c(t,e,n){var r=t[e];t[e]=s(r,n)}function f(){for(var t=arguments.length,e=new Array(t),n=0;n<t;++n)e[n]=arguments[n];return e}var u=t("ee"),d=t(33),p="nr@original",l=Object.prototype.hasOwnProperty,h=!1;e.exports=r,e.exports.wrapFunction=s,e.exports.wrapInPlace=c,e.exports.argsToArray=f},{}]},{},["loader",2,17,5,3,4]); ;NREUM.loader_config={accountID:"804283",trustKey:"804283",agentID:"402703674",licenseKey:"cf99e8d2a3",applicationID:"402703674"} ;NREUM.info={beacon:"bam.nr-data.net",errorBeacon:"bam.nr-data.net",licenseKey:"cf99e8d2a3", // Modified this value from the generated script, to pass prod vs dev applicationID: window.location.hostname.includes('journals.plos.org') ? "402703674" : "402694889", sa:1} </script>  <header> <div id="topslot" class="head-top"> <a id="skip-to-content" tabindex="0" class="button" href="#main-content"> Skip to main content </a> <div class="center"> <div class="title">Advertisement</div>  <div class='advertisement' id='div-gpt-ad-1458247671871-0' style='width:728px; height:90px;'> <script type='text/javascript'> googletag.cmd.push(function() { googletag.display('div-gpt-ad-1458247671871-0'); }); </script> </div> </div> </div> <div id="user" class="nav" data-user-management-url="https://community.plos.org"> </div> <div id="pagehdr"> <nav class="nav-main"> <h1 class="logo"> <a href="/digitalhealth/.">PLOS Digital Health</a> </h1> <section class="top-bar-section"> <ul class="nav-elements"> <li class="multi-col-parent menu-section-header has-dropdown" id="publish"> Publish <div class="dropdown mega "> <ul class="multi-col" id="publish-dropdown-list"> <li class="menu-section-header " id="submissions"> <span class="menu-section-header-title"> Submissions </span> <ul class="menu-section " id="submissions-dropdown-list"> <li> <a href="/digitalhealth/s/submission-guidelines" >Submission Guidelines</a> </li> <li> <a href="/digitalhealth/s/figures" >Figures</a> </li> <li> <a href="/digitalhealth/s/tables" >Tables</a> </li> <li> <a href="/digitalhealth/s/supporting-information" >Supporting Information</a> </li> <li> <a href="/digitalhealth/s/latex" >LaTeX</a> </li> <li> <a href="/digitalhealth/s/what-we-publish" >What We Publish</a> </li> <li> <a href="/digitalhealth/s/preprints" >Preprints</a> </li> <li> <a href="/digitalhealth/s/revising-your-manuscript" >Revising Your Manuscript</a> </li> <li> <a href="/digitalhealth/s/submit-now" >Submit Now</a> </li> </ul> </li> <li class="menu-section-header " id="policies"> <span class="menu-section-header-title"> Policies </span> <ul class="menu-section " id="policies-dropdown-list"> <li> <a href="/digitalhealth/s/best-practices-in-research-reporting" >Best Practices in Research Reporting</a> </li> <li> <a href="/digitalhealth/s/human-subjects-research" >Human Subjects Research</a> </li> <li> <a href="/digitalhealth/s/animal-research" >Animal Research</a> </li> <li> <a href="/digitalhealth/s/competing-interests" >Competing Interests</a> </li> <li> <a href="/digitalhealth/s/disclosure-of-funding-sources" >Disclosure of Funding Sources</a> </li> <li> <a href="/digitalhealth/s/licenses-and-copyright" >Licenses and Copyright</a> </li> <li> <a href="/digitalhealth/s/data-availability" >Data Availability</a> </li> <li> <a href="/digitalhealth/s/complementary-research" >Complementary Research</a> </li> <li> <a href="/digitalhealth/s/materials-software-and-code-sharing" >Materials, Software and Code Sharing</a> </li> <li> <a href="/digitalhealth/s/ethical-publishing-practice" >Ethical Publishing Practice</a> </li> <li> <a href="/digitalhealth/s/authorship" >Authorship</a> </li> <li> <a href="/digitalhealth/s/corrections-expressions-of-concern-and-retractions" >Corrections, Expressions of Concern, and Retractions</a> </li> </ul> </li> <li class="menu-section-header " id="manuscript-review-and-publication"> <span class="menu-section-header-title"> Manuscript Review and Publication </span> <ul class="menu-section " id="manuscript-review-and-publication-dropdown-list"> <li> <a href="/digitalhealth/s/editorial-and-peer-review-process" >Editorial and Peer Review Process</a> </li> <li> <a href="https://plos.org/resources/editor-center" >Editor Center</a> </li> <li> <a href="/digitalhealth/s/reviewer-guidelines" >Guidelines for Reviewers</a> </li> <li> <a href="/digitalhealth/s/accepted-manuscripts" >Accepted Manuscripts</a> </li> <li> <a href="/digitalhealth/s/comments" >Comments</a> </li> <li> <a href="/digitalhealth/s/guidelines-for-editors" >Guidelines for Editors</a> </li> </ul> </li> </ul> <div class="calloutcontainer"> <h3 class="callout-headline">Submit Your Manuscript</h3> <div class="action-contain"> <p class="callout-content"> <em>PLOS Digital Health</em> publishes boundary-breaking research that uses digital tools, technologies, and data to advance every aspect of health care. </p> <p class="button-contain"> <a class="button button-default" href="/digitalhealth/s/submit-now"> Get Started </a> </p> </div>  </div> </div> </li> <li class="menu-section-header has-dropdown " id="about"> <span class="menu-section-header-title"> About </span> <ul class="menu-section dropdown " id="about-dropdown-list"> <li> <a href="/digitalhealth/s/journal-information" >Journal Information</a> </li> <li> <a href="/digitalhealth/s/editors-in-chief" >Editors-in-Chief</a> </li> <li> <a href="/digitalhealth/s/editorial-board" >Editorial Board</a> </li> <li> <a href="/digitalhealth/s/publishing-information" >Publishing Information</a> </li> <li> <a href="https://plos.org/publication-fees" >Publication Fees</a> </li> <li> <a href="https://plos.org/press-and-media" >Press and Media</a> </li> <li> <a href="/digitalhealth/s/contact" >Contact</a> </li> </ul> </li> <li class="menu-section-header ignore-top-bar-li-style"> <a href="/digitalhealth/search?sortOrder=DATE_NEWEST_FIRST&filterStartDate=2021-10-01&filterJournals=PLOSDigitalHealth&q=&resultsPerPage=60">Browse</a> </li> <script src="/resource/js/vendor/jquery.hoverIntent.js" type="text/javascript"></script> <script src="/resource/js/components/menu_drop.js" type="text/javascript"></script> <script src="/resource/js/components/hover_delay.js" type="text/javascript"></script> <li id="navsearch" class="head-search"> <form name="searchForm" action="/digitalhealth/search" method="get"> <fieldset> <legend>Search</legend> <label for="search">Search</label> <div class="search-contain"> <input id="search" type="text" name="q" placeholder="SEARCH" required/> <button id="headerSearchButton" type="submit" aria-label="Submit search"> <i title="Submit search" class="search-icon"></i> </button> </div> </fieldset> <input type="hidden" name="filterJournals" value="PLOSDigitalHealth"/> </form> <a id="advSearch" href="/digitalhealth/search"> advanced search </a> <script src="/resource/js/components/placeholder_style.js" type="text/javascript"></script> </li> </ul> </section> </nav> </div> </header> <main id="main-content"> <div class="set-grid"> <header class="title-block"> <script src="/resource/js/components/signposts.js" type="text/javascript"></script> <ul id="almSignposts" class="signposts"> <li id="loadingMetrics"> <p>Loading metrics</p> </li> </ul> <script type="text/template" id="signpostsGeneralErrorTemplate"> <li id="metricsError">Article metrics are unavailable at this time. Please try again later.</li> </script> <script type="text/template" id="signpostsNewArticleErrorTemplate"> <li></li><li></li><li id="tooSoon">Article metrics are unavailable for recently published articles.</li> </script> <script type="text/template" id="signpostsTemplate"> <li id="almSaves"> <%= s.numberFormat(saveCount, 0) %> <div class="tools" data-js-tooltip-hover="trigger"> <a class="metric-term" href="/digitalhealth/article/metrics?id=10.1371/journal.pdig.0000115#savedHeader">Save</a> <p class="saves-tip" data-js-tooltip-hover="target"><a href="/digitalhealth/article/metrics?id=10.1371/journal.pdig.0000115#savedHeader">Total Mendeley and Citeulike bookmarks.</a></p> </div> </li> <li id="almCitations"> <%= s.numberFormat(citationCount, 0) %> <div class="tools" data-js-tooltip-hover="trigger"> <a class="metric-term" href="/digitalhealth/article/metrics?id=10.1371/journal.pdig.0000115#citedHeader">Citation</a> <p class="citations-tip" data-js-tooltip-hover="target"><a href="/digitalhealth/article/metrics?id=10.1371/journal.pdig.0000115#citedHeader">Paper's citation count computed by Dimensions.</a></p> </div> </li> <li id="almViews"> <%= s.numberFormat(viewCount, 0) %> <div class="tools" data-js-tooltip-hover="trigger"> <a class="metric-term" href="/digitalhealth/article/metrics?id=10.1371/journal.pdig.0000115#viewedHeader">View</a> <p class="views-tip" data-js-tooltip-hover="target"><a href="/digitalhealth/article/metrics?id=10.1371/journal.pdig.0000115#viewedHeader">PLOS views and downloads.</a></p> </div> </li> <li id="almShares"> <%= s.numberFormat(shareCount, 0) %> <div class="tools" data-js-tooltip-hover="trigger"> <a class="metric-term" href="/digitalhealth/article/metrics?id=10.1371/journal.pdig.0000115#discussedHeader">Share</a> <p class="shares-tip" data-js-tooltip-hover="target"><a href="/digitalhealth/article/metrics?id=10.1371/journal.pdig.0000115#discussedHeader">Sum of Facebook, Twitter, Reddit and Wikipedia activity.</a></p> </div> </li> </script> <div class="article-meta"> <div class="classifications"> <p class="license-short" id="licenseShort">Open Access</p> <p class="peer-reviewed" id="peerReviewed">Peer-reviewed</p> <div class="article-type" > <p class="type-article" id="artType">Research Article</p> </div> </div> </div> <div class="article-title-etc"> <div class="title-authors"> <h1 id="artTitle"><?xml version="1.0" encoding="UTF-8"?>Uncertain imputation for time-series forecasting: Application to COVID-19 daily mortality prediction</h1> <ul class="author-list clearfix" data-js-tooltip="tooltip_container" id="author-list"> <li data-js-tooltip="tooltip_trigger" > <a data-author-id="0" class="author-name" > Rayane Elimam,</a> <div id="author-meta-0" class="author-info" data-js-tooltip="tooltip_target"> <p class="roles" id="authRoles"> <span class="type">Roles</span> Conceptualization, Data curation, Formal analysis, Methodology, Software, Writing – original draft </p> <p id="authAffiliations-0"><span class="type">Affiliation</span> EuroMov Digital Health in Motion, Univ Montpellier, IMT Mines Ales, Ales, France </p> <div> <p class="orcid" id="authOrcid-0"> <span> <a id="connect-orcid-link" href="https://orcid.org/0000-0003-4533-594X" target="_blank" title="ORCID Registry"> <img id="orcid-id-logo" src="/resource/img/orcid_16x16.png" width="16" height="16" alt="ORCID logo"/> https://orcid.org/0000-0003-4533-594X </a> </span> </p> </div> <a data-js-tooltip="tooltip_close" class="close" id="tooltipClose0"> ⨯ </a> </div> </li> <li data-js-tooltip="tooltip_trigger" > <a data-author-id="1" class="author-name" > Nicolas Sutton-Charani <span class="contribute"> </span> <span class="email"> </span>,</a> <div id="author-meta-1" class="author-info" data-js-tooltip="tooltip_target"> <p> <span class="contribute"> </span> Contributed equally to this work with: Nicolas Sutton-Charani, Stéphane Perrey, Jacky Montmain </p> <p class="roles" id="authRoles"> <span class="type">Roles</span> Conceptualization, Formal analysis, Supervision, Validation, Visualization, Writing – review & editing </p> <p id="authCorresponding-1"> <span class="email">* E-mail:</span> <a href="mailto:nicolas.sutton-charani@mines-ales.fr">nicolas.sutton-charani@mines-ales.fr</a></p> <p id="authAffiliations-1"><span class="type">Affiliation</span> EuroMov Digital Health in Motion, Univ Montpellier, IMT Mines Ales, Ales, France </p> <div> <p class="orcid" id="authOrcid-1"> <span> <a id="connect-orcid-link" href="https://orcid.org/0000-0002-3065-0712" target="_blank" title="ORCID Registry"> <img id="orcid-id-logo" src="/resource/img/orcid_16x16.png" width="16" height="16" alt="ORCID logo"/> https://orcid.org/0000-0002-3065-0712 </a> </span> </p> </div> <a data-js-tooltip="tooltip_close" class="close" id="tooltipClose1"> ⨯ </a> </div> </li> <li data-js-tooltip="tooltip_trigger" > <a data-author-id="2" class="author-name" > Stéphane Perrey <span class="contribute"> </span>,</a> <div id="author-meta-2" class="author-info" data-js-tooltip="tooltip_target"> <p> <span class="contribute"> </span> Contributed equally to this work with: Nicolas Sutton-Charani, Stéphane Perrey, Jacky Montmain </p> <p class="roles" id="authRoles"> <span class="type">Roles</span> Funding acquisition, Project administration, Supervision, Validation, Writing – review & editing </p> <p id="authAffiliations-2"><span class="type">Affiliation</span> EuroMov Digital Health in Motion, Univ Montpellier, IMT Mines Ales, Ales, France </p> <div> <p class="orcid" id="authOrcid-2"> <span> <a id="connect-orcid-link" href="https://orcid.org/0000-0002-8741-629X" target="_blank" title="ORCID Registry"> <img id="orcid-id-logo" src="/resource/img/orcid_16x16.png" width="16" height="16" alt="ORCID logo"/> https://orcid.org/0000-0002-8741-629X </a> </span> </p> </div> <a data-js-tooltip="tooltip_close" class="close" id="tooltipClose2"> ⨯ </a> </div> </li> <li data-js-tooltip="tooltip_trigger" > <a data-author-id="3" class="author-name" > Jacky Montmain <span class="contribute"> </span></a> <div id="author-meta-3" class="author-info" data-js-tooltip="tooltip_target"> <p> <span class="contribute"> </span> Contributed equally to this work with: Nicolas Sutton-Charani, Stéphane Perrey, Jacky Montmain </p> <p class="roles" id="authRoles"> <span class="type">Roles</span> Conceptualization, Formal analysis, Investigation, Methodology, Project administration, Supervision, Validation, Writing – review & editing </p> <p id="authAffiliations-3"><span class="type">Affiliation</span> EuroMov Digital Health in Motion, Univ Montpellier, IMT Mines Ales, Ales, France </p> <div> <p class="orcid" id="authOrcid-3"> <span> <a id="connect-orcid-link" href="https://orcid.org/0000-0003-0918-5788" target="_blank" title="ORCID Registry"> <img id="orcid-id-logo" src="/resource/img/orcid_16x16.png" width="16" height="16" alt="ORCID logo"/> https://orcid.org/0000-0003-0918-5788 </a> </span> </p> </div> <a data-js-tooltip="tooltip_close" class="close" id="tooltipClose3"> ⨯ </a> </div> </li> </ul> <script src="/resource/js/components/tooltip.js" type="text/javascript"></script> </div> <div id="floatTitleTop" data-js-floater="title_author" class="float-title" role="presentation"> <div class="set-grid"> <div class="float-title-inner"> <h1><?xml version="1.0" encoding="UTF-8"?>Uncertain imputation for time-series forecasting: Application to COVID-19 daily mortality prediction</h1> <ul id="floatAuthorList" data-js-floater="floated_authors"> <li data-float-index="1">Rayane Elimam,  </li> <li data-float-index="2">Nicolas Sutton-Charani,  </li> <li data-float-index="3">Stéphane Perrey,  </li> <li data-float-index="4">Jacky Montmain </li> </ul> </div> <div class="logo-close" id="titleTopCloser"> <img src="/resource/img/logo-plos.png" style="height: 2em" alt="PLOS" /> <div class="close-floater" title="close">x</div> </div> </div> </div> <ul class="date-doi"> <li id="artPubDate">Published: October 25, 2022</li> <li id="artDoi"> <a href="https://doi.org/10.1371/journal.pdig.0000115">https://doi.org/10.1371/journal.pdig.0000115</a> </li> <li class="flex-spacer"></li> </ul> </div> <div> </div> </header> <section class="article-body"> <ul class="article-tabs"> <li class="tab-title active" id="tabArticle"> <a href="/digitalhealth/article?id=10.1371/journal.pdig.0000115" class="article-tab-1">Article</a> </li> <li class="tab-title " id="tabAuthors"> <a href="/digitalhealth/article/authors?id=10.1371/journal.pdig.0000115" class="article-tab-2">Authors</a> </li> <li class="tab-title " id="tabMetrics"> <a href="/digitalhealth/article/metrics?id=10.1371/journal.pdig.0000115" class="article-tab-3">Metrics</a> </li> <li class="tab-title " id="tabComments"> <a href="/digitalhealth/article/comments?id=10.1371/journal.pdig.0000115" class="article-tab-4">Comments</a> </li> <li class="tab-title" id="tabRelated"> <a class="article-tab-5" id="tabRelated-link">Media Coverage</a> <script>$(document).ready(function() { $.getMediaLink("10.1371/journal.pdig.0000115").then(function (url) { $("#tabRelated-link").attr("href", url) } ) })</script> </li> </ul> <div class="article-container"> <div id="nav-article"> <ul class="nav-secondary"> <li class="nav-comments" id="nav-comments"> <a href="article/comments?id=10.1371/journal.pdig.0000115">Reader Comments</a> </li> <li id="nav-figures"><a href="#" data-doi="10.1371/journal.pdig.0000115">Figures</a></li> </ul> <div id="nav-data-linking" data-data-url=""> </div> </div> <script src="/resource/js/components/scroll.js" type="text/javascript"></script> <script src="/resource/js/components/nav_builder.js" type="text/javascript"></script> <script src="/resource/js/components/floating_nav.js" type="text/javascript"></script> <div id="figure-lightbox-container"></div> <script id="figure-lightbox-template" type="text/template"> <div id="figure-lightbox" class="reveal-modal full" data-reveal aria-hidden="true" role="dialog"> <div class="lb-header"> <h1 id="lb-title"><%= articleTitle %></h1> <div id="lb-authors"> <span>Rayane Elimam</span> <span>Nicolas Sutton-Charani</span> <span>Stéphane Perrey</span> <span>Jacky Montmain</span> </div> <div class="lb-close" title="close"> </div> </div> <div class="img-container"> <div class="loader"> <i class="fa-spinner"></i> </div> <img class="main-lightbox-image" src=""/> <aside id="figures-list"> <% figureList.each(function (ix, figure) { %> <div class="change-img" data-doi="<%= figure.getAttribute('data-doi') %>"> <img class="aside-figure" src="/digitalhealth/article/figure/image?size=inline&id=<%= figure.getAttribute('data-doi') %>" /> </div> <% }) %> <div class="dummy-figure"> </div> </aside> </div> <div id="lightbox-footer"> <div id="btns-container" class="lightbox-row <% if(figureList.length <= 1) { print('one-figure-only') } %>"> <div class="fig-btns-container reset-zoom-wrapper left"> <span class="fig-btn reset-zoom-btn">Reset zoom</span> </div> <div class="zoom-slider-container"> <div class="range-slider-container"> <span id="lb-zoom-min"></span> <div class="range-slider round" data-slider data-options="start: 20; end: 200; initial: 20;"> <span class="range-slider-handle" role="slider" tabindex="0"></span> <span class="range-slider-active-segment"></span> <input type="hidden"> </div> <span id="lb-zoom-max"></span> </div> </div> <% if(figureList.length > 1) { %> <div class="fig-btns-container"> <span class="fig-btn all-fig-btn"><i class="icon icon-all"></i> All Figures</span> <span class="fig-btn next-fig-btn"><i class="icon icon-next"></i> Next</span> <span class="fig-btn prev-fig-btn"><i class="icon icon-prev"></i> Previous</span> </div> <% } %> </div> <div id="image-context"> </div> </div> </div> </script> <script id="image-context-template" type="text/template"> <div class="footer-text"> <div id="figure-description-wrapper"> <div id="view-more-wrapper" style="<% descriptionExpanded? print('display:none;') : '' %>"> <span id="figure-title"><%= title %></span> <p id="figure-description"> <%= description %>   </p> <span id="view-more">show more<i class="icon-arrow-right"></i></span> </div> <div id="view-less-wrapper" style="<% descriptionExpanded? print('display:inline-block;') : '' %>" > <span id="figure-title"><%= title %></span> <p id="full-figure-description"> <%= description %>   <span id="view-less">show less<i class="icon-arrow-left"></i></span> </p> </div> </div> </div> <div id="show-context-container"> <a class="btn show-context" href="<%= showInContext(strippedDoi) %>">Show in Context</a> </div> <div id="download-buttons"> <h3>Download:</h3> <div class="item"> <a href="/digitalhealth/article/figure/image?size=original&download=&id=<%= doi %>" title="original image"> <span class="download-btn">TIFF</span> </a> <span class="file-size"><%= fileSizes.original %></span> </div> <div class="item"> <a href="/digitalhealth/article/figure/image?size=large&download=&id=<%= doi %>" title="large image"> <span class="download-btn">PNG</span> </a> <span class="file-size"><%= fileSizes.large %></span> </div> <div class="item"> <a href="/digitalhealth/article/figure/powerpoint?id=<%= doi %>" title="PowerPoint slide"> <span class="download-btn">PPT</span> </a> </div> </div> </script> <div class="article-content"> <div id="figure-carousel-section"> <h2>Figures</h2> <div id="figure-carousel"> <div class="carousel-wrapper"> <div class="slider"> <div class="carousel-item lightbox-figure" data-doi="10.1371/journal.pdig.0000115.g001"> <img src="/digitalhealth/article/figure/image?size=inline&id=10.1371/journal.pdig.0000115.g001" loading="lazy" alt="Fig 1" /> </div> <div class="carousel-item lightbox-figure" data-doi="10.1371/journal.pdig.0000115.g002"> <img src="/digitalhealth/article/figure/image?size=inline&id=10.1371/journal.pdig.0000115.g002" loading="lazy" alt="Fig 2" /> </div> <div class="carousel-item lightbox-figure" data-doi="10.1371/journal.pdig.0000115.g003"> <img src="/digitalhealth/article/figure/image?size=inline&id=10.1371/journal.pdig.0000115.g003" loading="lazy" alt="Fig 3" /> </div> <div class="carousel-item lightbox-figure" data-doi="10.1371/journal.pdig.0000115.g004"> <img src="/digitalhealth/article/figure/image?size=inline&id=10.1371/journal.pdig.0000115.g004" loading="lazy" alt="Fig 4" /> </div> <div class="carousel-item lightbox-figure" data-doi="10.1371/journal.pdig.0000115.g005"> <img src="/digitalhealth/article/figure/image?size=inline&id=10.1371/journal.pdig.0000115.g005" loading="lazy" alt="Fig 5" /> </div> <div class="carousel-item lightbox-figure" data-doi="10.1371/journal.pdig.0000115.g006"> <img src="/digitalhealth/article/figure/image?size=inline&id=10.1371/journal.pdig.0000115.g006" loading="lazy" alt="Fig 6" /> </div> <div class="carousel-item lightbox-figure" data-doi="10.1371/journal.pdig.0000115.g007"> <img src="/digitalhealth/article/figure/image?size=inline&id=10.1371/journal.pdig.0000115.g007" loading="lazy" alt="Fig 7" /> </div> <div class="carousel-item lightbox-figure" data-doi="10.1371/journal.pdig.0000115.g008"> <img src="/digitalhealth/article/figure/image?size=inline&id=10.1371/journal.pdig.0000115.g008" loading="lazy" alt="Fig 8" /> </div> <div class="carousel-item lightbox-figure" data-doi="10.1371/journal.pdig.0000115.g009"> <img src="/digitalhealth/article/figure/image?size=inline&id=10.1371/journal.pdig.0000115.g009" loading="lazy" alt="Fig 9" /> </div> <div class="carousel-item lightbox-figure" data-doi="10.1371/journal.pdig.0000115.g010"> <img src="/digitalhealth/article/figure/image?size=inline&id=10.1371/journal.pdig.0000115.g010" loading="lazy" alt="Fig 10" /> </div> <div class="carousel-item lightbox-figure" data-doi="10.1371/journal.pdig.0000115.g011"> <img src="/digitalhealth/article/figure/image?size=inline&id=10.1371/journal.pdig.0000115.g011" loading="lazy" alt="Fig 11" /> </div> <div class="carousel-item lightbox-figure" data-doi="10.1371/journal.pdig.0000115.g012"> <img src="/digitalhealth/article/figure/image?size=inline&id=10.1371/journal.pdig.0000115.g012" loading="lazy" alt="Fig 12" /> </div> <div class="carousel-item lightbox-figure" data-doi="10.1371/journal.pdig.0000115.g013"> <img src="/digitalhealth/article/figure/image?size=inline&id=10.1371/journal.pdig.0000115.g013" loading="lazy" alt="Fig 13" /> </div> </div> </div> <div class="carousel-control"> <span class="button previous"></span> <span class="button next"></span> </div> <div class="carousel-page-buttons"> </div> </div> </div> <script src="/resource/js/vendor/jquery.touchswipe.js" type="text/javascript"></script> <script src="/resource/js/components/figure_carousel.js" type="text/javascript"></script> <script src="/resource/js/vendor/jquery.dotdotdot.js" type="text/javascript"></script> <div class="article-text" id="artText"> <div xmlns:plos="http://plos.org" class="abstract toc-section abstract-type-"><a id="abstract0" name="abstract0" data-toc="abstract0" class="link-target" title="Abstract"></a><h2>Abstract</h2><div class="abstract-content"><a id="article1.front1.article-meta1.abstract1.p1" name="article1.front1.article-meta1.abstract1.p1" class="link-target"></a><p>The object of this study is to put forward uncertainty modeling associated with missing time series data imputation in a predictive context. We propose three imputation methods associated with uncertainty modeling. These methods are evaluated on a COVID-19 dataset out of which some values have been randomly removed. The dataset contains the numbers of daily COVID-19 confirmed diagnoses (“new cases”) and daily deaths (“new deaths”) recorded since the start of the pandemic up to July 2021. The considered task is to predict the number of new deaths 7 days in advance. The more values are missing, the higher the imputation impact is on the predictive performances. The Evidential <em>K</em>-Nearest Neighbors (E<em>K</em>NN) algorithm is used for its ability to take into account labels uncertainty. Experiments are provided to measure the benefits of the label uncertainty models. Results show the positive impact of uncertainty models on imputation performances, especially in a noisy context where the number of missing values is high.</p> </div></div><div xmlns:plos="http://plos.org" class="abstract toc-section abstract-type-summary"><a id="abstract1" name="abstract1" data-toc="abstract1" class="link-target" title="Author Summary"></a> <h2>Author Summary</h2> <div class="abstract-content"><a id="article1.front1.article-meta1.abstract2.p1" name="article1.front1.article-meta1.abstract2.p1" class="link-target"></a><p>The methodological aim of this study was to take advantage of missing data chronology in the imputation process in order to handle missing time series data. The practical goal of COVID application was to study the link between the numbers of chronological COVID confirmed cases and death. To achieve these goals we proposed 3 imputation methods of missing time series data each of them associated with an uncertainty model. For the COVID number of death prediction task, we set up a non-linear regression modeling for the number of COVID deaths prediction from past deaths and confirmed cases data. This led us to extend the Evidential K-Nearest Neighbor method to regression problems and to assess the impact of uncertainty modeling within imputation process in regards to predictive task. Finally, we showed the superiority of the time-E<em>K</em>NN (TEKNN) in terms of predictive performances compared to the Last Observation Carried Forward (LOCF) and Centered Moving Average (CMA) methods. More globally, we showed the interest of modeling the uncertainty in the imputation process to better control the prediction error, especially during relative stable periods.</p> </div></div> <div xmlns:plos="http://plos.org" class="articleinfo"><p><strong>Citation: </strong>Elimam R, Sutton-Charani N, Perrey S, Montmain J (2022) Uncertain imputation for time-series forecasting: Application to COVID-19 daily mortality prediction. PLOS Digit Health 1(10): e0000115. https://doi.org/10.1371/journal.pdig.0000115</p><p><strong>Editor: </strong>Rutwik Shah, UCSF: University of California San Francisco, UNITED STATES </p><p><strong>Received: </strong>April 27, 2022; <strong>Accepted: </strong>August 30, 2022; <strong>Published: </strong> October 25, 2022</p><p><strong>Copyright: </strong> © 2022 Elimam et al. This is an open access article distributed under the terms of the <a href="http://creativecommons.org/licenses/by/4.0/">Creative Commons Attribution License</a>, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.</p><p><strong>Data Availability: </strong>The data used in this article can be accessed at the public website <a href="https://ourworldindata.org/coronavirus">https://ourworldindata.org/coronavirus</a>.</p><p><strong>Funding: </strong>The authors received no specific funding for this work.</p><p><strong>Competing interests: </strong> The authors have no competing interests to declare.</p></div> <div xmlns:plos="http://plos.org" id="section1" class="section toc-section"><a id="sec001" name="sec001" data-toc="sec001" class="link-target" title="1 Introduction"></a><h2>1 Introduction</h2><a id="article1.body1.sec1.p1" name="article1.body1.sec1.p1" class="link-target"></a><p>With an increasing number of machine learning applications, data availability is becoming very important. Yet available datasets are often incomplete due to different measurement failures, especially when the data collection involves human participation. The treatment of missing values for predictive tasks has become an important issue giving rise to a wide range of research. Many methods have been proposed to handle missing values (average, omission, learning, etc.), one of the most popular being simply to exclude incomplete examples from the learning set, due to the incapacity to deal with missing values of most predictive models [<a href="#pdig.0000115.ref001" class="ref-tip">1</a>, <a href="#pdig.0000115.ref002" class="ref-tip">2</a>]. That type of treatment remains undesirable with limited amounts of available data or in a chronological data structure.</p> <a id="article1.body1.sec1.p2" name="article1.body1.sec1.p2" class="link-target"></a><p>The chosen method also depends on the nature of the missing values, which is often categorized in <em>Missing At Random</em> (MAR) for missing values that are dependent on observed values, <em>Not Missing At Random</em> (NMAR) missing values which depend on unobserved values and <em>Missing Completely At Random</em> (MCAR) missing values which are independent of observed or unobserved values [<a href="#pdig.0000115.ref003" class="ref-tip">3</a>, <a href="#pdig.0000115.ref004" class="ref-tip">4</a>]. Those categories indicate why data are missing, an information to be taken into account in the imputation method [<a href="#pdig.0000115.ref005" class="ref-tip">5</a>].</p> <a id="article1.body1.sec1.p3" name="article1.body1.sec1.p3" class="link-target"></a><p>Moreover, in a time-series forecasting context, missing values introduce irregular time stamps that contradict the most common hypothesis of standard time series methods. In terms of uncertainty, missing values can be interpreted as total ignorance or complete imprecision about the actual values. Some soft computing methods are designed to handle data uncertainty by modeling its degree [<a href="#pdig.0000115.ref006" class="ref-tip">6</a>–<a href="#pdig.0000115.ref008" class="ref-tip">8</a>]. In such frameworks, ignorance corresponds to the highest level of uncertainty and therefore missing values can be incorporated in models that take into account the uncertainty level. In this study our aim is to predict COVID-19 daily deaths in an artificially noised dataset out of which some labels (number of new deaths) have been randomly removed, resulting in MCAR missing values since the missingness is not related to any observed or unobserved values. The benefits of associating uncertainty models to imputation methods are studied. We evaluate the predictive performance of the Evidential-<em>K</em> Nearest Neighbors algorithm once missing data are imputed with and without uncertainty models (in the latter case the imputed labels are considered as certain).</p> <a id="article1.body1.sec1.p4" name="article1.body1.sec1.p4" class="link-target"></a><p>The structure of the dataset is adapted to time series forecasting. We propose a methodology to handle the uncertainty inherent to missing values imputation methods. Some theories allow representation of uncertainty in a broader way than classical probability theory. Missing values uncertainty can be handled in different frameworks, <em>e.g.</em> fuzzy sets [<a href="#pdig.0000115.ref009" class="ref-tip">9</a>], possibilities distribution [<a href="#pdig.0000115.ref010" class="ref-tip">10</a>], probability sets [<a href="#pdig.0000115.ref011" class="ref-tip">11</a>], belief functions [<a href="#pdig.0000115.ref012" class="ref-tip">12</a>, <a href="#pdig.0000115.ref013" class="ref-tip">13</a>]. We chose the belief functions framework for its flexibility and relative simplicity and also because recognized machine learning algorithms based on that framework are available [<a href="#pdig.0000115.ref014" class="ref-tip">14</a>–<a href="#pdig.0000115.ref017" class="ref-tip">17</a>].</p> <a id="article1.body1.sec1.p5" name="article1.body1.sec1.p5" class="link-target"></a><p>Beyond standard machine learning researches on missing data imputation methods [<a href="#pdig.0000115.ref001" class="ref-tip">1</a>, <a href="#pdig.0000115.ref002" class="ref-tip">2</a>], some soft computing imputation methods have been proposed [<a href="#pdig.0000115.ref018" class="ref-tip">18</a>–<a href="#pdig.0000115.ref020" class="ref-tip">20</a>]. In [<a href="#pdig.0000115.ref021" class="ref-tip">21</a>], a method is proposed to categorize missing data and to remove noise with a kernel-based approach that enables classification within the belief function framework. The purpose of the method is to design an imputation strategy providing uncertainty <em>resistance</em>; however the method does not handle the uncertainty at the predictive level. In [<a href="#pdig.0000115.ref022" class="ref-tip">22</a>] the authors propose a method to minimize the classification errors due to uncertainty caused by missing values. Multiple precise missing values estimations are performed and the corresponding predictions are finally combined in predictive belief functions. In the context of information retrieval, Jousselme <em>et al.</em> proposed a missing values uncertainty representation [<a href="#pdig.0000115.ref023" class="ref-tip">23</a>]. Missing data are modeled as a belief function defined over the variables spaces. The method shows good performance for information retrieval task. As a matter of fact, none of those methods allows for the taking into account the uncertainty associated with imputation at the predictive level. In this study, we propose an approach to impute missing data in a chronological dataset and to model the resulting uncertainty in the belief functions framework. Finally an evidential classification model (E<em>K</em>NN) is extended to regression tasks in order to take into account the uncertainty associated with the imputation process.</p> <a id="article1.body1.sec1.p6" name="article1.body1.sec1.p6" class="link-target"></a><p>The rest of this paper is organized as follows: first we present the main results of this study in Section 2, then we present our conclusion and perspectives in Section 3. All the details of our approach are given in Section 4 where we briefly recall the basis of the belief functions framework basis and the E<em>K</em>NN algorithm in the first subsection 4.1. After the presentation of the time series forecasting problem in an incomplete data context in the following subsection 4.2, three missing value imputation methods are proposed in subsection 4.3. In subsection 4.4 we present the uncertainty models associated with the previously introduced imputation methods; The uncertainty we are handling in this study is epistemic as we have no information about the missing label values. The chosen predictive model is Evidential-<em>K</em> Nearest Neighbors for its simplicity and its ability to deal with uncertain labels [<a href="#pdig.0000115.ref014" class="ref-tip">14</a>].</p> </div> <div xmlns:plos="http://plos.org" id="section2" class="section toc-section"><a id="sec002" name="sec002" data-toc="sec002" class="link-target" title="2 Results"></a><h2>2 Results</h2><a id="article1.body1.sec2.p1" name="article1.body1.sec2.p1" class="link-target"></a><p>First, we observe in <a href="#pdig-0000115-g001">Fig 1</a> that the three imputation methods are comparable in terms of performance. The TEKNN approach seems to perform better than the LOCF and CMA methods and its superiority grows as the noise level increases. Except for a small noise level of 0.1, the TEKNN model seems to be the best imputation method.</p> <a class="link-target" id="pdig-0000115-g001" name="pdig-0000115-g001"></a><div class="figure" data-doi="10.1371/journal.pdig.0000115.g001"><div class="img-box"><a title="Click for larger image" href="article/figure/image?size=medium&id=10.1371/journal.pdig.0000115.g001" data-doi="10.1371/journal.pdig.0000115" data-uri="10.1371/journal.pdig.0000115.g001"><img src="article/figure/image?size=inline&id=10.1371/journal.pdig.0000115.g001" alt="thumbnail" class="thumbnail" loading="lazy"></a><div class="expand"></div></div><div class="figure-inline-download"> Download: <ul><li><a href="article/figure/powerpoint?id=10.1371/journal.pdig.0000115.g001"><div class="definition-label">PPT</div><div class="definition-description">PowerPoint slide</div></a></li><li><a href="article/figure/image?download&size=large&id=10.1371/journal.pdig.0000115.g001"><div class="definition-label">PNG</div><div class="definition-description">larger image</div></a></li><li><a href="article/figure/image?download&size=original&id=10.1371/journal.pdig.0000115.g001"><div class="definition-label">TIFF</div><div class="definition-description">original image</div></a></li></ul></div><div class="figcaption"><span>Fig 1. </span> Imputation errors.</div><p class="caption_target"></p><p class="caption_object"><a href="https://doi.org/10.1371/journal.pdig.0000115.g001"> https://doi.org/10.1371/journal.pdig.0000115.g001</a></p></div><a id="article1.body1.sec2.p2" name="article1.body1.sec2.p2" class="link-target"></a><p>On the chronological evaluation with a noise level of 0 (<a href="#pdig-0000115-g002">Fig 2</a>), we observe that the E<em>K</em>NN predictions with and without uncertainty models associated to label imputation blend together. This observation was expected as the data are not noised, <em>i.e.</em> there is no uncertainty associated with training labels. During the increasing and decreasing phases of the number of deaths, the E<em>K</em>NN seems to perform better than the baseline approach (blue and green curves are closer to the purple one than the red curve during those periods). During the periods of relative stability when the evolution of the pandemic slows down, there seems to be no significant differences between the considered approaches.</p> <a class="link-target" id="pdig-0000115-g002" name="pdig-0000115-g002"></a><div class="figure" data-doi="10.1371/journal.pdig.0000115.g002"><div class="img-box"><a title="Click for larger image" href="article/figure/image?size=medium&id=10.1371/journal.pdig.0000115.g002" data-doi="10.1371/journal.pdig.0000115" data-uri="10.1371/journal.pdig.0000115.g002"><img src="article/figure/image?size=inline&id=10.1371/journal.pdig.0000115.g002" alt="thumbnail" class="thumbnail" loading="lazy"></a><div class="expand"></div></div><div class="figure-inline-download"> Download: <ul><li><a href="article/figure/powerpoint?id=10.1371/journal.pdig.0000115.g002"><div class="definition-label">PPT</div><div class="definition-description">PowerPoint slide</div></a></li><li><a href="article/figure/image?download&size=large&id=10.1371/journal.pdig.0000115.g002"><div class="definition-label">PNG</div><div class="definition-description">larger image</div></a></li><li><a href="article/figure/image?download&size=original&id=10.1371/journal.pdig.0000115.g002"><div class="definition-label">TIFF</div><div class="definition-description">original image</div></a></li></ul></div><div class="figcaption"><span>Fig 2. </span> Predictive results on data imputed with time-E<em>K</em>NN imputation method, comparison with true labels: <em>K</em> = 1: <em>q</em> = 4: <em>v</em> = 0.</div><p class="caption_target"></p><p class="caption_object"><a href="https://doi.org/10.1371/journal.pdig.0000115.g002"> https://doi.org/10.1371/journal.pdig.0000115.g002</a></p></div><a id="article1.body1.sec2.p3" name="article1.body1.sec2.p3" class="link-target"></a><p>We note a small time shift between the real number of daily deaths and all predictive models, especially at the beginning of the wave. This is due to the fact that the model needs high number of deaths in the past to be able to predict high values in the future.</p> <a id="article1.body1.sec2.p4" name="article1.body1.sec2.p4" class="link-target"></a><p>We observe a phase shift at the start of the wave, due to the fact that, before, there is no neighbor labelled with a high number of deaths, therefore the predicted values are under-estimated until we have data in the training set presenting a high number of deaths. We also note that the phase shift reduces thereafter.</p> <a id="article1.body1.sec2.p5" name="article1.body1.sec2.p5" class="link-target"></a><p>On <a href="#pdig-0000115-g003">Fig 3</a>, we observe that, with time-E<em>K</em>NN imputations, the “E<em>K</em>NN uncertain labels” and the “E<em>K</em>NN” make predictions reaching quite similar performances with a slight superiority for the “E<em>K</em>NN” (without uncertain model) during the beginning and the end phases of the pandemic wave. During the relatively stable periods, the “E<em>K</em>NN” associated with an uncertain model performs better than without imputation uncertain modelling.</p> <a class="link-target" id="pdig-0000115-g003" name="pdig-0000115-g003"></a><div class="figure" data-doi="10.1371/journal.pdig.0000115.g003"><div class="img-box"><a title="Click for larger image" href="article/figure/image?size=medium&id=10.1371/journal.pdig.0000115.g003" data-doi="10.1371/journal.pdig.0000115" data-uri="10.1371/journal.pdig.0000115.g003"><img src="article/figure/image?size=inline&id=10.1371/journal.pdig.0000115.g003" alt="thumbnail" class="thumbnail" loading="lazy"></a><div class="expand"></div></div><div class="figure-inline-download"> Download: <ul><li><a href="article/figure/powerpoint?id=10.1371/journal.pdig.0000115.g003"><div class="definition-label">PPT</div><div class="definition-description">PowerPoint slide</div></a></li><li><a href="article/figure/image?download&size=large&id=10.1371/journal.pdig.0000115.g003"><div class="definition-label">PNG</div><div class="definition-description">larger image</div></a></li><li><a href="article/figure/image?download&size=original&id=10.1371/journal.pdig.0000115.g003"><div class="definition-label">TIFF</div><div class="definition-description">original image</div></a></li></ul></div><div class="figcaption"><span>Fig 3. </span> Predictive results on data imputed with time-E<em>K</em>NN imputation method, comparison with true labels: <em>K</em> = 1: <em>q</em> = 3—<em>v</em> = 0.3.</div><p class="caption_target"></p><p class="caption_object"><a href="https://doi.org/10.1371/journal.pdig.0000115.g003"> https://doi.org/10.1371/journal.pdig.0000115.g003</a></p></div><a id="article1.body1.sec2.p6" name="article1.body1.sec2.p6" class="link-target"></a><p>The best results with a noise level of zero were obtained with <em>K</em> = 1, and <em>q</em> = 4. We see on Fig 5 that thanks to the uncertainty model of the TE<em>K</em>NN imputation method we have a better predictive performance up to a high noise level. The superiority of the standard E<em>K</em>NN after a noise level of about 0.5 is due to the fact that a high missing value rate induces highly uncertain neighborhoods and thus very uncertain predictions, a large mass being attributed to ignorance. The pignistic transformation applied to the mass function output of the E<em>K</em>NN distributes the mass on the Ω space in a uniform way on all the singletons; if this mass is too high, the predictions tend to the center of the space. Except for <em>K</em> = 1, for all the other configurations <em>K</em> = {10, 20} and <em>q</em> = {1, 2, …, 7} the use of uncertainty models allows us to have better predictive performances (Figs <a href="#pdig-0000115-g004">4</a>, <a href="#pdig-0000115-g005">5</a> and <a href="#pdig-0000115-g006">6</a>).</p> <a class="link-target" id="pdig-0000115-g004" name="pdig-0000115-g004"></a><div class="figure" data-doi="10.1371/journal.pdig.0000115.g004"><div class="img-box"><a title="Click for larger image" href="article/figure/image?size=medium&id=10.1371/journal.pdig.0000115.g004" data-doi="10.1371/journal.pdig.0000115" data-uri="10.1371/journal.pdig.0000115.g004"><img src="article/figure/image?size=inline&id=10.1371/journal.pdig.0000115.g004" alt="thumbnail" class="thumbnail" loading="lazy"></a><div class="expand"></div></div><div class="figure-inline-download"> Download: <ul><li><a href="article/figure/powerpoint?id=10.1371/journal.pdig.0000115.g004"><div class="definition-label">PPT</div><div class="definition-description">PowerPoint slide</div></a></li><li><a href="article/figure/image?download&size=large&id=10.1371/journal.pdig.0000115.g004"><div class="definition-label">PNG</div><div class="definition-description">larger image</div></a></li><li><a href="article/figure/image?download&size=original&id=10.1371/journal.pdig.0000115.g004"><div class="definition-label">TIFF</div><div class="definition-description">original image</div></a></li></ul></div><div class="figcaption"><span>Fig 4. </span> Predictive performances relative to noise levels for data imputed with LOCF imputation method: K = 10: z = 4.</div><p class="caption_target"></p><p class="caption_object"><a href="https://doi.org/10.1371/journal.pdig.0000115.g004"> https://doi.org/10.1371/journal.pdig.0000115.g004</a></p></div><a class="link-target" id="pdig-0000115-g005" name="pdig-0000115-g005"></a><div class="figure" data-doi="10.1371/journal.pdig.0000115.g005"><div class="img-box"><a title="Click for larger image" href="article/figure/image?size=medium&id=10.1371/journal.pdig.0000115.g005" data-doi="10.1371/journal.pdig.0000115" data-uri="10.1371/journal.pdig.0000115.g005"><img src="article/figure/image?size=inline&id=10.1371/journal.pdig.0000115.g005" alt="thumbnail" class="thumbnail" loading="lazy"></a><div class="expand"></div></div><div class="figure-inline-download"> Download: <ul><li><a href="article/figure/powerpoint?id=10.1371/journal.pdig.0000115.g005"><div class="definition-label">PPT</div><div class="definition-description">PowerPoint slide</div></a></li><li><a href="article/figure/image?download&size=large&id=10.1371/journal.pdig.0000115.g005"><div class="definition-label">PNG</div><div class="definition-description">larger image</div></a></li><li><a href="article/figure/image?download&size=original&id=10.1371/journal.pdig.0000115.g005"><div class="definition-label">TIFF</div><div class="definition-description">original image</div></a></li></ul></div><div class="figcaption"><span>Fig 5. </span> Predictive performances relative to noise levels for data imputed with time-E<em>K</em>NN imputation method: <em>K</em> = 1: <em>q</em> = 4.</div><p class="caption_target"></p><p class="caption_object"><a href="https://doi.org/10.1371/journal.pdig.0000115.g005"> https://doi.org/10.1371/journal.pdig.0000115.g005</a></p></div><a class="link-target" id="pdig-0000115-g006" name="pdig-0000115-g006"></a><div class="figure" data-doi="10.1371/journal.pdig.0000115.g006"><div class="img-box"><a title="Click for larger image" href="article/figure/image?size=medium&id=10.1371/journal.pdig.0000115.g006" data-doi="10.1371/journal.pdig.0000115" data-uri="10.1371/journal.pdig.0000115.g006"><img src="article/figure/image?size=inline&id=10.1371/journal.pdig.0000115.g006" alt="thumbnail" class="thumbnail" loading="lazy"></a><div class="expand"></div></div><div class="figure-inline-download"> Download: <ul><li><a href="article/figure/powerpoint?id=10.1371/journal.pdig.0000115.g006"><div class="definition-label">PPT</div><div class="definition-description">PowerPoint slide</div></a></li><li><a href="article/figure/image?download&size=large&id=10.1371/journal.pdig.0000115.g006"><div class="definition-label">PNG</div><div class="definition-description">larger image</div></a></li><li><a href="article/figure/image?download&size=original&id=10.1371/journal.pdig.0000115.g006"><div class="definition-label">TIFF</div><div class="definition-description">original image</div></a></li></ul></div><div class="figcaption"><span>Fig 6. </span> Predictive performances relative to noise levels for data imputed with CMA imputation method: <em>K</em> = 20: <em>q</em> = 4.</div><p class="caption_target"></p><p class="caption_object"><a href="https://doi.org/10.1371/journal.pdig.0000115.g006"> https://doi.org/10.1371/journal.pdig.0000115.g006</a></p></div><a id="article1.body1.sec2.p7" name="article1.body1.sec2.p7" class="link-target"></a><p>Redefining Ω at each iteration depending on the maximum number of deaths observed is a conservative way to proceed, but it allows both models to predict any “reasonable” unobserved value.</p> </div> <div xmlns:plos="http://plos.org" id="section3" class="section toc-section"><a id="sec003" name="sec003" data-toc="sec003" class="link-target" title="3 Conclusion"></a><h2>3 Conclusion</h2><a id="article1.body1.sec3.p1" name="article1.body1.sec3.p1" class="link-target"></a><p>The aim of this study was to propose uncertainty models associated with missing chronological data imputation methods. The objective was the prediction of the number of daily COVID 19 deaths at a prediction horizon of 7 days with an artificially noised dataset. We proposed three imputation methods (LOCF,CMA,time-E<em>K</em>NN) that showed good imputation performances.</p> <a id="article1.body1.sec3.p2" name="article1.body1.sec3.p2" class="link-target"></a><p>For our experiment we extended the E<em>K</em>NN methodology proposed in [<a href="#pdig.0000115.ref014" class="ref-tip">14</a>] to regression problems. We were able to compare the predictive performances of the “E<em>K</em>NN” and the “E<em>K</em>NN uncertain labels” with three imputation methods of comparable performances. The experiment showed the benefit of uncertainty modeling for chronological imputed values throughout several hyper-parameters configurations.</p> <a id="article1.body1.sec3.p3" name="article1.body1.sec3.p3" class="link-target"></a><p>The use of incomplete past values (<em>x</em><sub><em>t</em></sub>, <em>y</em><sub><em>t</em></sub>)<sub><em>t</em>=<em>t</em>−<em>q</em>,…,<em>t</em></sub> as features leads to uncertain feature values. A logical continuation of this work could be to use other predictive models than the E<em>K</em>NN, especially the ones that handle uncertain attributes during learning [<a href="#pdig.0000115.ref017" class="ref-tip">17</a>, <a href="#pdig.0000115.ref024" class="ref-tip">24</a>, <a href="#pdig.0000115.ref025" class="ref-tip">25</a>].</p> <a id="article1.body1.sec3.p4" name="article1.body1.sec3.p4" class="link-target"></a><p>The problem of predicting COVID 19 daily deaths led us to a numerical regression problem, therefore the time based uncertainty model is not adapted to classification. It would be interesting to extend it to classification in a categorical time series context. We also know from health experts that the number of new COVID 19 cases is not a good indicator for predicting deaths, therefore it would be interesting to weigh the importance of the attributes in the <em>K</em> nearest neighbors computing [<a href="#pdig.0000115.ref026" class="ref-tip">26</a>].</p> <a id="article1.body1.sec3.p5" name="article1.body1.sec3.p5" class="link-target"></a><p>Another perspective could be to compare the predictive performance we can get with soft predictive models that handle missing values without any need of imputation.</p> <a id="article1.body1.sec3.p6" name="article1.body1.sec3.p6" class="link-target"></a><p>Additionally, there are some algorithms like E<em>K</em>NN that use this framework. The theory of belief functions permits us to have the enhancement of uncertainty modeling as a perspective, for example by using imputation with intervals instead of precise values.</p> </div> <div xmlns:plos="http://plos.org" id="section4" class="section toc-section"><a id="sec004" name="sec004" data-toc="sec004" class="link-target" title="4 Materials and methods"></a><h2>4 Materials and methods</h2> <div id="section1" class="section toc-section"><a id="sec005" name="sec005" class="link-target" title="4.1 Background"></a> <h3>4.1 Background</h3> <a id="article1.body1.sec4.sec1.p1" name="article1.body1.sec4.sec1.p1" class="link-target"></a><p>In this section we expose the basics of belief functions theory, also known as Dempster-Shafer or evidence theory [<a href="#pdig.0000115.ref012" class="ref-tip">12</a>, <a href="#pdig.0000115.ref013" class="ref-tip">13</a>] and we detail the Evidential <em>K</em>-Nearest Neighbors algorithm [<a href="#pdig.0000115.ref014" class="ref-tip">14</a>].</p> <div id="section1" class="section toc-section"><a id="sec006" name="sec006" class="link-target" title="4.1.1 Belief functions"></a><h4>4.1.1 Belief functions.</h4><a id="article1.body1.sec4.sec1.sec1.p1" name="article1.body1.sec4.sec1.sec1.p1" class="link-target"></a><p>Let Ω = {<em>ω</em><sub>1</sub>, <em>ω</em><sub>2</sub>, …, <em>ω</em><sub><em>H</em></sub>} be the so-called frame of discernment, <em>i.e.</em> the universe of possible outcomes or hypotheses. The mass function <em>m</em> represents our degree of knowledge about all subsets of Ω, <em>i.e.</em> about the powerset 2<sup>Ω</sup> of Ω. The elements <em>A</em> ⊆ Ω such as <em>m</em>(<em>A</em>) > 0 are called focal elements and their weights sum to 1: <a name="pdig.0000115.e001" id="pdig.0000115.e001" class="link-target"></a><span class="equation"><img src="article/file?type=thumbnail&id=10.1371/journal.pdig.0000115.e001" loading="lazy" class="inline-graphic"><span class="note">(1)</span></span> The quantity <em>m</em>(Ω) represents the degree of ignorance. From the mass function <em>m</em>, different uncertainty measures can be computed such as the <em>belief</em> and <em>plausibility</em> functions defined in Eqs (<a href="#pdig.0000115.e002">2</a>) and (<a href="#pdig.0000115.e003">3</a>) which can be interpreted as lower and upper “bounds of probability”. <a name="pdig.0000115.e002" id="pdig.0000115.e002" class="link-target"></a><span class="equation"><img src="article/file?type=thumbnail&id=10.1371/journal.pdig.0000115.e002" loading="lazy" class="inline-graphic"><span class="note">(2)</span></span> <a name="pdig.0000115.e003" id="pdig.0000115.e003" class="link-target"></a><span class="equation"><img src="article/file?type=thumbnail&id=10.1371/journal.pdig.0000115.e003" loading="lazy" class="inline-graphic"><span class="note">(3)</span></span></p> <a id="article1.body1.sec4.sec1.sec1.p2" name="article1.body1.sec4.sec1.sec1.p2" class="link-target"></a><p>Different mass functions can represent different sources of information. At the decision level, it may be necessary to combine them into a single mass function expressing all the knowledge we can infer from these sources.</p> <a id="article1.body1.sec4.sec1.sec1.p3" name="article1.body1.sec4.sec1.sec1.p3" class="link-target"></a><p><strong>Mass combination</strong></p> <a id="article1.body1.sec4.sec1.sec1.p4" name="article1.body1.sec4.sec1.sec1.p4" class="link-target"></a><p>There are multiple methods of information fusion through mass combination rules [<a href="#pdig.0000115.ref027" class="ref-tip">27</a>, <a href="#pdig.0000115.ref028" class="ref-tip">28</a>]. One of the most famous is the Dempster’s conjunctive rule of combination ⊕ [<a href="#pdig.0000115.ref012" class="ref-tip">12</a>] (see Eq (<a href="#pdig.0000115.e004">4</a>)): <a name="pdig.0000115.e004" id="pdig.0000115.e004" class="link-target"></a><span class="equation"><img src="article/file?type=thumbnail&id=10.1371/journal.pdig.0000115.e004" loading="lazy" class="inline-graphic"><span class="note">(4)</span></span> where <span class="inline-formula"><img src="article/file?type=thumbnail&id=10.1371/journal.pdig.0000115.e005" loading="lazy" class="inline-graphic"></span> is the degree of <em>conflict</em> between sources <em>m</em><sub>1</sub> and <em>m</em><sub>2</sub>.</p> <a id="article1.body1.sec4.sec1.sec1.p5" name="article1.body1.sec4.sec1.sec1.p5" class="link-target"></a><p>The main idea of this rule is to consider all sources reliable. After the combination, we get a new mass function that can be used at the decision making level.</p> <a id="article1.body1.sec4.sec1.sec1.p6" name="article1.body1.sec4.sec1.sec1.p6" class="link-target"></a><p><strong>Decision making</strong></p> <a id="article1.body1.sec4.sec1.sec1.p7" name="article1.body1.sec4.sec1.sec1.p7" class="link-target"></a><p>In cases where the degree of ignorance <em>m</em>(Ω) is too important, some authors recommend rejecting decision making [<a href="#pdig.0000115.ref014" class="ref-tip">14</a>]. Otherwise, the choice of uncertainty measure to make a decision presents a dilemma [<a href="#pdig.0000115.ref029" class="ref-tip">29</a>].</p> <a id="article1.body1.sec4.sec1.sec1.p8" name="article1.body1.sec4.sec1.sec1.p8" class="link-target"></a><p>For instance, depending on the application goal and the chosen strategy in terms of conservatism, any uncertainty measure lying between the <em>belief</em> (<a href="#pdig.0000115.e002">2</a>) and the <em>plausibility</em> (<a href="#pdig.0000115.e003">3</a>) measures can be used. However, those measures are not additive, <em>i.e.</em> we do not have <em>Bel</em>(<em>A</em> ∪ <em>B</em>) ≠ <em>Bel</em>(<em>A</em>) + <em>Bel</em>(<em>B</em>) ∀<em>A</em>, <em>B</em> ∈ Ω such as <em>A</em> ∩ <em>B</em> = ∅ (same thing for the <em>Pl</em> function). For that reason many data science tools are incompatible with those <em>soft</em> uncertainty measures since most of them have been developed within the standard probability framework.</p> <a id="article1.body1.sec4.sec1.sec1.p9" name="article1.body1.sec4.sec1.sec1.p9" class="link-target"></a><p>For pragmatic reasons many researchers choose to project the information content of mass functions into the standard probability framework [<a href="#pdig.0000115.ref014" class="ref-tip">14</a>, <a href="#pdig.0000115.ref017" class="ref-tip">17</a>]. The <em>Transferable Belief Model</em> was proposed by Smets [<a href="#pdig.0000115.ref029" class="ref-tip">29</a>, <a href="#pdig.0000115.ref030" class="ref-tip">30</a>] where the <em>pignistic</em> transformation allows to convert mass functions into standard probability distributions. Despite known drawbacks [<a href="#pdig.0000115.ref031" class="ref-tip">31</a>], ignorance degrees are projected into uniform distributions. The pignistic transform defined in <a href="#pdig.0000115.e006">Eq (5)</a> remains a natural solution for computing probability distributions from mass functions that mainly relies on uniform ignorance modeling.</p> <a id="article1.body1.sec4.sec1.sec1.p10" name="article1.body1.sec4.sec1.sec1.p10" class="link-target"></a><p>In the machine learning context many classifiers make soft predictions expressed in more complex spaces than the standard probability one [<a href="#pdig.0000115.ref014" class="ref-tip">14</a>, <a href="#pdig.0000115.ref032" class="ref-tip">32</a>]. When the learning data are uncertain (evidential), in order to get <em>handy</em> predictions some authors [<a href="#pdig.0000115.ref017" class="ref-tip">17</a>] have proposed to maximize the evidential extension of the likelihood function [<a href="#pdig.0000115.ref014" class="ref-tip">14</a>] in order to estimate probability distributions. When the evidential likelihood maximization is not straightforward, the Evidential Expectation Maximization (E<sup>2</sup>M) algorithm can be used. However the iterative nature of the E<sup>2</sup>M algorithm can lead to high levels of complexity. For the sake of simplicity, the pignistic transform is preferred in this study. <a name="pdig.0000115.e006" id="pdig.0000115.e006" class="link-target"></a><span class="equation"><img src="article/file?type=thumbnail&id=10.1371/journal.pdig.0000115.e006" loading="lazy" class="inline-graphic"><span class="note">(5)</span></span></p> </div> <div id="section2" class="section toc-section"><a id="sec007" name="sec007" class="link-target" title="4.1.2 Evidential KNearest Neighbors—EKNN"></a><h4>4.1.2 Evidential <em>K</em>Nearest Neighbors—EKNN.</h4><a id="article1.body1.sec4.sec1.sec2.p1" name="article1.body1.sec4.sec1.sec2.p1" class="link-target"></a><p>The E<em>K</em>NN extends <em>K</em>-Nearest Neighbors algorithm to the belief functions framework [<a href="#pdig.0000115.ref014" class="ref-tip">14</a>] and is based on Dempster’s conjunctive rule of combination. Let (<em>x</em>, <em>y</em>) = (<em>x</em><sub><em>i</em></sub>, <em>y</em><sub><em>i</em></sub>)<sub><em>i</em>=1,…,<em>n</em></sub> be a training set and Ω = {<em>ω</em><sub>1</sub>, <em>ω</em><sub>2</sub>, …., <em>ω</em><sub><em>H</em></sub>} the frame of discernment of the class label <em>Y</em>. Let <em>x</em><sub><em>s</em></sub> be a new observation to classify. The first step is to compute the distances between <em>x</em><sub><em>s</em></sub> and all training examples <em>x</em><sub><em>i</em></sub> to get the set of the <em>K</em> “nearest” neighbors of <em>x</em><sub><em>s</em></sub>. In the E<em>K</em>NN approach, each neighbor is considered as a source of information. For each neighbor <em>x</em><sub><em>i</em></sub> labelled with {<em>ω</em><sub><em>l</em></sub>}, a mass function <em>m</em><sub><em>s</em>,<em>i</em></sub> is computed: <a name="pdig.0000115.e007" id="pdig.0000115.e007" class="link-target"></a><span class="equation"><img src="article/file?type=thumbnail&id=10.1371/journal.pdig.0000115.e007" loading="lazy" class="inline-graphic"><span class="note">(6)</span></span></p> <a id="article1.body1.sec4.sec1.sec2.p2" name="article1.body1.sec4.sec1.sec2.p2" class="link-target"></a><p>The quantity <em>m</em><sub><em>s</em>,<em>i</em></sub>({<em>ω</em><sub><em>l</em></sub>}) represents the mass assigned to the label <em>ω</em><sub><em>l</em></sub> by neighbor <em>x</em><sub><em>i</em></sub> to <em>x</em><sub><em>s</em></sub>. The parameters <em>α</em><sub>0</sub>, <em>γ</em><sub><em>l</em></sub> can be estimated with classical optimization procedure as gradient descent. The parameter <em>γ</em><sub><em>l</em></sub> > 0 relates to the label <em>ω</em><sub><em>l</em></sub>, in [<a href="#pdig.0000115.ref014" class="ref-tip">14</a>] the author recommends to set the parameter <em>α</em><sub>0</sub> (which prevents dogmatic mass functions) to 0.95, <em>d</em><sub><em>s</em>,<em>i</em></sub> stands for the euclidean distance between <em>x</em><sub><em>s</em></sub> and its neighbor <em>x</em><sub><em>i</em></sub> and λ ∈ {1, 2, 3, …} is a parameter that penalizes the farthest neighbors.</p> <a id="article1.body1.sec4.sec1.sec2.p3" name="article1.body1.sec4.sec1.sec2.p3" class="link-target"></a><p>Once all the masses <em>m</em><sub><em>s</em>,<em>i</em></sub> have been computed, they are combined with Dempster’s rule of combination into a final mass <span class="inline-formula"><img src="article/file?type=thumbnail&id=10.1371/journal.pdig.0000115.e008" loading="lazy" class="inline-graphic"></span> where <span class="inline-formula"><img src="article/file?type=thumbnail&id=10.1371/journal.pdig.0000115.e009" loading="lazy" class="inline-graphic"></span> represent the knowledge associated with the <em>K</em> nearest neighbors of <em>x</em><sub><em>s</em></sub>. Finally, decision can be made according to <em>m</em><sub><em>s</em></sub>, the approach chosen in [<a href="#pdig.0000115.ref014" class="ref-tip">14</a>] is to predict the label corresponding to the maximum of belief.</p> <a id="article1.body1.sec4.sec1.sec2.p4" name="article1.body1.sec4.sec1.sec2.p4" class="link-target"></a><p>In the case of uncertain or imperfect labels modeled by mass functions, <em>i.e.</em> the learning set is now <span class="inline-formula"><img src="article/file?type=thumbnail&id=10.1371/journal.pdig.0000115.e010" loading="lazy" class="inline-graphic"></span>, for each neighbor <em>x</em><sub><em>i</em></sub> we have a mass function <span class="inline-formula"><img src="article/file?type=thumbnail&id=10.1371/journal.pdig.0000115.e011" loading="lazy" class="inline-graphic"></span> on the label variable <em>Y</em>. In [<a href="#pdig.0000115.ref014" class="ref-tip">14</a>], the author proposes to discount the mass functions of all neighbors with the uncertainty level of their labels. In <a href="#pdig.0000115.e007">Eq (6)</a>, the term corresponding to the uncertainty level of the labels <span class="inline-formula"><img src="article/file?type=thumbnail&id=10.1371/journal.pdig.0000115.e012" loading="lazy" class="inline-graphic"></span> is added which results in <a href="#pdig.0000115.e013">Eq (7)</a>. <a name="pdig.0000115.e013" id="pdig.0000115.e013" class="link-target"></a><span class="equation"><img src="article/file?type=thumbnail&id=10.1371/journal.pdig.0000115.e013" loading="lazy" class="inline-graphic"><span class="note">(7)</span></span></p> <a id="article1.body1.sec4.sec1.sec2.p5" name="article1.body1.sec4.sec1.sec2.p5" class="link-target"></a><p>In this study we deal with a regression problem since the number of COVID-19 daily deaths is numerical. We therefore extend the original E<em>K</em>NN model, that was initially designed for classification problems, to discrete regression tasks. To do so we removed all <em>γ</em><sub><em>l</em></sub> parameters (defined relatively to categorical class labels) from Eqs (<a href="#pdig.0000115.e007">6</a>) and (<a href="#pdig.0000115.e013">7</a>), which results in Eqs (<a href="#pdig.0000115.e014">8</a>) and (<a href="#pdig.0000115.e015">9</a>).</p> <a id="article1.body1.sec4.sec1.sec2.p6" name="article1.body1.sec4.sec1.sec2.p6" class="link-target"></a><p>E<em>K</em>NN uncertainty model for regression: <a name="pdig.0000115.e014" id="pdig.0000115.e014" class="link-target"></a><span class="equation"><img src="article/file?type=thumbnail&id=10.1371/journal.pdig.0000115.e014" loading="lazy" class="inline-graphic"><span class="note">(8)</span></span> <a name="pdig.0000115.e015" id="pdig.0000115.e015" class="link-target"></a><span class="equation"><img src="article/file?type=thumbnail&id=10.1371/journal.pdig.0000115.e015" loading="lazy" class="inline-graphic"><span class="note">(9)</span></span></p> <a id="article1.body1.sec4.sec1.sec2.p7" name="article1.body1.sec4.sec1.sec2.p7" class="link-target"></a><p>The implementation of this extension of the E<em>K</em>NN algorithm to regression is available on our <a href="https://github.com/lgi2p/evidential_imputation">github</a> (<a href="https://github.com/lgi2p/evidential_imputation">https://github.com/lgi2p/evidential_imputation</a>).</p> <a id="article1.body1.sec4.sec1.sec2.p8" name="article1.body1.sec4.sec1.sec2.p8" class="link-target"></a><p>For decision making (<em>i.e.</em> prediction) we used the pignistic transform <em>BetP</em><sub><em>s</em></sub> of <em>m</em><sub><em>s</em></sub> in order to predict the pignistic expectation.</p> <a id="article1.body1.sec4.sec1.sec2.p9" name="article1.body1.sec4.sec1.sec2.p9" class="link-target"></a><p>The predicted label of a new observation <em>x</em><sub><em>s</em></sub> is: <a name="pdig.0000115.e016" id="pdig.0000115.e016" class="link-target"></a><span class="equation"><img src="article/file?type=thumbnail&id=10.1371/journal.pdig.0000115.e016" loading="lazy" class="inline-graphic"><span class="note">(10)</span></span></p> </div> </div> <div id="section2" class="section toc-section"><a id="sec008" name="sec008" class="link-target" title="4.2 Formalism"></a> <h3>4.2 Formalism</h3> <a id="article1.body1.sec4.sec2.p1" name="article1.body1.sec4.sec2.p1" class="link-target"></a><p>In this section we present the formalism of both the regression problem and the missing value imputation task.</p> <div id="section1" class="section toc-section"><a id="sec009" name="sec009" class="link-target" title="4.2.1 Predictive problem"></a><h4>4.2.1 Predictive problem.</h4><a id="article1.body1.sec4.sec2.sec1.p1" name="article1.body1.sec4.sec2.sec1.p1" class="link-target"></a><p>Let <em>D</em> = (<em>x</em><sub><em>t</em></sub>, <em>y</em><sub><em>t</em></sub>)<sub><em>t</em>=0,…,<em>T</em></sub> be a dataset where <span class="inline-formula"><img src="article/file?type=thumbnail&id=10.1371/journal.pdig.0000115.e017" loading="lazy" class="inline-graphic"></span> and <span class="inline-formula"><img src="article/file?type=thumbnail&id=10.1371/journal.pdig.0000115.e018" loading="lazy" class="inline-graphic"></span> are respectively the feature and label values at time <em>t</em>. We suppose that some label values are missing, <em>i.e.</em> some <em>y</em><sub><em>t</em></sub> values are not known. The aim of the regression task is to approximate a function <em>f</em> mapping current and past features values to future labels: <a name="pdig.0000115.e019" id="pdig.0000115.e019" class="link-target"></a><span class="equation"><img src="article/file?type=thumbnail&id=10.1371/journal.pdig.0000115.e019" loading="lazy" class="inline-graphic"><span class="note">(11)</span></span> where <em>h</em> is the prediction horizon, <em>q</em> a number of past features and label values to consider. This regression modelling implies that, at any timestamp <em>t</em>, the number of deaths <em>y</em><sub><em>t</em></sub> can be predicted from the concatenation of the sets of previous number of death (<em>y</em><sub><em>t</em>−<em>h</em></sub>, …, <em>y</em><sub><em>t</em>−(<em>h</em>+<em>q</em>)</sub>) and of previous number of cases (<em>x</em><sub><em>t</em>−<em>h</em></sub>, …, <em>x</em><sub><em>t</em>−(<em>h</em>+<em>q</em>)</sub>).</p> <a id="article1.body1.sec4.sec2.sec1.p2" name="article1.body1.sec4.sec2.sec1.p2" class="link-target"></a><p>Predicting deaths from data restricted to past number of death and cases is not usual in COVID forecasting works since some useful variables as the <em>basic reproduction number</em> <em>R</em><sub>0</sub>, hospital entries, exits and intensive care daily numbers, state health measures (confinement, etc) are generally used for predicting future deaths. In our case we chose to restrict to deaths and cases variables as most of the other previously stated variables are usually incomplete in public datasets. Indeed, our work is based on the E<em>K</em>NN model which can deal with uncertain labels but not uncertain features (in its initial form).</p> <a id="article1.body1.sec4.sec2.sec1.p3" name="article1.body1.sec4.sec2.sec1.p3" class="link-target"></a><p>Moreover, restricting ourselves to only 2 types of data (deaths and cases) makes experiments easier to run. Nevertheless, all this work can be easily extended to high-dimensionality features provided they are not incomplete in dataset. It is worth mentioning that some work has extended the E<em>K</em>NN model to uncertain features by computing distances between examples based on Jousselme distance which can be computed between belief function and thus between uncertain features [<a href="#pdig.0000115.ref024" class="ref-tip">24</a>].</p> <a id="article1.body1.sec4.sec2.sec1.p4" name="article1.body1.sec4.sec2.sec1.p4" class="link-target"></a><p>Since some of <em>D</em>’s values are missing, the imputation process has to occur upstream. In <a href="#pdig.0000115.e016">Eq (10)</a> past labels <em>y</em><sub><em>t</em></sub> are inputs of the function as historic features. Therefore removing incomplete examples introduces irregular timestamps in the data, which is inconsistent with the regularity hypothesis of most time series treatments.</p> </div> <div id="section2" class="section toc-section"><a id="sec010" name="sec010" class="link-target" title="4.2.2 Imputation problem"></a><h4>4.2.2 Imputation problem.</h4><a id="article1.body1.sec4.sec2.sec2.p1" name="article1.body1.sec4.sec2.sec2.p1" class="link-target"></a><p>Let us consider <span class="inline-formula"><img src="article/file?type=thumbnail&id=10.1371/journal.pdig.0000115.e020" loading="lazy" class="inline-graphic"></span> the <em>U</em> known previous label values with <em>p</em><sub><em>U</em></sub> < … < <em>p</em><sub>1</sub> before a missing label <em>y</em><sub><em>t</em></sub> and <span class="inline-formula"><img src="article/file?type=thumbnail&id=10.1371/journal.pdig.0000115.e021" loading="lazy" class="inline-graphic"></span> with <em>n</em><sub>1</sub> < … < <em>n</em><sub><em>R</em></sub> the next <em>N</em> known values, <em>U</em> and <em>R</em> are hyper-parameters that have to be tuned.</p> <a id="article1.body1.sec4.sec2.sec2.p2" name="article1.body1.sec4.sec2.sec2.p2" class="link-target"></a><p>In the example presented in <a href="#pdig-0000115-g007">Fig 7</a> we have <em>P</em> = 2 and <em>N</em> = 3.</p> <a class="link-target" id="pdig-0000115-g007" name="pdig-0000115-g007"></a><div class="figure" data-doi="10.1371/journal.pdig.0000115.g007"><div class="img-box"><a title="Click for larger image" href="article/figure/image?size=medium&id=10.1371/journal.pdig.0000115.g007" data-doi="10.1371/journal.pdig.0000115" data-uri="10.1371/journal.pdig.0000115.g007"><img src="article/figure/image?size=inline&id=10.1371/journal.pdig.0000115.g007" alt="thumbnail" class="thumbnail" loading="lazy"></a><div class="expand"></div></div><div class="figure-inline-download"> Download: <ul><li><a href="article/figure/powerpoint?id=10.1371/journal.pdig.0000115.g007"><div class="definition-label">PPT</div><div class="definition-description">PowerPoint slide</div></a></li><li><a href="article/figure/image?download&size=large&id=10.1371/journal.pdig.0000115.g007"><div class="definition-label">PNG</div><div class="definition-description">larger image</div></a></li><li><a href="article/figure/image?download&size=original&id=10.1371/journal.pdig.0000115.g007"><div class="definition-label">TIFF</div><div class="definition-description">original image</div></a></li></ul></div><div class="figcaption"><span>Fig 7. </span> Chronological data imputation.</div><p class="caption_target"></p><p class="caption_object"><a href="https://doi.org/10.1371/journal.pdig.0000115.g007"> https://doi.org/10.1371/journal.pdig.0000115.g007</a></p></div><a id="article1.body1.sec4.sec2.sec2.p3" name="article1.body1.sec4.sec2.sec2.p3" class="link-target"></a><p>The aim of the imputation process is to compute or <em>impute</em> a value <span class="inline-formula"><img src="article/file?type=thumbnail&id=10.1371/journal.pdig.0000115.e022" loading="lazy" class="inline-graphic"></span> for all missing <em>y</em><sub><em>t</em></sub>.</p> </div> </div> <div id="section3" class="section toc-section"><a id="sec011" name="sec011" class="link-target" title="4.3 Imputation methods"></a> <h3>4.3 Imputation methods</h3> <a id="article1.body1.sec4.sec3.p1" name="article1.body1.sec4.sec3.p1" class="link-target"></a><p>In this section we present three imputation methods to impute <span class="inline-formula"><img src="article/file?type=thumbnail&id=10.1371/journal.pdig.0000115.e023" loading="lazy" class="inline-graphic"></span>. The first one is the Last Observation Carried Forward (LOCF) method that replaces missing values with the last known value. The second one is the Centered Moving Average imputation (CMA) method that takes into account the dynamical nature of the data, and imputes missing values from the nearest future and past values. The last one is the “time-E<em>K</em>NN” (TEKNN) imputation method that applies the E<em>K</em>NN algorithm with a temporal distance to predict missing values, this method also takes into account the dynamical nature of the data as CMA method. The three imputation methods considered in this study are based on the use of those past and future label values.</p> <div id="section1" class="section toc-section"><a id="sec012" name="sec012" class="link-target" title="4.3.1 Last Observation Carried Forward (LOCF)"></a><h4>4.3.1 Last Observation Carried Forward (LOCF).</h4><a id="article1.body1.sec4.sec3.sec1.p1" name="article1.body1.sec4.sec3.sec1.p1" class="link-target"></a><p>Let <em>y</em><sub><em>t</em></sub> be a missing label value and <span class="inline-formula"><img src="article/file?type=thumbnail&id=10.1371/journal.pdig.0000115.e024" loading="lazy" class="inline-graphic"></span> the last known value. The LOCF imputation is simply <span class="inline-formula"><img src="article/file?type=thumbnail&id=10.1371/journal.pdig.0000115.e025" loading="lazy" class="inline-graphic"></span> as illustrated in <a href="#pdig-0000115-g008">Fig 8</a>.</p> <a class="link-target" id="pdig-0000115-g008" name="pdig-0000115-g008"></a><div class="figure" data-doi="10.1371/journal.pdig.0000115.g008"><div class="img-box"><a title="Click for larger image" href="article/figure/image?size=medium&id=10.1371/journal.pdig.0000115.g008" data-doi="10.1371/journal.pdig.0000115" data-uri="10.1371/journal.pdig.0000115.g008"><img src="article/figure/image?size=inline&id=10.1371/journal.pdig.0000115.g008" alt="thumbnail" class="thumbnail" loading="lazy"></a><div class="expand"></div></div><div class="figure-inline-download"> Download: <ul><li><a href="article/figure/powerpoint?id=10.1371/journal.pdig.0000115.g008"><div class="definition-label">PPT</div><div class="definition-description">PowerPoint slide</div></a></li><li><a href="article/figure/image?download&size=large&id=10.1371/journal.pdig.0000115.g008"><div class="definition-label">PNG</div><div class="definition-description">larger image</div></a></li><li><a href="article/figure/image?download&size=original&id=10.1371/journal.pdig.0000115.g008"><div class="definition-label">TIFF</div><div class="definition-description">original image</div></a></li></ul></div><div class="figcaption"><span>Fig 8. </span> LOCF imputation.</div><p class="caption_target"></p><p class="caption_object"><a href="https://doi.org/10.1371/journal.pdig.0000115.g008"> https://doi.org/10.1371/journal.pdig.0000115.g008</a></p></div></div> <div id="section2" class="section toc-section"><a id="sec013" name="sec013" class="link-target" title="4.3.2 Centered Moving Average (CMA) imputation"></a><h4>4.3.2 Centered Moving Average (CMA) imputation.</h4><a id="article1.body1.sec4.sec3.sec2.p1" name="article1.body1.sec4.sec3.sec2.p1" class="link-target"></a><p>Here we preset a method taking into account the dynamical nature of the data. It is based on the intuition that labels close in time are likely to have close values. More simply, the idea of the CMA imputation method is to impute the missing <em>y</em><sub><em>t</em></sub> from the nearest known past and future labels (see <a href="#pdig-0000115-g009">Fig 9</a>).</p> <a class="link-target" id="pdig-0000115-g009" name="pdig-0000115-g009"></a><div class="figure" data-doi="10.1371/journal.pdig.0000115.g009"><div class="img-box"><a title="Click for larger image" href="article/figure/image?size=medium&id=10.1371/journal.pdig.0000115.g009" data-doi="10.1371/journal.pdig.0000115" data-uri="10.1371/journal.pdig.0000115.g009"><img src="article/figure/image?size=inline&id=10.1371/journal.pdig.0000115.g009" alt="thumbnail" class="thumbnail" loading="lazy"></a><div class="expand"></div></div><div class="figure-inline-download"> Download: <ul><li><a href="article/figure/powerpoint?id=10.1371/journal.pdig.0000115.g009"><div class="definition-label">PPT</div><div class="definition-description">PowerPoint slide</div></a></li><li><a href="article/figure/image?download&size=large&id=10.1371/journal.pdig.0000115.g009"><div class="definition-label">PNG</div><div class="definition-description">larger image</div></a></li><li><a href="article/figure/image?download&size=original&id=10.1371/journal.pdig.0000115.g009"><div class="definition-label">TIFF</div><div class="definition-description">original image</div></a></li></ul></div><div class="figcaption"><span>Fig 9. </span> CMA imputation.</div><p class="caption_target"></p><p class="caption_object"><a href="https://doi.org/10.1371/journal.pdig.0000115.g009"> https://doi.org/10.1371/journal.pdig.0000115.g009</a></p></div><a id="article1.body1.sec4.sec3.sec2.p2" name="article1.body1.sec4.sec3.sec2.p2" class="link-target"></a><p>The past and future label values are averaged according to the duration between them and the missing label <em>y</em><sub><em>t</em></sub>. Let <span class="inline-formula"><img src="article/file?type=thumbnail&id=10.1371/journal.pdig.0000115.e026" loading="lazy" class="inline-graphic"></span> and <span class="inline-formula"><img src="article/file?type=thumbnail&id=10.1371/journal.pdig.0000115.e027" loading="lazy" class="inline-graphic"></span> be respectively the time shifts between the time <em>t</em> of a missing label <em>y</em><sub><em>t</em></sub> and the time of the previous and next known label values, we have: <a name="pdig.0000115.e028" id="pdig.0000115.e028" class="link-target"></a><span class="equation"><img src="article/file?type=thumbnail&id=10.1371/journal.pdig.0000115.e028" loading="lazy" class="inline-graphic"><span class="note">(12)</span></span> with <span class="inline-formula"><img src="article/file?type=thumbnail&id=10.1371/journal.pdig.0000115.e029" loading="lazy" class="inline-graphic"></span> <ul class="bulleted"> <li><a name="pdig.0000115.e030" id="pdig.0000115.e030" class="link-target"></a><span class="equation"><img src="article/file?type=thumbnail&id=10.1371/journal.pdig.0000115.e030" loading="lazy" class="inline-graphic"><span class="note">(13)</span></span></li> <li><a name="pdig.0000115.e031" id="pdig.0000115.e031" class="link-target"></a><span class="equation"><img src="article/file?type=thumbnail&id=10.1371/journal.pdig.0000115.e031" loading="lazy" class="inline-graphic"><span class="note">(14)</span></span></li> </ul></p> <a id="article1.body1.sec4.sec3.sec2.p3" name="article1.body1.sec4.sec3.sec2.p3" class="link-target"></a><p>Eqs (<a href="#pdig.0000115.e030">13</a>) and (<a href="#pdig.0000115.e031">14</a>) define a normalized temporal similarity <em>Sim</em>(<em>t</em>, <em>p</em><sub><em>u</em></sub>) between a measurement time <em>t</em> and one of the previous measurement times <em>p</em><sub><em>u</em></sub>. Those similarities are used to weigh past label values in <a href="#pdig.0000115.e028">Eq (12)</a>. Note that these equations can be directly transposed to measure the similarity <em>Sim</em>(<em>t</em>, <em>n</em><sub><em>r</em></sub>) between <em>t</em> and any next measurement time <em>n</em><sub><em>r</em></sub>.</p> <a id="article1.body1.sec4.sec3.sec2.p4" name="article1.body1.sec4.sec3.sec2.p4" class="link-target"></a><p>For the example of <a href="#pdig-0000115-g007">Fig 7</a>, the CMA imputation with <em>U</em> = <em>R</em> = 2 leads to <span class="inline-formula"><img src="article/file?type=thumbnail&id=10.1371/journal.pdig.0000115.e032" loading="lazy" class="inline-graphic"></span></p> </div> <div id="section3" class="section toc-section"><a id="sec014" name="sec014" class="link-target" title="4.3.3 Time-EKNN (TEKNN) imputation"></a><h4>4.3.3 Time-E<em>K</em>NN (TEKNN) imputation.</h4><a id="article1.body1.sec4.sec3.sec3.p1" name="article1.body1.sec4.sec3.sec3.p1" class="link-target"></a><p>The idea behind this imputation approach is to use the E<em>K</em>NN regression model to predict the missing label values based on the complete examples, <em>i.e.</em> where label values are known, that are the closest in time. This method could be seen as a de-centered extension of the CMA approach where the points used for imputation are the closest regardless of their temporal disposition around the missing value (before and/or after) as in <a href="#pdig-0000115-g010">Fig 10</a>. For this model too, the nearest neighbors on time have more weights on the imputed values <span class="inline-formula"><img src="article/file?type=thumbnail&id=10.1371/journal.pdig.0000115.e033" loading="lazy" class="inline-graphic"></span> (see <a href="#pdig.0000115.e014">Eq (8)</a>).</p> <a class="link-target" id="pdig-0000115-g010" name="pdig-0000115-g010"></a><div class="figure" data-doi="10.1371/journal.pdig.0000115.g010"><div class="img-box"><a title="Click for larger image" href="article/figure/image?size=medium&id=10.1371/journal.pdig.0000115.g010" data-doi="10.1371/journal.pdig.0000115" data-uri="10.1371/journal.pdig.0000115.g010"><img src="article/figure/image?size=inline&id=10.1371/journal.pdig.0000115.g010" alt="thumbnail" class="thumbnail" loading="lazy"></a><div class="expand"></div></div><div class="figure-inline-download"> Download: <ul><li><a href="article/figure/powerpoint?id=10.1371/journal.pdig.0000115.g010"><div class="definition-label">PPT</div><div class="definition-description">PowerPoint slide</div></a></li><li><a href="article/figure/image?download&size=large&id=10.1371/journal.pdig.0000115.g010"><div class="definition-label">PNG</div><div class="definition-description">larger image</div></a></li><li><a href="article/figure/image?download&size=original&id=10.1371/journal.pdig.0000115.g010"><div class="definition-label">TIFF</div><div class="definition-description">original image</div></a></li></ul></div><div class="figcaption"><span>Fig 10. </span> TE<em>K</em>NN imputation.</div><p class="caption_target"></p><p class="caption_object"><a href="https://doi.org/10.1371/journal.pdig.0000115.g010"> https://doi.org/10.1371/journal.pdig.0000115.g010</a></p></div><a id="article1.body1.sec4.sec3.sec3.p2" name="article1.body1.sec4.sec3.sec3.p2" class="link-target"></a><p>Regardless of the considered method, by nature the imputation of missing data involves some uncertainty associated with the imputed values. The next subsection proposes an uncertainty model for imputed time series data within the belief function framework.</p> </div> </div> <div id="section4" class="section toc-section"><a id="sec015" name="sec015" class="link-target" title="4.4 Uncertainty modeling for imputation methods"></a> <h3>4.4 Uncertainty modeling for imputation methods</h3> <a id="article1.body1.sec4.sec4.p1" name="article1.body1.sec4.sec4.p1" class="link-target"></a><p>In this subsection we present the uncertainty models associated with the 3 imputation methods described in the previous subsection. After uncertainty modeling, mass functions <span class="inline-formula"><img src="article/file?type=thumbnail&id=10.1371/journal.pdig.0000115.e034" loading="lazy" class="inline-graphic"></span> are assigned to each label <em>y</em><sub><em>t</em></sub> whether its value is missing or not. For known label values, categorical mass functions <span class="inline-formula"><img src="article/file?type=thumbnail&id=10.1371/journal.pdig.0000115.e035" loading="lazy" class="inline-graphic"></span> are assigned. The imputation methods associated with the uncertainty model allow the conversion of a precise but incomplete (in terms of labels) dataset (<em>x</em>, <em>y</em>) into a complete evidential dataset <span class="inline-formula"><img src="article/file?type=thumbnail&id=10.1371/journal.pdig.0000115.e036" loading="lazy" class="inline-graphic"></span>.</p> <a id="article1.body1.sec4.sec4.p2" name="article1.body1.sec4.sec4.p2" class="link-target"></a><p>Because of their dynamical nature, the LOCF and CMA imputation methods are associated with a time-based uncertainty model in the rest of this article. The idea behind this is that the closer in time the values used for the imputation of the missing labels are, the less uncertain the resulting imputed labels will be.</p> <a id="article1.body1.sec4.sec4.p3" name="article1.body1.sec4.sec4.p3" class="link-target"></a><p>The uncertainty model of the time-E<em>K</em>NN imputation approach is the E<em>K</em>NN’s evidential output.</p> </div> <div id="section5" class="section toc-section"><a id="sec016" name="sec016" class="link-target" title="4.5 Time based model"></a> <h3>4.5 Time based model</h3> <a id="article1.body1.sec4.sec5.p1" name="article1.body1.sec4.sec5.p1" class="link-target"></a><p>Once missing labels <em>y</em><sub><em>t</em></sub> have been imputed with the LOCF or CMA method into precise computed values <span class="inline-formula"><img src="article/file?type=thumbnail&id=10.1371/journal.pdig.0000115.e037" loading="lazy" class="inline-graphic"></span>, this paragraph describes the evidential uncertainty model associated with the <span class="inline-formula"><img src="article/file?type=thumbnail&id=10.1371/journal.pdig.0000115.e038" loading="lazy" class="inline-graphic"></span> values. This modeling aims at discounting or softening these imputed label values according to the duration without available data before and after them. This model is therefore based on the time shifts <span class="inline-formula"><img src="article/file?type=thumbnail&id=10.1371/journal.pdig.0000115.e039" loading="lazy" class="inline-graphic"></span> and <span class="inline-formula"><img src="article/file?type=thumbnail&id=10.1371/journal.pdig.0000115.e040" loading="lazy" class="inline-graphic"></span> between the missing values and the closest known ones. The larger those time shifts, the more uncertain the corresponding imputed label <span class="inline-formula"><img src="article/file?type=thumbnail&id=10.1371/journal.pdig.0000115.e041" loading="lazy" class="inline-graphic"></span>. Let <em>β</em> ∈ [0, 1] be an hyper-parameter controlling the uncertainty level, <em>i.e.</em> the decreasing speed of masses <span class="inline-formula"><img src="article/file?type=thumbnail&id=10.1371/journal.pdig.0000115.e042" loading="lazy" class="inline-graphic"></span> in regards to the time between the missing values and the closest ones. The mass function associated with the CMA and LOCF imputation methods is: <a name="pdig.0000115.e043" id="pdig.0000115.e043" class="link-target"></a><span class="equation"><img src="article/file?type=thumbnail&id=10.1371/journal.pdig.0000115.e043" loading="lazy" class="inline-graphic"><span class="note">(15)</span></span> This model was tested on several experimental set-ups to study its predictive performance.</p> <a id="article1.body1.sec4.sec5.p2" name="article1.body1.sec4.sec5.p2" class="link-target"></a><p>The overall articulation between imputation and associated uncertainty models is described in <a href="#pdig-0000115-g011">Fig 11</a>.</p> <a class="link-target" id="pdig-0000115-g011" name="pdig-0000115-g011"></a><div class="figure" data-doi="10.1371/journal.pdig.0000115.g011"><div class="img-box"><a title="Click for larger image" href="article/figure/image?size=medium&id=10.1371/journal.pdig.0000115.g011" data-doi="10.1371/journal.pdig.0000115" data-uri="10.1371/journal.pdig.0000115.g011"><img src="article/figure/image?size=inline&id=10.1371/journal.pdig.0000115.g011" alt="thumbnail" class="thumbnail" loading="lazy"></a><div class="expand"></div></div><div class="figure-inline-download"> Download: <ul><li><a href="article/figure/powerpoint?id=10.1371/journal.pdig.0000115.g011"><div class="definition-label">PPT</div><div class="definition-description">PowerPoint slide</div></a></li><li><a href="article/figure/image?download&size=large&id=10.1371/journal.pdig.0000115.g011"><div class="definition-label">PNG</div><div class="definition-description">larger image</div></a></li><li><a href="article/figure/image?download&size=original&id=10.1371/journal.pdig.0000115.g011"><div class="definition-label">TIFF</div><div class="definition-description">original image</div></a></li></ul></div><div class="figcaption"><span>Fig 11. </span> Evidential imputation scheme.</div><p class="caption_target"></p><p class="caption_object"><a href="https://doi.org/10.1371/journal.pdig.0000115.g011"> https://doi.org/10.1371/journal.pdig.0000115.g011</a></p></div><a id="article1.body1.sec4.sec5.p3" name="article1.body1.sec4.sec5.p3" class="link-target"></a><p>In this section an experiment is presented on a public COVID-19 dataset in which some labels (<em>i.e.</em> daily number of deaths) are <em>noised</em>, <em>i.e.</em> randomly removed and then imputed before learning and testing phases. After describing the dataset, we give the details of our noise procedure and the experimental set-up and finally we analyze the results.</p> </div> <div id="section6" class="section toc-section"><a id="sec017" name="sec017" class="link-target" title="4.6 Dataset"></a> <h3>4.6 Dataset</h3> <a id="article1.body1.sec4.sec6.p1" name="article1.body1.sec4.sec6.p1" class="link-target"></a><p>On the website <a href="http://ourworldindata.org/">ourworldindata</a> (<a href="https://ourworldindata.org/">https://ourworldindata.org/</a>) we used the French dataset containing the number of daily confirmed new cases (<em>x</em><sub><em>t</em></sub>)<sub><em>t</em>=1,…,<em>T</em></sub> and new deaths (<em>y</em><sub><em>t</em></sub>)<sub><em>t</em>=1,…,<em>T</em></sub>. <a href="#pdig-0000115-g012">Fig 12</a> shows the evolution of new cases and new deaths.</p> <a class="link-target" id="pdig-0000115-g012" name="pdig-0000115-g012"></a><div class="figure" data-doi="10.1371/journal.pdig.0000115.g012"><div class="img-box"><a title="Click for larger image" href="article/figure/image?size=medium&id=10.1371/journal.pdig.0000115.g012" data-doi="10.1371/journal.pdig.0000115" data-uri="10.1371/journal.pdig.0000115.g012"><img src="article/figure/image?size=inline&id=10.1371/journal.pdig.0000115.g012" alt="thumbnail" class="thumbnail" loading="lazy"></a><div class="expand"></div></div><div class="figure-inline-download"> Download: <ul><li><a href="article/figure/powerpoint?id=10.1371/journal.pdig.0000115.g012"><div class="definition-label">PPT</div><div class="definition-description">PowerPoint slide</div></a></li><li><a href="article/figure/image?download&size=large&id=10.1371/journal.pdig.0000115.g012"><div class="definition-label">PNG</div><div class="definition-description">larger image</div></a></li><li><a href="article/figure/image?download&size=original&id=10.1371/journal.pdig.0000115.g012"><div class="definition-label">TIFF</div><div class="definition-description">original image</div></a></li></ul></div><div class="figcaption"><span>Fig 12. </span> Evolution of new deaths and new cases.</div><p class="caption_target"></p><p class="caption_object"><a href="https://doi.org/10.1371/journal.pdig.0000115.g012"> https://doi.org/10.1371/journal.pdig.0000115.g012</a></p></div><a id="article1.body1.sec4.sec6.p2" name="article1.body1.sec4.sec6.p2" class="link-target"></a><p>As the detection policy has evolved between the 2 pandemic waves, the link between new cases and new deaths seems radically different during those 2 periods. As the number of daily new cases was clearly underestimated during the first wave, we restricted the experiment to the second wave. We finally had 367 complete daily observations for this dataset.</p> <a id="article1.body1.sec4.sec6.p3" name="article1.body1.sec4.sec6.p3" class="link-target"></a><p>As there are no missing values in the dataset, we randomly removed or <em>noised</em> some label values (<em>i.e.</em> new deaths). In the next subsection we give the details of our noise injection procedure.</p> </div> <div id="section7" class="section toc-section"><a id="sec018" name="sec018" class="link-target" title="4.7 Noise procedure"></a> <h3>4.7 Noise procedure</h3> <a id="article1.body1.sec4.sec7.p1" name="article1.body1.sec4.sec7.p1" class="link-target"></a><p>The proportion <em>v</em> ∈ [0, 1] of label values <em>y</em><sub><em>t</em></sub> to remove is the input of the procedure. In order to simulate plausible measurement errors, we removed labels <em>y</em><sub><em>t</em></sub> by time frame. Having randomly picked the frames centers at random, we generated reasonable frame sizes. The procedure is iterative until the proportion of removed labels reaches <em>v</em>.</p> <a id="article1.body1.sec4.sec7.p2" name="article1.body1.sec4.sec7.p2" class="link-target"></a><p><strong>Algorithm 1:</strong> Noise procedure for label values removing.</p> <a id="article1.body1.sec4.sec7.p3" name="article1.body1.sec4.sec7.p3" class="link-target"></a><p><strong>Data:</strong> original dataset, <em>v</em>: noise level</p> <a id="article1.body1.sec4.sec7.p4" name="article1.body1.sec4.sec7.p4" class="link-target"></a><p><strong>Result:</strong> noised dataset containing v% of missing labels</p> <a id="article1.body1.sec4.sec7.p5" name="article1.body1.sec4.sec7.p5" class="link-target"></a><p>removed ← 0</p> <a id="article1.body1.sec4.sec7.p6" name="article1.body1.sec4.sec7.p6" class="link-target"></a><p><em>s</em> = ⌊<em>v</em> ⋅ <em>T</em>⌋ number of labels to remove;</p> <a id="article1.body1.sec4.sec7.p7" name="article1.body1.sec4.sec7.p7" class="link-target"></a><p><strong>while</strong> <em>removed</em> < <em>s</em> <strong>do</strong></p> <a id="article1.body1.sec4.sec7.p8" name="article1.body1.sec4.sec7.p8" class="link-target"></a><p> frame center uniform random generation <em>c</em> ∈ [1:T];</p> <a id="article1.body1.sec4.sec7.p9" name="article1.body1.sec4.sec7.p9" class="link-target"></a><p> frame size <em>s</em> uniform random generation in ∈ {1, 2, 3};</p> <a id="article1.body1.sec4.sec7.p10" name="article1.body1.sec4.sec7.p10" class="link-target"></a><p> computation of label indices to remove {<em>c</em> − <em>s</em>, ….<em>c</em> + <em>s</em>};</p> <a id="article1.body1.sec4.sec7.p11" name="article1.body1.sec4.sec7.p11" class="link-target"></a><p> labels removal: <span class="inline-formula"><img src="article/file?type=thumbnail&id=10.1371/journal.pdig.0000115.e044" loading="lazy" class="inline-graphic"></span>;</p> <a id="article1.body1.sec4.sec7.p12" name="article1.body1.sec4.sec7.p12" class="link-target"></a><p> removed ← removed + 1 + 2<em>s</em>;</p> <a id="article1.body1.sec4.sec7.p13" name="article1.body1.sec4.sec7.p13" class="link-target"></a><p><strong>end</strong></p> </div> <div id="section8" class="section toc-section"><a id="sec019" name="sec019" class="link-target" title="4.8 Smoothing"></a> <h3>4.8 Smoothing</h3> <a id="article1.body1.sec4.sec8.p1" name="article1.body1.sec4.sec8.p1" class="link-target"></a><p>Because of the weekly constraint in health policy, the raw COVID-19 data are usually <em>sawtooth-shaped</em> curves. This implies that smoothing method can and should be applied in order to get values corresponding more to the reality than the very noisy raw ones. We chose to use a moving median of 7 days (labels and features). Doing so, we avoided biases without creating unreasonable values. Since smoothed data are usually very regular, imputing missing values on smoothed data is not a real issue, we therefore decided to smooth our noised dataset after imputation. All the implemented predictive models in our study were trained and tested on smooth data because of their higher level of reliability compared to the <em>sawtooth-shaped</em> ones. The whole process is represented in <a href="#pdig-0000115-g013">Fig 13</a></p> <a class="link-target" id="pdig-0000115-g013" name="pdig-0000115-g013"></a><div class="figure" data-doi="10.1371/journal.pdig.0000115.g013"><div class="img-box"><a title="Click for larger image" href="article/figure/image?size=medium&id=10.1371/journal.pdig.0000115.g013" data-doi="10.1371/journal.pdig.0000115" data-uri="10.1371/journal.pdig.0000115.g013"><img src="article/figure/image?size=inline&id=10.1371/journal.pdig.0000115.g013" alt="thumbnail" class="thumbnail" loading="lazy"></a><div class="expand"></div></div><div class="figure-inline-download"> Download: <ul><li><a href="article/figure/powerpoint?id=10.1371/journal.pdig.0000115.g013"><div class="definition-label">PPT</div><div class="definition-description">PowerPoint slide</div></a></li><li><a href="article/figure/image?download&size=large&id=10.1371/journal.pdig.0000115.g013"><div class="definition-label">PNG</div><div class="definition-description">larger image</div></a></li><li><a href="article/figure/image?download&size=original&id=10.1371/journal.pdig.0000115.g013"><div class="definition-label">TIFF</div><div class="definition-description">original image</div></a></li></ul></div><div class="figcaption"><span>Fig 13. </span> Experimental process.</div><p class="caption_target"></p><p class="caption_object"><a href="https://doi.org/10.1371/journal.pdig.0000115.g013"> https://doi.org/10.1371/journal.pdig.0000115.g013</a></p></div></div> <div id="section9" class="section toc-section"><a id="sec020" name="sec020" class="link-target" title="4.9 Experimental set-up"></a> <h3>4.9 Experimental set-up</h3> <a id="article1.body1.sec4.sec9.p1" name="article1.body1.sec4.sec9.p1" class="link-target"></a><p>In this subsection we present the chronological set-up of our experiment and the considered hyper-parameters spaces. Some hyper-parameters have been set <em>a priori</em>: <em>R</em> and <em>U</em> representing the width of the CMA approach (<a href="#pdig.0000115.e028">12</a>) were both fixed at 5 days. As hospital reorganization involves strong administrative constraints incompatible with too short or too large horizon, the considered prediction horizon <em>h</em> was 7 days. Finally, the uncertainty hyper-parameter of the time-based uncertainty model (<a href="#pdig.0000115.e043">15</a>) <em>β</em> was set at 0.05.</p> <a id="article1.body1.sec4.sec9.p2" name="article1.body1.sec4.sec9.p2" class="link-target"></a><p>For the other hyper-parameters, several configurations were considered: <ul class="bulleted"> <li>noise level: <em>v</em> ∈ {0, 0.1, …, 0.7}</li> <li>data historical length: <em>q</em> ∈ {1, 2, …, 7}</li> <li>number of neighbors for the E<em>K</em>NN regressors: <em>K</em> ∈ {1, 10, 20}</li> </ul></p> <a id="article1.body1.sec4.sec9.p3" name="article1.body1.sec4.sec9.p3" class="link-target"></a><p>The data historical length <em>q</em> represents the number of past data (deaths and cases) representing each training example. The first 21 dates are set aside for training, the predictions <em>y</em><sub><em>t</em></sub> are then computed iteratively at each date <em>t</em> from all the past couples data (<em>x</em><sub><em>t</em><sup>′</sup></sub>, <em>y</em><sub><em>t</em><sup>′</sup></sub>)<sub><em>t</em><sup>′</sup>=<em>t</em>−<em>h</em>,…,<em>t</em>−(<em>h</em>+<em>q</em>)</sub>. At first iteration of the chronological evaluation, the 21 first days are used as training data in order to predict the label value of the 21 + 7 = 28<sup><em>th</em></sup> day (with a prediction horizon of <em>h</em> = 7 days). After that, the training data are augmented by one date at each iteration, for example at the second iteration we use the 22 first days to predict the label values of the 29<sup><em>th</em></sup> day.</p> <a id="article1.body1.sec4.sec9.p4" name="article1.body1.sec4.sec9.p4" class="link-target"></a><p>Each complete chronological evaluation is repeated 50 times because of the randomness of the noise procedure and predictions are averaged. <a name="pdig.0000115.e045" id="pdig.0000115.e045" class="link-target"></a><span class="equation"><img src="article/file?type=thumbnail&id=10.1371/journal.pdig.0000115.e045" loading="lazy" class="inline-graphic"><span class="note">(16)</span></span> Since Ω<sub><em>Y</em></sub> must be defined before the prediction step in the regression of the E<em>K</em>NN we propose (see Eqs (<a href="#pdig.0000115.e014">8</a>), (<a href="#pdig.0000115.e015">9</a>) and (<a href="#pdig.0000115.e016">10</a>)), we redefine it at each iteration according to <a href="#pdig.0000115.e045">Eq 16</a> by updating the maximum label value in the training data. We chose a safety margin of 15% in regards to the real maximum number of deaths observed. As a baseline we considered the moving average that predicts the number of deaths for the next week as the average of the previous 2 weeks.</p> <a id="article1.body1.sec4.sec9.p5" name="article1.body1.sec4.sec9.p5" class="link-target"></a><p>Two types of figures are presented. The chronological evaluations (see Figs <a href="#pdig-0000115-g002">2</a> and <a href="#pdig-0000115-g003">3</a>) allow us to visually evaluate the predictions of the regression for different noise levels according to different imputation methods by comparing the predicted labels with the real ones. E<em>K</em>NN regression models are evaluated without and with uncertain models (“E<em>K</em>NN” and “E<em>K</em>NN uncertain labels”). In the former case, imputed training labels are considered certain whereas in the latter case uncertain models (see Eqs (<a href="#pdig.0000115.e043">15</a>) and (<a href="#pdig.0000115.e014">8</a>)) allow the E<em>K</em>NN regression to take into account data imputation uncertainty.</p> <a id="article1.body1.sec4.sec9.p6" name="article1.body1.sec4.sec9.p6" class="link-target"></a><p>The noise level sensitivity of the complete evaluations is represented in Figs <a href="#pdig-0000115-g006">6</a> and <a href="#pdig-0000115-g005">5</a> where the predictive performance is measured according to the noise level. The evaluation metric we considered is the Root Median Squared Error (RMedSE). We chose it rather than the standard Root Mean Square Error (RMSE) because of the high sensitivity of the mean operator to extreme values which are quite usual in the COVID-19 data.</p> <a id="article1.body1.sec4.sec9.p7" name="article1.body1.sec4.sec9.p7" class="link-target"></a><p>We evaluate the imputation methods by comparing the initial dataset with the imputed ones in terms of RMSE in order to evaluate the imputation errors that were unlikely to contain extreme values (see <a href="#pdig-0000115-g001">Fig 1</a>).</p> </div> </div> <div xmlns:plos="http://plos.org" class="toc-section"><a id="references" name="references" class="link-target" data-toc="references" title="References"></a><h2>References</h2><ol class="references"><li id="ref1"><span class="order">1. </span><a name="pdig.0000115.ref001" id="pdig.0000115.ref001" class="link-target"></a> Jerez J., Molina I., García-Laencina P., Alba E., Ribelles N., Martín, M., Franco L. Missing data imputation using statistical and machine learning methods in a real breast cancer problem. Artificial Intelligence In Medicine. 2010;50(2):105–115. pmid:20638252 <ul class="reflinks" data-doi="10.1016/j.artmed.2010.05.002"><li><a href="https://doi.org/10.1016/j.artmed.2010.05.002" data-author="doi-provided" data-cit="doi-provided" data-title="doi-provided" target="_new" title="Go to article"> View Article </a></li><li><a href="http://www.ncbi.nlm.nih.gov/pubmed/20638252" target="_new" title="Go to article in PubMed"> PubMed/NCBI </a></li><li><a href="http://scholar.google.com/scholar?q=Missing+data+imputation+using+statistical+and+machine+learning+methods+in+a+real+breast+cancer+problem+Jerez+2010" target="_new" title="Go to article in Google Scholar"> Google Scholar </a></li></ul></li><li id="ref2"><span class="order">2. </span><a name="pdig.0000115.ref002" id="pdig.0000115.ref002" class="link-target"></a> Lakshminaryan K., Harp S., Samed T. Imputation of missing data in industrial databases. Applied Intelligence. 1999;11(3):259–275. <ul class="reflinks" data-doi="10.1023/A:1008334909089"><li><a href="https://doi.org/10.1023/A:1008334909089" data-author="doi-provided" data-cit="doi-provided" data-title="doi-provided" target="_new" title="Go to article"> View Article </a></li><li><a href="http://scholar.google.com/scholar?q=Imputation+of+missing+data+in+industrial+databases+Lakshminaryan+1999" target="_new" title="Go to article in Google Scholar"> Google Scholar </a></li></ul></li><li id="ref3"><span class="order">3. </span><a name="pdig.0000115.ref003" id="pdig.0000115.ref003" class="link-target"></a> Rubin D. Inference and missing data. Biometrika. 1976; 63(3):581–592. <ul class="reflinks" data-doi="10.1093/biomet/63.3.581"><li><a href="https://doi.org/10.1093/biomet/63.3.581" data-author="doi-provided" data-cit="doi-provided" data-title="doi-provided" target="_new" title="Go to article"> View Article </a></li><li><a href="http://scholar.google.com/scholar?q=Inference+and+missing+data+Rubin+1976" target="_new" title="Go to article in Google Scholar"> Google Scholar </a></li></ul></li><li id="ref4"><span class="order">4. </span><a name="pdig.0000115.ref004" id="pdig.0000115.ref004" class="link-target"></a> Little R., Rubin D. Statistical analysis with missing data. (Wiley,2002). <a href="http://books.google.com/books?id=aYPwAAAAMAAJ">http://books.google.com/books?id=aYPwAAAAMAAJ</a>. <ul class="find-nolinks"></ul></li><li id="ref5"><span class="order">5. </span><a name="pdig.0000115.ref005" id="pdig.0000115.ref005" class="link-target"></a> Farhangfar A., Kurgan L., Dy J. Impact of imputation of missing values on classification error for discrete data. Pattern Recognition. 2008; 41(12):3692–3705. <ul class="reflinks" data-doi="10.1016/j.patcog.2008.05.019"><li><a href="https://doi.org/10.1016/j.patcog.2008.05.019" data-author="doi-provided" data-cit="doi-provided" data-title="doi-provided" target="_new" title="Go to article"> View Article </a></li><li><a href="http://scholar.google.com/scholar?q=Impact+of+imputation+of+missing+values+on+classification+error+for+discrete+data+Farhangfar+2008" target="_new" title="Go to article in Google Scholar"> Google Scholar </a></li></ul></li><li id="ref6"><span class="order">6. </span><a name="pdig.0000115.ref006" id="pdig.0000115.ref006" class="link-target"></a> Jacquin L., Imoussaten A., Trousset F., Montmain J., Perrin D. Evidential classification of incomplete data via imprecise relabelling: Application to plastic sorting. International Conference On Scalable Uncertainty Management. 2019;122–135. <ul class="reflinks" data-doi="10.1007/978-3-030-35514-2_10"><li><a href="https://doi.org/10.1007/978-3-030-35514-2_10" data-author="doi-provided" data-cit="doi-provided" data-title="doi-provided" target="_new" title="Go to article"> View Article </a></li><li><a href="http://scholar.google.com/scholar?q=Evidential+classification+of+incomplete+data+via+imprecise+relabelling%3A+Application+to+plastic+sorting+Jacquin+2019" target="_new" title="Go to article in Google Scholar"> Google Scholar </a></li></ul></li><li id="ref7"><span class="order">7. </span><a name="pdig.0000115.ref007" id="pdig.0000115.ref007" class="link-target"></a> Alizadehsani R., Roshanzamir M., Hussain S., Khosravi A., Koohestani A., Zangooei M., Abdar M., Beykikhoshk A., Shoeibi A., Zare A., Panahiazar M., Nahavandi S., Srinivasan D., Atiya A., Acharya U. Handling of uncertainty in medical data using machine learning and probability theory techniques: A review of 30 years (1991–2020). Annals Of Operations Research. 2021;1–42. pmid:33776178 <ul class="reflinks" data-doi="10.1007/s10479-021-04006-2"><li><a href="https://doi.org/10.1007/s10479-021-04006-2" data-author="doi-provided" data-cit="doi-provided" data-title="doi-provided" target="_new" title="Go to article"> View Article </a></li><li><a href="http://www.ncbi.nlm.nih.gov/pubmed/33776178" target="_new" title="Go to article in PubMed"> PubMed/NCBI </a></li><li><a href="http://scholar.google.com/scholar?q=Handling+of+uncertainty+in+medical+data+using+machine+learning+and+probability+theory+techniques%3A+A+review+of+30+years+%281991%E2%80%932020%29+Alizadehsani+2021" target="_new" title="Go to article in Google Scholar"> Google Scholar </a></li></ul></li><li id="ref8"><span class="order">8. </span><a name="pdig.0000115.ref008" id="pdig.0000115.ref008" class="link-target"></a> Zadeh L. Fuzzy logic, neural networks, and soft computing. Fuzzy Sets, Fuzzy Logic, And Fuzzy Systems: Selected Papers By Lotfi A Zadeh. 1996;775–782. <ul class="reflinks" data-doi="10.1142/9789814261302_0040"><li><a href="https://doi.org/10.1142/9789814261302_0040" data-author="doi-provided" data-cit="doi-provided" data-title="doi-provided" target="_new" title="Go to article"> View Article </a></li><li><a href="http://scholar.google.com/scholar?q=Fuzzy+logic%2C+neural+networks%2C+and+soft+computing+Zadeh+1996" target="_new" title="Go to article in Google Scholar"> Google Scholar </a></li></ul></li><li id="ref9"><span class="order">9. </span><a name="pdig.0000115.ref009" id="pdig.0000115.ref009" class="link-target"></a> Zadeh L. Fuzzy Sets. Information And Control. 1965;8:338–353, <a href="http://www-bisc.cs.berkeley.edu/Zadeh-1965.pdf">http://www-bisc.cs.berkeley.edu/Zadeh-1965.pdf</a>. <ul class="reflinks"><li><a href="#" data-author="Zadeh" data-cit="%0AZadehL.%0AFuzzy%20Sets.%20Information%20And%20Control.%201965%3B8%3A338%E2%80%93353%2C%20http%3A%2F%2Fwww-bisc.cs.berkeley.edu%2FZadeh-1965.pdf.%20doi%3A%2010.1016%2FS0019-9958%2865%2990241-X" data-title="Fuzzy%20Sets" target="_new" title="Go to article in CrossRef"> View Article </a></li><li><a href="http://scholar.google.com/scholar?q=Fuzzy+Sets+Zadeh+1965" target="_new" title="Go to article in Google Scholar"> Google Scholar </a></li></ul></li><li id="ref10"><span class="order">10. </span><a name="pdig.0000115.ref010" id="pdig.0000115.ref010" class="link-target"></a> Dubois D., Prade H. Possibility theory: qualitative and quantitative aspects. Quantified Representation Of Uncertainty And Imprecision,1998; 169–226. <ul class="reflinks" data-doi="10.1007/978-94-017-1735-9_6"><li><a href="https://doi.org/10.1007/978-94-017-1735-9_6" data-author="doi-provided" data-cit="doi-provided" data-title="doi-provided" target="_new" title="Go to article"> View Article </a></li><li><a href="http://scholar.google.com/scholar?q=Possibility+theory%3A+qualitative+and+quantitative+aspects+Dubois+1998" target="_new" title="Go to article in Google Scholar"> Google Scholar </a></li></ul></li><li id="ref11"><span class="order">11. </span><a name="pdig.0000115.ref011" id="pdig.0000115.ref011" class="link-target"></a>Walley, P. Statistical reasoning with imprecise probabilities. (Chapman,1991). ISBN: 0412286602 9780412286605. <ul class="find-nolinks"></ul></li><li id="ref12"><span class="order">12. </span><a name="pdig.0000115.ref012" id="pdig.0000115.ref012" class="link-target"></a> Dempster A. Upper and lower probabilities induced by a multivalued mapping. Classic Works Of The Dempster-Shafer Theory Of Belief Functions. 2008;57–72. <ul class="reflinks" data-doi="10.1007/978-3-540-44792-4_3"><li><a href="https://doi.org/10.1007/978-3-540-44792-4_3" data-author="doi-provided" data-cit="doi-provided" data-title="doi-provided" target="_new" title="Go to article"> View Article </a></li><li><a href="http://scholar.google.com/scholar?q=Upper+and+lower+probabilities+induced+by+a+multivalued+mapping+Dempster+2008" target="_new" title="Go to article in Google Scholar"> Google Scholar </a></li></ul></li><li id="ref13"><span class="order">13. </span><a name="pdig.0000115.ref013" id="pdig.0000115.ref013" class="link-target"></a>Shafer, G. A mathematical theory of evidence. (Princeton university press,1976) <ul class="find-nolinks"></ul></li><li id="ref14"><span class="order">14. </span><a name="pdig.0000115.ref014" id="pdig.0000115.ref014" class="link-target"></a> Denœux T. A k-nearest neighbor classification rule based on Dempster-Shafer theory. IEEE Transactions On Systems, Man, And Cybernetics. 1995;25(5):804–813. <ul class="reflinks" data-doi="10.1109/21.376493"><li><a href="https://doi.org/10.1109/21.376493" data-author="doi-provided" data-cit="doi-provided" data-title="doi-provided" target="_new" title="Go to article"> View Article </a></li><li><a href="http://scholar.google.com/scholar?q=A+k-nearest+neighbor+classification+rule+based+on+Dempster-Shafer+theory+Den%C5%93ux+1995" target="_new" title="Go to article in Google Scholar"> Google Scholar </a></li></ul></li><li id="ref15"><span class="order">15. </span><a name="pdig.0000115.ref015" id="pdig.0000115.ref015" class="link-target"></a> Denœux T. A neural network classifier based on Dempster-Shafer theory. IEEE Transactions On Systems, Man, And Cybernetics-Part A: Systems And Humans. 2001; 30(2):131–150. <ul class="reflinks"><li><a href="#" data-author="Den%C5%93ux" data-cit="%0ADen%C5%93uxT.%20A%20neural%20network%20classifier%20based%20on%20Dempster-Shafer%20theory.%20IEEE%20Transactions%20On%20Systems%2C%20Man%2C%20And%20Cybernetics-Part%20A%3A%20Systems%20And%20Humans.%202001%3B%2030%282%29%3A131%E2%80%93150." data-title="A%20neural%20network%20classifier%20based%20on%20Dempster-Shafer%20theory" target="_new" title="Go to article in CrossRef"> View Article </a></li><li><a href="http://scholar.google.com/scholar?q=A+neural+network+classifier+based+on+Dempster-Shafer+theory+Den%C5%93ux+2001" target="_new" title="Go to article in Google Scholar"> Google Scholar </a></li></ul></li><li id="ref16"><span class="order">16. </span><a name="pdig.0000115.ref016" id="pdig.0000115.ref016" class="link-target"></a> Elouedi Z., Mellouli K., Smets P. Belief decision trees: theoretical foundations. International Journal Of Approximate Reasoning. 2001;28(2-3):91–124. <ul class="reflinks" data-doi="10.1016/S0888-613X(01)00045-7"><li><a href="https://doi.org/10.1016/S0888-613X(01)00045-7" data-author="doi-provided" data-cit="doi-provided" data-title="doi-provided" target="_new" title="Go to article"> View Article </a></li><li><a href="http://scholar.google.com/scholar?q=Belief+decision+trees%3A+theoretical+foundations+Elouedi+2001" target="_new" title="Go to article in Google Scholar"> Google Scholar </a></li></ul></li><li id="ref17"><span class="order">17. </span><a name="pdig.0000115.ref017" id="pdig.0000115.ref017" class="link-target"></a>Sutton-Charani, N., Destercke, S., Denœux, T. Learning decision trees from uncertain data with an evidential EM approach. 2013 12th International Conference On Machine Learning And Applications. 2013;1:111-116. <ul class="find-nolinks"></ul></li><li id="ref18"><span class="order">18. </span><a name="pdig.0000115.ref018" id="pdig.0000115.ref018" class="link-target"></a>Azim, S. & Aggarwal, S. Hybrid model for data imputation: using fuzzy c means and multi layer perceptron. <em>2014 IEEE International Advance Computing Conference (IACC)</em>. 2014; 1281-1285 <ul class="find-nolinks"></ul></li><li id="ref19"><span class="order">19. </span><a name="pdig.0000115.ref019" id="pdig.0000115.ref019" class="link-target"></a> Li D., Deogun J., Spaulding W., Shuart B. Towards missing data imputation: a study of fuzzy k-means clustering method. International Conference On Rough Sets And Current Trends In Computing. 2004;573–579. <ul class="reflinks" data-doi="10.1007/978-3-540-25929-9_70"><li><a href="https://doi.org/10.1007/978-3-540-25929-9_70" data-author="doi-provided" data-cit="doi-provided" data-title="doi-provided" target="_new" title="Go to article"> View Article </a></li><li><a href="http://scholar.google.com/scholar?q=Towards+missing+data+imputation%3A+a+study+of+fuzzy+k-means+clustering+method+Li+2004" target="_new" title="Go to article in Google Scholar"> Google Scholar </a></li></ul></li><li id="ref20"><span class="order">20. </span><a name="pdig.0000115.ref020" id="pdig.0000115.ref020" class="link-target"></a> Nishanth K., Ravi V. Probabilistic neural network based categorical data imputation. Neurocomputing, 2016; 218:17–25. <ul class="reflinks" data-doi="10.1016/j.neucom.2016.08.044"><li><a href="https://doi.org/10.1016/j.neucom.2016.08.044" data-author="doi-provided" data-cit="doi-provided" data-title="doi-provided" target="_new" title="Go to article"> View Article </a></li><li><a href="http://scholar.google.com/scholar?q=Probabilistic+neural+network+based+categorical+data+imputation+Nishanth+2016" target="_new" title="Go to article in Google Scholar"> Google Scholar </a></li></ul></li><li id="ref21"><span class="order">21. </span><a name="pdig.0000115.ref021" id="pdig.0000115.ref021" class="link-target"></a> Hamizadeh J., Moradi M. Enhancing data analysis: uncertainty-resistance method for handling incomplete data. Applied Intelligence. 2020;50(1):74–86. <ul class="reflinks" data-doi="10.1007/s10489-019-01514-4"><li><a href="https://doi.org/10.1007/s10489-019-01514-4" data-author="doi-provided" data-cit="doi-provided" data-title="doi-provided" target="_new" title="Go to article"> View Article </a></li><li><a href="http://scholar.google.com/scholar?q=Enhancing+data+analysis%3A+uncertainty-resistance+method+for+handling+incomplete+data+Hamizadeh+2020" target="_new" title="Go to article in Google Scholar"> Google Scholar </a></li></ul></li><li id="ref22"><span class="order">22. </span><a name="pdig.0000115.ref022" id="pdig.0000115.ref022" class="link-target"></a>Liu, Z., Pan, Q., Mercier, G., Dezert, J. Pattern classification with missing data using belief functions. 17th International Conference On Information Fusion (FUSION 2014;1-8, <a href="https://hal-onera.archives-ouvertes.fr/hal-01070496">https://hal-onera.archives-ouvertes.fr/hal-01070496</a>. <ul class="find-nolinks"></ul></li><li id="ref23"><span class="order">23. </span><a name="pdig.0000115.ref023" id="pdig.0000115.ref023" class="link-target"></a> Jousselme A., Maupin P. Comparison of uncertainty representations for missing data in information retrieval. Proceedings Of The 16th International Conference On Information Fusion. 2013;1902–1909. <ul class="reflinks"><li><a href="#" data-author="Jousselme" data-cit="%0AJousselmeA.%2C%20MaupinP.%20Comparison%20of%20uncertainty%20representations%20for%20missing%20data%20in%20information%20retrieval.%20Proceedings%20Of%20The%2016th%20International%20Conference%20On%20Information%20Fusion.%202013%3B1902%E2%80%931909." data-title="Comparison%20of%20uncertainty%20representations%20for%20missing%20data%20in%20information%20retrieval" target="_new" title="Go to article in CrossRef"> View Article </a></li><li><a href="http://scholar.google.com/scholar?q=Comparison+of+uncertainty+representations+for+missing+data+in+information+retrieval+Jousselme+2013" target="_new" title="Go to article in Google Scholar"> Google Scholar </a></li></ul></li><li id="ref24"><span class="order">24. </span><a name="pdig.0000115.ref024" id="pdig.0000115.ref024" class="link-target"></a> Trabelsi A., Elouedi Z., Lefevre E. Decision tree classifiers for evidential attribute values and class labels. Fuzzy Sets And Systems. 2019;366:46–62. <ul class="reflinks" data-doi="10.1016/j.fss.2018.11.006"><li><a href="https://doi.org/10.1016/j.fss.2018.11.006" data-author="doi-provided" data-cit="doi-provided" data-title="doi-provided" target="_new" title="Go to article"> View Article </a></li><li><a href="http://scholar.google.com/scholar?q=Decision+tree+classifiers+for+evidential+attribute+values+and+class+labels+Trabelsi+2019" target="_new" title="Go to article in Google Scholar"> Google Scholar </a></li></ul></li><li id="ref25"><span class="order">25. </span><a name="pdig.0000115.ref025" id="pdig.0000115.ref025" class="link-target"></a> Tsang S., Kao B., Yip K., Ho W., Lee S. Decision trees for uncertain data. IEEE Transactions On Knowledge And Data Engineering. 2009;23(1):64–78. <ul class="reflinks" data-doi="10.1109/TKDE.2009.175"><li><a href="https://doi.org/10.1109/TKDE.2009.175" data-author="doi-provided" data-cit="doi-provided" data-title="doi-provided" target="_new" title="Go to article"> View Article </a></li><li><a href="http://scholar.google.com/scholar?q=Decision+trees+for+uncertain+data+Tsang+2009" target="_new" title="Go to article in Google Scholar"> Google Scholar </a></li></ul></li><li id="ref26"><span class="order">26. </span><a name="pdig.0000115.ref026" id="pdig.0000115.ref026" class="link-target"></a> Jiao L., Pan Q., Feng X., Yang F. An evidential k-nearest neighbor classification method with weighted attributes. Proceedings Of The 16th International Conference On Information Fusion. 2013;145–150. <ul class="reflinks"><li><a href="#" data-author="Jiao" data-cit="%0AJiaoL.%2C%20PanQ.%2C%20FengX.%2C%20YangF.%20An%20evidential%20k-nearest%20neighbor%20classification%20method%20with%20weighted%20attributes.%20Proceedings%20Of%20The%2016th%20International%20Conference%20On%20Information%20Fusion.%202013%3B145%E2%80%93150." data-title="An%20evidential%20k-nearest%20neighbor%20classification%20method%20with%20weighted%20attributes" target="_new" title="Go to article in CrossRef"> View Article </a></li><li><a href="http://scholar.google.com/scholar?q=An+evidential+k-nearest+neighbor+classification+method+with+weighted+attributes+Jiao+2013" target="_new" title="Go to article in Google Scholar"> Google Scholar </a></li></ul></li><li id="ref27"><span class="order">27. </span><a name="pdig.0000115.ref027" id="pdig.0000115.ref027" class="link-target"></a> Florea M., Jousselme A., Bossé, É., Grenier D. Robust combination rules for evidence theory. Information Fusion, 2009;10(2):183–197. <ul class="reflinks" data-doi="10.1016/j.inffus.2008.08.007"><li><a href="https://doi.org/10.1016/j.inffus.2008.08.007" data-author="doi-provided" data-cit="doi-provided" data-title="doi-provided" target="_new" title="Go to article"> View Article </a></li><li><a href="http://scholar.google.com/scholar?q=Robust+combination+rules+for+evidence+theory+Florea+2009" target="_new" title="Go to article in Google Scholar"> Google Scholar </a></li></ul></li><li id="ref28"><span class="order">28. </span><a name="pdig.0000115.ref028" id="pdig.0000115.ref028" class="link-target"></a> Smets P. Belief functions: the disjunctive rule of combination and the generalized Bayesian theorem. International Journal Of Approximate Reasoning, 1993;9(1):1–35. <ul class="reflinks" data-doi="10.1016/0888-613X(93)90005-X"><li><a href="https://doi.org/10.1016/0888-613X(93)90005-X" data-author="doi-provided" data-cit="doi-provided" data-title="doi-provided" target="_new" title="Go to article"> View Article </a></li><li><a href="http://scholar.google.com/scholar?q=Belief+functions%3A+the+disjunctive+rule+of+combination+and+the+generalized+Bayesian+theorem+Smets+1993" target="_new" title="Go to article in Google Scholar"> Google Scholar </a></li></ul></li><li id="ref29"><span class="order">29. </span><a name="pdig.0000115.ref029" id="pdig.0000115.ref029" class="link-target"></a> Smets P., Kennes R. The transferable belief model. Artificial Intelligence, 1994;66(2):191–234. <ul class="reflinks" data-doi="10.1016/0004-3702(94)90026-4"><li><a href="https://doi.org/10.1016/0004-3702(94)90026-4" data-author="doi-provided" data-cit="doi-provided" data-title="doi-provided" target="_new" title="Go to article"> View Article </a></li><li><a href="http://scholar.google.com/scholar?q=The+transferable+belief+model+Smets+1994" target="_new" title="Go to article in Google Scholar"> Google Scholar </a></li></ul></li><li id="ref30"><span class="order">30. </span><a name="pdig.0000115.ref030" id="pdig.0000115.ref030" class="link-target"></a> Smets P. Constructing the Pignistic Probability Function in a Context of Uncertainty. UAI. 1989;89:29–40. <ul class="reflinks"><li><a href="#" data-author="Smets" data-cit="%0ASmetsP.%20Constructing%20the%20Pignistic%20Probability%20Function%20in%20a%20Context%20of%20Uncertainty.%20UAI.%201989%3B89%3A29%E2%80%9340." data-title="Constructing%20the%20Pignistic%20Probability%20Function%20in%20a%20Context%20of%20Uncertainty" target="_new" title="Go to article in CrossRef"> View Article </a></li><li><a href="http://scholar.google.com/scholar?q=Constructing+the+Pignistic+Probability+Function+in+a+Context+of+Uncertainty+Smets+1989" target="_new" title="Go to article in Google Scholar"> Google Scholar </a></li></ul></li><li id="ref31"><span class="order">31. </span><a name="pdig.0000115.ref031" id="pdig.0000115.ref031" class="link-target"></a> Smets P. Decision making in the TBM: the necessity of the pignistic transformation. International Journal Of Approximate Reasoning. 2005;38(2):133–147. <ul class="reflinks" data-doi="10.1016/j.ijar.2004.05.003"><li><a href="https://doi.org/10.1016/j.ijar.2004.05.003" data-author="doi-provided" data-cit="doi-provided" data-title="doi-provided" target="_new" title="Go to article"> View Article </a></li><li><a href="http://scholar.google.com/scholar?q=Decision+making+in+the+TBM%3A+the+necessity+of+the+pignistic+transformation+Smets+2005" target="_new" title="Go to article in Google Scholar"> Google Scholar </a></li></ul></li><li id="ref32"><span class="order">32. </span><a name="pdig.0000115.ref032" id="pdig.0000115.ref032" class="link-target"></a> Yuan B., Yue X., Lv Y., Denœux T. Evidential deep neural networks for uncertain data classification. International Conference On Knowledge Science, Engineering And Management. 2020;427–437. <ul class="reflinks"><li><a href="#" data-author="Yuan" data-cit="%0AYuanB.%2C%20YueX.%2C%20LvY.%2C%20Den%C5%93uxT.%20Evidential%20deep%20neural%20networks%20for%20uncertain%20data%20classification.%20International%20Conference%20On%20Knowledge%20Science%2C%20Engineering%20And%20Management.%202020%3B427%E2%80%93437." data-title="Evidential%20deep%20neural%20networks%20for%20uncertain%20data%20classification" target="_new" title="Go to article in CrossRef"> View Article </a></li><li><a href="http://scholar.google.com/scholar?q=Evidential+deep+neural+networks+for+uncertain+data+classification+Yuan+2020" target="_new" title="Go to article in Google Scholar"> Google Scholar </a></li></ul></li></ol></div> <div class="ref-tooltip"> <div class="ref_tooltip-content"> </div> </div> </div> </div> </div> </section> <aside class="article-aside">  <div class="dload-menu"> <div class="dload-pdf"> <a href="/digitalhealth/article/file?id=10.1371/journal.pdig.0000115&type=printable" id="downloadPdf" target="_blank">Download PDF</a> </div> <div data-js-tooltip-hover="trigger" class="dload-hover">  <ul class="dload-xml" data-js-tooltip-hover="target"> <li><a href="/digitalhealth/article/citation?id=10.1371/journal.pdig.0000115" id="downloadCitation">Citation</a></li> <li><a href="/digitalhealth/article/file?id=10.1371/journal.pdig.0000115&type=manuscript" id="downloadXml">XML</a> </li> </ul> </div> </div> <div class="aside-container"> <div class="print-article" id="printArticle" data-js-tooltip-hover="trigger"> <a href="#" onclick="window.print(); return false;" class="preventDefault" id="printBrowser">Print</a> </div> <div class="share-article" id="shareArticle" data-js-tooltip-hover="trigger"> Share <ul data-js-tooltip-hover="target" class="share-options" id="share-options"> <li><a href="https://www.reddit.com/submit?url=https%3A%2F%2Fdx.plos.org%2F10.1371%2Fjournal.pdig.0000115" id="shareReddit" target="_blank" title="Submit to Reddit"><img src="/resource/img/icon.reddit.16.png" width="16" height="16" alt="Reddit">Reddit</a></li> <li><a href="https://www.facebook.com/share.php?u=https%3A%2F%2Fdx.plos.org%2F10.1371%2Fjournal.pdig.0000115&t=Uncertain imputation for time-series forecasting: Application to COVID-19 daily mortality prediction" id="shareFacebook" target="_blank" title="Share on Facebook"><img src="/resource/img/icon.fb.16.png" width="16" height="16" alt="Facebook">Facebook</a></li> <li><a href="https://www.linkedin.com/shareArticle?url=https%3A%2F%2Fdx.plos.org%2F10.1371%2Fjournal.pdig.0000115&title=Uncertain imputation for time-series forecasting: Application to COVID-19 daily mortality prediction&summary=Checkout this article I found at PLOS" id="shareLinkedIn" target="_blank" title="Add to LinkedIn"><img src="/resource/img/icon.linkedin.16.png" width="16" height="16" alt="LinkedIn">LinkedIn</a></li> <li><a href="https://www.mendeley.com/import/?url=https%3A%2F%2Fdx.plos.org%2F10.1371%2Fjournal.pdig.0000115" id="shareMendeley" target="_blank" title="Add to Mendeley"><img src="/resource/img/icon.mendeley.16.png" width="16" height="16" alt="Mendeley">Mendeley</a></li> <li><a href="https://twitter.com/intent/tweet?url=https%3A%2F%2Fdx.plos.org%2F10.1371%2Fjournal.pdig.0000115&text=%23PLOSDigitalHealth%3A%20Uncertain imputation for time-series forecasting: Application to COVID-19 daily mortality prediction" target="_blank" title="share on Twitter" id="twitter-share-link"><img src="/resource/img/icon.twtr.16.png" width="16" height="16" alt="Twitter">Twitter</a></li> <li><a href="mailto:?subject=Uncertain imputation for time-series forecasting: Application to COVID-19 daily mortality prediction&body=I%20thought%20you%20would%20find%20this%20article%20interesting.%20From%20PLOS Digital Health:%20https%3A%2F%2Fdx.plos.org%2F10.1371%2Fjournal.pdig.0000115" id="shareEmail" rel="noreferrer" aria-label="Email"><img src="/resource/img/icon.email.16.png" width="16" height="16" alt="Email">Email</a></li> <script src="/resource/js/components/tweet140.js" type="text/javascript"></script> </ul> </div> </div>  <script src="https://crossmark-cdn.crossref.org/widget/v2.0/widget.js"></script> <a aria-label="Check for updates via CrossMark" data-target="crossmark"> <img alt="Check for updates via CrossMark" width="150" src="https://crossmark-cdn.crossref.org/widget/v2.0/logos/CROSSMARK_BW_horizontal.svg"> </a>  <div class="aside-container collections-aside-container"></div> <div class="skyscraper-container"> <div class="title">Advertisement</div>  <div class='advertisement' id='div-gpt-ad-1458247671871-1' style='width:160px; height:600px;'> <script type='text/javascript'> googletag.cmd.push(function() { googletag.display('div-gpt-ad-1458247671871-1'); }); </script> </div> </div> <div class="subject-areas-container"> <h3>Subject Areas <div id="subjInfo">?</div> <div id="subjInfoText"> <p>For more information about PLOS Subject Areas, click <a href="https://github.com/PLOS/plos-thesaurus/blob/master/README.md" target="_blank" title="Link opens in new window">here</a>.</p> <span class="inline-intro">We want your feedback.</span> Do these Subject Areas make sense for this article? Click the target next to the incorrect Subject Area and let us know. Thanks for your help! </div> </h3> <ul id="subjectList"> <li> <a class="taxo-term" title="Search for articles about Forecasting" href="/digitalhealth/search?filterSubjects=Forecasting&filterJournals=PLOSDigitalHealth&q=">Forecasting</a> <span class="taxo-flag"> </span> <div class="taxo-tooltip" data-categoryname="Forecasting"><p class="taxo-explain">Is the Subject Area <strong>"Forecasting"</strong> applicable to this article? <button id="noFlag" data-action="remove">Yes</button> <button id="flagIt" value="flagno" data-action="add">No</button></p> <p class="taxo-confirm">Thanks for your feedback.</p> </div> </li> <li> <a class="taxo-term" title="Search for articles about COVID 19" href="/digitalhealth/search?filterSubjects=COVID+19&filterJournals=PLOSDigitalHealth&q=">COVID 19</a> <span class="taxo-flag"> </span> <div class="taxo-tooltip" data-categoryname="COVID 19"><p class="taxo-explain">Is the Subject Area <strong>"COVID 19"</strong> applicable to this article? <button id="noFlag" data-action="remove">Yes</button> <button id="flagIt" value="flagno" data-action="add">No</button></p> <p class="taxo-confirm">Thanks for your feedback.</p> </div> </li> <li> <a class="taxo-term" title="Search for articles about Machine learning" href="/digitalhealth/search?filterSubjects=Machine+learning&filterJournals=PLOSDigitalHealth&q=">Machine learning</a> <span class="taxo-flag"> </span> <div class="taxo-tooltip" data-categoryname="Machine learning"><p class="taxo-explain">Is the Subject Area <strong>"Machine learning"</strong> applicable to this article? <button id="noFlag" data-action="remove">Yes</button> <button id="flagIt" value="flagno" data-action="add">No</button></p> <p class="taxo-confirm">Thanks for your feedback.</p> </div> </li> <li> <a class="taxo-term" title="Search for articles about Machine learning algorithms" href="/digitalhealth/search?filterSubjects=Machine+learning+algorithms&filterJournals=PLOSDigitalHealth&q=">Machine learning algorithms</a> <span class="taxo-flag"> </span> <div class="taxo-tooltip" data-categoryname="Machine learning algorithms"><p class="taxo-explain">Is the Subject Area <strong>"Machine learning algorithms"</strong> applicable to this article? <button id="noFlag" data-action="remove">Yes</button> <button id="flagIt" value="flagno" data-action="add">No</button></p> <p class="taxo-confirm">Thanks for your feedback.</p> </div> </li> <li> <a class="taxo-term" title="Search for articles about Decision making" href="/digitalhealth/search?filterSubjects=Decision+making&filterJournals=PLOSDigitalHealth&q=">Decision making</a> <span class="taxo-flag"> </span> <div class="taxo-tooltip" data-categoryname="Decision making"><p class="taxo-explain">Is the Subject Area <strong>"Decision making"</strong> applicable to this article? <button id="noFlag" data-action="remove">Yes</button> <button id="flagIt" value="flagno" data-action="add">No</button></p> <p class="taxo-confirm">Thanks for your feedback.</p> </div> </li> <li> <a class="taxo-term" title="Search for articles about Pandemics" href="/digitalhealth/search?filterSubjects=Pandemics&filterJournals=PLOSDigitalHealth&q=">Pandemics</a> <span class="taxo-flag"> </span> <div class="taxo-tooltip" data-categoryname="Pandemics"><p class="taxo-explain">Is the Subject Area <strong>"Pandemics"</strong> applicable to this article? <button id="noFlag" data-action="remove">Yes</button> <button id="flagIt" value="flagno" data-action="add">No</button></p> <p class="taxo-confirm">Thanks for your feedback.</p> </div> </li> <li> <a class="taxo-term" title="Search for articles about Probability distribution" href="/digitalhealth/search?filterSubjects=Probability+distribution&filterJournals=PLOSDigitalHealth&q=">Probability distribution</a> <span class="taxo-flag"> </span> <div class="taxo-tooltip" data-categoryname="Probability distribution"><p class="taxo-explain">Is the Subject Area <strong>"Probability distribution"</strong> applicable to this article? <button id="noFlag" data-action="remove">Yes</button> <button id="flagIt" value="flagno" data-action="add">No</button></p> <p class="taxo-confirm">Thanks for your feedback.</p> </div> </li> <li> <a class="taxo-term" title="Search for articles about Time measurement" href="/digitalhealth/search?filterSubjects=Time+measurement&filterJournals=PLOSDigitalHealth&q=">Time measurement</a> <span class="taxo-flag"> </span> <div class="taxo-tooltip" data-categoryname="Time measurement"><p class="taxo-explain">Is the Subject Area <strong>"Time measurement"</strong> applicable to this article? <button id="noFlag" data-action="remove">Yes</button> <button id="flagIt" value="flagno" data-action="add">No</button></p> <p class="taxo-confirm">Thanks for your feedback.</p> </div> </li> </ul> </div> <div id="subjectErrors"></div> </aside> </div> </main> <footer id="pageftr"> <div class="row"> <div class="block x-small"> <ul class="nav nav-secondary"> <li class="ftr-header"><a href="https://plos.org/publications/journals/">Publications</a></li> <li><a href="/plosbiology/" id="ftr-bio">PLOS Biology</a></li> <li><a href="/climate/" id="ftr-climate">PLOS Climate</a></li> <li><a href="/complexsystems/" id="ftr-complex-systems">PLOS Complex Systems</a></li> <li><a href="/ploscompbiol/" id="ftr-compbio">PLOS Computational Biology</a></li> <li><a href="/digitalhealth/" id="ftr-digitalhealth">PLOS Digital Health</a></li> <li><a href="/plosgenetics/" id="ftr-gen">PLOS Genetics</a></li> <li><a href="/globalpublichealth/" id="ftr-globalpublichealth">PLOS Global Public Health</a></li> </ul> </div> <div class="block x-small"> <ul class="nav nav-secondary"> <li class="ftr-header"> </li> <li><a href="/plosmedicine/" id="ftr-med">PLOS Medicine</a></li> <li><a href="/mentalhealth/" id="ftr-mental-health">PLOS Mental Health</a></li> <li><a href="/plosntds/" id="ftr-ntds">PLOS Neglected Tropical Diseases</a></li> <li><a href="/plosone/" id="ftr-one">PLOS ONE</a></li> <li><a href="/plospathogens/" id="ftr-path">PLOS Pathogens</a></li> <li><a href="/sustainabilitytransformation/" id="ftr-sustainabilitytransformation">PLOS Sustainability and Transformation</a></li> <li><a href="/water/" id="ftr-water">PLOS Water</a></li> </ul> </div> <div class="block xx-small"> <ul class="nav nav-tertiary"> <li> <a href="https://plos.org" id="ftr-home">Home</a> </li> <li> <a href="https://blogs.plos.org" id="ftr-blog">Blogs</a> </li> <li> <a href="https://collections.plos.org/" id="ftr-collections">Collections</a> </li> <li> <a href="mailto:webmaster@plos.org" id="ftr-feedback">Give feedback</a> </li> <li> <a href="/digitalhealth/lockss-manifest" id="ftr-lockss">LOCKSS</a> </li> </ul> </div> <div class="block xx-small"> <ul class="nav nav-primary"> <li><a href="https://plos.org/privacy-policy" id="ftr-privacy">Privacy Policy</a></li> <li><a href="https://plos.org/terms-of-use" id="ftr-terms">Terms of Use</a></li> <li><a href="https://plos.org/advertise/" id="ftr-advertise">Advertise</a></li> <li><a href="https://plos.org/media-inquiries" id="ftr-media">Media Inquiries</a></li> <li><a href="https://plos.org/contact" id="ftr-contact">Contact</a></li> </ul> </div> </div> <div class="row"> <p> <img src="/resource/img/logo-plos-footer.png" alt="PLOS" class="logo-footer"/> <span class="footer-non-profit-statement">PLOS is a nonprofit 501(c)(3) corporation, #C2354500, based in San Francisco, California, US</span> </p> <div class="block"> </div> </div> <script src="/resource/js/global.js" type="text/javascript"></script> </footer> <script type="text/javascript"> var ArticleData = { doi: '10.1371/journal.pdig.0000115', title: '<article-title xmlns:mml=\"http://www.w3.org/1998/Math/MathML\" xmlns:xlink=\"http://www.w3.org/1999/xlink\">Uncertain imputation for time-series forecasting: Application to COVID-19 daily mortality prediction<\/article-title>', date: 'Oct 25, 2022' }; </script> <script src="/resource/js/components/show_onscroll.js" type="text/javascript"></script> <script src="/resource/js/components/pagination.js" type="text/javascript"></script> <script src="/resource/js/vendor/spin.js" type="text/javascript"></script> <script src="/resource/js/pages/article.js" type="text/javascript"></script> <script src="/resource/js/pages/article_references.js" type="text/javascript"></script> <script src="/resource/js/pages/article_sidebar.js" type="text/javascript"></script> <script src="/resource/js/vendor/foundation/foundation.dropdown.js" type="text/javascript"></script> <script src="/resource/js/components/table_open.js" type="text/javascript"></script> <script src="/resource/js/components/figshare.js" type="text/javascript"></script> <script src="/resource/js/vendor/jquery.panzoom.min.js" type="text/javascript"></script> <script src="/resource/js/vendor/jquery.mousewheel.js" type="text/javascript"></script> <script src="/resource/js/components/lightbox.js" type="text/javascript"></script> <script src="/resource/js/pages/article_body.js" type="text/javascript"></script>    <script type="text/x-mathjax-config"> MathJax.Hub.Config({ "HTML-CSS": { scale: 100, availableFonts: ["STIX","TeX"], preferredFont: "STIX", webFont: "STIX-Web", linebreaks: { automatic: false } }, jax: ["input/MathML", "output/HTML-CSS"] }); </script> <script type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.1/MathJax.js?config=MML_HTMLorMML"></script> <div class="reveal-modal-bg"></div> </body> </html>