CINXE.COM
The role of machine learning in clinical research: transforming the future of evidence generation | Trials | Full Text
<!DOCTYPE html> <html lang="en" class="no-js"> <head> <meta charset="UTF-8"> <meta http-equiv="X-UA-Compatible" content="IE=edge"> <meta name="applicable-device" content="pc,mobile"> <meta name="viewport" content="width=device-width, initial-scale=1"> <title>The role of machine learning in clinical research: transforming the future of evidence generation | Trials | Full Text</title> <meta name="citation_abstract" content="Interest in the application of machine learning (ML) to the design, conduct, and analysis of clinical trials has grown, but the evidence base for such applications has not been surveyed. This manuscript reviews the proceedings of a multi-stakeholder conference to discuss the current and future state of ML for clinical research. Key areas of clinical trial methodology in which ML holds particular promise and priority areas for further investigation are presented alongside a narrative review of evidence supporting the use of ML across the clinical trial spectrum. Conference attendees included stakeholders, such as biomedical and ML researchers, representatives from the US Food and Drug Administration (FDA), artificial intelligence technology and data analytics companies, non-profit organizations, patient advocacy groups, and pharmaceutical companies. ML contributions to clinical research were highlighted in the pre-trial phase, cohort selection and participant management, and data collection and analysis. A particular focus was paid to the operational and philosophical barriers to ML in clinical research. Peer-reviewed evidence was noted to be lacking in several areas. ML holds great promise for improving the efficiency and quality of clinical research, but substantial barriers remain, the surmounting of which will require addressing significant gaps in evidence."/> <meta name="journal_id" content="13063"/> <meta name="dc.title" content="The role of machine learning in clinical research: transforming the future of evidence generation"/> <meta name="dc.source" content="Trials 2021 22:1"/> <meta name="dc.format" content="text/html"/> <meta name="dc.publisher" content="BioMed Central"/> <meta name="dc.date" content="2021-08-16"/> <meta name="dc.type" content="Letter"/> <meta name="dc.language" content="En"/> <meta name="dc.copyright" content="2021 The Author(s)"/> <meta name="dc.rights" content="2021 The Author(s)"/> <meta name="dc.rightsAgent" content="reprints@biomedcentral.com"/> <meta name="dc.description" content="Interest in the application of machine learning (ML) to the design, conduct, and analysis of clinical trials has grown, but the evidence base for such applications has not been surveyed. This manuscript reviews the proceedings of a multi-stakeholder conference to discuss the current and future state of ML for clinical research. Key areas of clinical trial methodology in which ML holds particular promise and priority areas for further investigation are presented alongside a narrative review of evidence supporting the use of ML across the clinical trial spectrum. Conference attendees included stakeholders, such as biomedical and ML researchers, representatives from the US Food and Drug Administration (FDA), artificial intelligence technology and data analytics companies, non-profit organizations, patient advocacy groups, and pharmaceutical companies. ML contributions to clinical research were highlighted in the pre-trial phase, cohort selection and participant management, and data collection and analysis. A particular focus was paid to the operational and philosophical barriers to ML in clinical research. Peer-reviewed evidence was noted to be lacking in several areas. ML holds great promise for improving the efficiency and quality of clinical research, but substantial barriers remain, the surmounting of which will require addressing significant gaps in evidence."/> <meta name="prism.issn" content="1745-6215"/> <meta name="prism.publicationName" content="Trials"/> <meta name="prism.publicationDate" content="2021-08-16"/> <meta name="prism.volume" content="22"/> <meta name="prism.number" content="1"/> <meta name="prism.section" content="Letter"/> <meta name="prism.startingPage" content="1"/> <meta name="prism.endingPage" content="15"/> <meta name="prism.copyright" content="2021 The Author(s)"/> <meta name="prism.rightsAgent" content="reprints@biomedcentral.com"/> <meta name="prism.url" content="https://trialsjournal.biomedcentral.com/articles/10.1186/s13063-021-05489-x"/> <meta name="prism.doi" content="doi:10.1186/s13063-021-05489-x"/> <meta name="citation_pdf_url" content="https://trialsjournal.biomedcentral.com/counter/pdf/10.1186/s13063-021-05489-x"/> <meta name="citation_fulltext_html_url" content="https://trialsjournal.biomedcentral.com/articles/10.1186/s13063-021-05489-x"/> <meta name="citation_journal_title" content="Trials"/> <meta name="citation_journal_abbrev" content="Trials"/> <meta name="citation_publisher" content="BioMed Central"/> <meta name="citation_issn" content="1745-6215"/> <meta name="citation_title" content="The role of machine learning in clinical research: transforming the future of evidence generation"/> <meta name="citation_volume" content="22"/> <meta name="citation_issue" content="1"/> <meta name="citation_publication_date" content="2021/12"/> <meta name="citation_online_date" content="2021/08/16"/> <meta name="citation_firstpage" content="1"/> <meta name="citation_lastpage" content="15"/> <meta name="citation_article_type" content="Commentary"/> <meta name="citation_fulltext_world_readable" content=""/> <meta name="citation_language" content="en"/> <meta name="dc.identifier" content="doi:10.1186/s13063-021-05489-x"/> <meta name="DOI" content="10.1186/s13063-021-05489-x"/> <meta name="size" content="303830"/> <meta name="citation_doi" content="10.1186/s13063-021-05489-x"/> <meta name="citation_springer_api_url" content="http://api.springer.com/xmldata/jats?q=doi:10.1186/s13063-021-05489-x&api_key="/> <meta name="description" content="Interest in the application of machine learning (ML) to the design, conduct, and analysis of clinical trials has grown, but the evidence base for such applications has not been surveyed. This manuscript reviews the proceedings of a multi-stakeholder conference to discuss the current and future state of ML for clinical research. Key areas of clinical trial methodology in which ML holds particular promise and priority areas for further investigation are presented alongside a narrative review of evidence supporting the use of ML across the clinical trial spectrum. Conference attendees included stakeholders, such as biomedical and ML researchers, representatives from the US Food and Drug Administration (FDA), artificial intelligence technology and data analytics companies, non-profit organizations, patient advocacy groups, and pharmaceutical companies. ML contributions to clinical research were highlighted in the pre-trial phase, cohort selection and participant management, and data collection and analysis. A particular focus was paid to the operational and philosophical barriers to ML in clinical research. Peer-reviewed evidence was noted to be lacking in several areas. ML holds great promise for improving the efficiency and quality of clinical research, but substantial barriers remain, the surmounting of which will require addressing significant gaps in evidence."/> <meta name="dc.creator" content="Weissler, E. Hope"/> <meta name="dc.creator" content="Naumann, Tristan"/> <meta name="dc.creator" content="Andersson, Tomas"/> <meta name="dc.creator" content="Ranganath, Rajesh"/> <meta name="dc.creator" content="Elemento, Olivier"/> <meta name="dc.creator" content="Luo, Yuan"/> <meta name="dc.creator" content="Freitag, Daniel F."/> <meta name="dc.creator" content="Benoit, James"/> <meta name="dc.creator" content="Hughes, Michael C."/> <meta name="dc.creator" content="Khan, Faisal"/> <meta name="dc.creator" content="Slater, Paul"/> <meta name="dc.creator" content="Shameer, Khader"/> <meta name="dc.creator" content="Roe, Matthew"/> <meta name="dc.creator" content="Hutchison, Emmette"/> <meta name="dc.creator" content="Kollins, Scott H."/> <meta name="dc.creator" content="Broedl, Uli"/> <meta name="dc.creator" content="Meng, Zhaoling"/> <meta name="dc.creator" content="Wong, Jennifer L."/> <meta name="dc.creator" content="Curtis, Lesley"/> <meta name="dc.creator" content="Huang, Erich"/> <meta name="dc.creator" content="Ghassemi, Marzyeh"/> <meta name="dc.subject" content="Medicine/Public Health, general"/> <meta name="dc.subject" content="Biomedicine, general"/> <meta name="dc.subject" content="Statistics for Life Sciences, Medicine, Health Sciences"/> <meta name="citation_reference" content="citation_journal_title=Nature.; citation_title=Improved protein structure prediction using potentials from deep learning; citation_author=AW Senior, R Evans, J Jumper, J Kirkpatrick, L Sifre, T Green, C Qin, A Žídek, AWR Nelson, A Bridgland, H Penedones, S Petersen, K Simonyan, S Crossan, P Kohli, DT Jones, D Silver, K Kavukcuoglu, D Hassabis; citation_volume=577; citation_issue=7792; citation_publication_date=2020; citation_pages=706-710; citation_doi=10.1038/s41586-019-1923-7; citation_id=CR1"/> <meta name="citation_reference" content="citation_title=Constructing large scale biomedical knowledge bases from scratch with rapid annotation of interpretable patterns; citation_inbook_title=Proceedings of the 18th BioNLP Workshop and Shared Task; citation_publication_date=2019; citation_id=CR2; citation_author=JTA Fauqueur; citation_author=T Togia"/> <meta name="citation_reference" content="Jia R, Wong C, Poon H. Document-level N-ary relation extraction with multiscale representation learning. Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers); 2019; Minneapolis: Association for Computational Linguistics.  https://ui.adsabs.harvard.edu/abs/2019arXiv190402347J ."/> <meta name="citation_reference" content="citation_journal_title=BMC Bioinformatics.; citation_title=Machine learning prediction of oncology drug targets based on protein and network properties; citation_author=Z Dezso, M Ceccarelli; citation_volume=21; citation_issue=1; citation_publication_date=2020; citation_pages=104; citation_doi=10.1186/s12859-020-3442-9; citation_id=CR4"/> <meta name="citation_reference" content="Bagherian M, Sabeti E, Wang K, Sartor MA, Nikolovska-Coleska Z, Najarian K. Machine learning approaches and databases for prediction of drug-target interaction: a survey paper. Brief Bioinform. 2021;22(1):247–69. https://doi.org/10.1093/bib/bbz157 ."/> <meta name="citation_reference" content="Liu QAM, Brockschmidt M, Gaunt AL. Constrained graph variational autoencoders for molecule design. NeurIPS 2018. 2018;arXiv:1805.09076:7806–15."/> <meta name="citation_reference" content="citation_journal_title=Nat Commun.; citation_title=A Bayesian machine learning approach for drug target identification using diverse data types; citation_author=NS Madhukar, PK Khade, L Huang, K Gayvert, G Galletti, M Stogniew, JE Allen, P Giannakakou, O Elemento; citation_volume=10; citation_issue=1; citation_publication_date=2019; citation_pages=5221; citation_doi=10.1038/s41467-019-12928-6; citation_id=CR7"/> <meta name="citation_reference" content="citation_journal_title=Adv Mater.; citation_title=Beyond ternary OPV: high-throughput experimentation and self-driving laboratories optimize multicomponent systems; citation_author=S Langner, F Hase, JD Perea, T Stubhan, J Hauch, LM Roch; citation_volume=32; citation_issue=14; citation_publication_date=2020; citation_doi=10.1002/adma.201907801; citation_id=CR8"/> <meta name="citation_reference" content="citation_journal_title=Nature.; citation_title=Controlling an organic synthesis robot with machine learning to search for new reactivity; citation_author=JM Granda, L Donina, V Dragone, DL Long, L Cronin; citation_volume=559; citation_issue=7714; citation_publication_date=2018; citation_pages=377-381; citation_doi=10.1038/s41586-018-0307-8; citation_id=CR9"/> <meta name="citation_reference" content="Koh D. Sumitomo Dainippon Pharma and Exscientia achieve breakthrough in AI drug discovery: Healthcare IT News - Portland, ME: Healthcare IT News; 2020."/> <meta name="citation_reference" content="citation_journal_title=Clin Pharmacol Ther.; citation_title=The future is now: model-based clinical trial design for Alzheimer's disease; citation_author=K Romero, K Ito, JA Rogers, D Polhamus, R Qiu, D Stephenson, R Mohs, R Lalonde, V Sinha, Y Wang, D Brown, M Isaac, S Vamvakas, R Hemmings, L Pani, LJ Bain, B Corrigan; citation_volume=97; citation_issue=3; citation_publication_date=2015; citation_pages=210-214; citation_doi=10.1002/cpt.16; citation_id=CR11"/> <meta name="citation_reference" content="citation_journal_title=Biometrics.; citation_title=Reinforcement learning strategies for clinical trials in nonsmall cell lung cancer; citation_author=Y Zhao, D Zeng, MA Socinski, MR Kosorok; citation_volume=67; citation_issue=4; citation_publication_date=2011; citation_pages=1422-1433; citation_doi=10.1111/j.1541-0420.2011.01572.x; citation_id=CR12"/> <meta name="citation_reference" content="trials.ai 2019 [cited 2021 February 2]. Available from: trials.ai ."/> <meta name="citation_reference" content="citation_journal_title=Biostatistics.; citation_title=Estimation of clinical trial success rates and related parameters; citation_author=CH Wong, KW Siah, AW Lo; citation_volume=20; citation_issue=2; citation_publication_date=2019; citation_pages=273-286; citation_doi=10.1093/biostatistics/kxx069; citation_id=CR14"/> <meta name="citation_reference" content="citation_journal_title=Nature.; citation_title=Personalized medicine: time for one-person trials; citation_author=NJ Schork; citation_volume=520; citation_issue=7549; citation_publication_date=2015; citation_pages=609-611; citation_doi=10.1038/520609a; citation_id=CR15"/> <meta name="citation_reference" content="citation_journal_title=Pac Symp Biocomput.; citation_title=Automated disease cohort selection using word embeddings from electronic health records; citation_author=BS Glicksberg, R Miotto, KW Johnson, K Shameer, L Li, R Chen, JT Dudley; citation_volume=23; citation_publication_date=2018; citation_pages=145-156; citation_id=CR16"/> <meta name="citation_reference" content="citation_journal_title=BMJ.; citation_title=Development of phenotype algorithms using electronic medical records and incorporating natural language processing; citation_author=KP Liao, T Cai, GK Savova, SN Murphy, EW Karlson, AN Ananthakrishnan, VS Gainer, SY Shaw, Z Xia, P Szolovits, S Churchill, I Kohane; citation_volume=350; citation_issue=apr24 11; citation_publication_date=2015; citation_pages=h1885; citation_doi=10.1136/bmj.h1885; citation_id=CR17"/> <meta name="citation_reference" content="citation_journal_title=Sci Transl Med; citation_title=Identification of type 2 diabetes subgroups through topological analysis of patient similarity; citation_author=L Li, WY Cheng, BS Glicksberg, O Gottesman, R Tamler, R Chen; citation_volume=7; citation_issue=311; citation_publication_date=2015; citation_pages=311ra174; citation_doi=10.1126/scitranslmed.aaa9364; citation_id=CR18"/> <meta name="citation_reference" content="Our Solution 2021 [cited 2021 February 2]. Available from: https://www.bullfrogai.com/our-solution/ ."/> <meta name="citation_reference" content="citation_title=DeepEnroll: patient-trial matching with deep embedding and entailment prediction; citation_inbook_title=Proceedings of the Web Conference 2020; citation_publication_date=2020; citation_pages=1029-1037; citation_id=CR20; citation_author=X Zhang; citation_author=C Xiao; citation_author=LM Glass; citation_author=J Sun; citation_publisher=Association for Computing Machinery"/> <meta name="citation_reference" content="citation_journal_title=Ther Innov Regul Sci.; citation_title=Improving clinical trial participant prescreening with artificial intelligence (AI): a comparison of the results of AI-assisted vs standard methods in 3 oncology trials; citation_author=D Calaprice-Whitty, K Galil, W Salloum, A Zariv, B Jimenez; citation_volume=54; citation_issue=1; citation_publication_date=2020; citation_pages=69-74; citation_doi=10.1007/s43441-019-00030-4; citation_id=CR21"/> <meta name="citation_reference" content="How it works 2019 [cited 2021 February 2]. Available from: https://deep6.ai/how-it-works/ ."/> <meta name="citation_reference" content="citation_journal_title=J Biomed Inform.; citation_title=Yield and bias in defining a cohort study baseline from electronic health record data; citation_author=JL Vassy, YL Ho, J Honerlaw, K Cho, JM Gaziano, PWF Wilson, DR Gagnon; citation_volume=78; citation_publication_date=2018; citation_pages=54-59; citation_doi=10.1016/j.jbi.2017.12.017; citation_id=CR23"/> <meta name="citation_reference" content="citation_journal_title=J Am Med Inform Assoc.; citation_title=Biases introduced by filtering electronic health records for patients with “complete data”; citation_author=GM Weber, WG Adams, EV Bernstam, JP Bickel, KP Fox, K Marsolo, VA Raghavan, A Turchin, X Zhou, SN Murphy, KD Mandl; citation_volume=24; citation_issue=6; citation_publication_date=2017; citation_pages=1134-1141; citation_doi=10.1093/jamia/ocx071; citation_id=CR24"/> <meta name="citation_reference" content="citation_journal_title=JMIR Mhealth Uhealth.; citation_title=Use of a novel artificial intelligence platform on mobile devices to assess dosing compliance in a phase 2 clinical trial in subjects with schizophrenia; citation_author=EE Bain, L Shafner, DP Walling, AA Othman, C Chuang-Stein, J Hinkle, A Hanina; citation_volume=5; citation_issue=2; citation_publication_date=2017; citation_doi=10.2196/mhealth.7030; citation_id=CR25"/> <meta name="citation_reference" content="citation_journal_title=Stroke.; citation_title=Using artificial intelligence to reduce the risk of nonadherence in patients on anticoagulation therapy; citation_author=DL Labovitz, L Shafner, M Reyes Gil, D Virmani, A Hanina; citation_volume=48; citation_issue=5; citation_publication_date=2017; citation_pages=1416-1419; citation_doi=10.1161/STROKEAHA.116.016281; citation_id=CR26"/> <meta name="citation_reference" content="citation_journal_title=JAMA Dermatol.; citation_title=Machine learning and health care disparities in dermatology; citation_author=AS Adamson, A Smith; citation_volume=154; citation_issue=11; citation_publication_date=2018; citation_pages=1247-1248; citation_doi=10.1001/jamadermatol.2018.2348; citation_id=CR27"/> <meta name="citation_reference" content="Burlingame EA, Margolin AA, Gray JW, Chang YH. SHIFT: speedy histopathological-to-immunofluorescent translation of whole slide images using conditional generative adversarial networks. Proc SPIE Int Soc Opt Eng. 2018;10581.  https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6166432/ ."/> <meta name="citation_reference" content="citation_journal_title=JMIR Med Inform.; citation_title=Improving the efficacy of the data entry process for clinical research with a natural language processing-driven medical information extraction system: quantitative field research; citation_author=J Han, K Chen, L Fang, S Zhang, F Wang, H Ma, L Zhao, S Liu; citation_volume=7; citation_issue=3; citation_publication_date=2019; citation_doi=10.2196/13331; citation_id=CR29"/> <meta name="citation_reference" content="citation_journal_title=BMJ Open.; citation_title=Using natural language processing to extract structured epilepsy data from unstructured clinic letters: development and validation of the ExECT (extraction of epilepsy clinical text) system; citation_author=B Fonferko-Shadrach, AS Lacey, A Roberts, A Akbari, S Thompson, DV Ford, RA Lyons, MI Rees, WO Pickrell; citation_volume=9; citation_issue=4; citation_publication_date=2019; citation_doi=10.1136/bmjopen-2018-023232; citation_id=CR30"/> <meta name="citation_reference" content="citation_journal_title=Pharmacoepidemiol Drug Saf.; citation_title=Comparison of text processing methods in social media-based signal detection; citation_author=N Gavrielov-Yusim, ML Kurzinger, C Nishikawa, C Pan, J Pouget, LB Epstein; citation_volume=28; citation_issue=10; citation_publication_date=2019; citation_pages=1309-1317; citation_doi=10.1002/pds.4857; citation_id=CR31"/> <meta name="citation_reference" content="citation_journal_title=Neuropsychopharmacology.; citation_title=Relapse prediction in schizophrenia through digital phenotyping: a pilot study; citation_author=I Barnett, J Torous, P Staples, L Sandoval, M Keshavan, JP Onnela; citation_volume=43; citation_issue=8; citation_publication_date=2018; citation_pages=1660-1666; citation_doi=10.1038/s41386-018-0030-z; citation_id=CR32"/> <meta name="citation_reference" content="citation_journal_title=J Am Geriatr Soc.; citation_title=Real-world accuracy and use of a wearable fall detection device by older adults; citation_author=S Chaudhuri, D Oudejans, HJ Thompson, G Demiris; citation_volume=63; citation_issue=11; citation_publication_date=2015; citation_pages=2415-2416; citation_doi=10.1111/jgs.13804; citation_id=CR33"/> <meta name="citation_reference" content="citation_title=Developing measures of cognitive impairment in the real world from consumer-grade multimodal sensor streams; citation_inbook_title=Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining; citation_publication_date=2019; citation_pages=2145-2155; citation_id=CR34; citation_author=R Chen; citation_author=F Jankovic; citation_author=N Marinsek; citation_author=L Foschini; citation_author=L Kourtis; citation_author=A Signorini; citation_publisher=Association for Computing Machinery"/> <meta name="citation_reference" content="Yurtman A, Barshan B, Fidan B. Activity recognition invariant to wearable sensor unit orientation using differential rotational transformations represented by quaternions. Sensors (Basel). 2018;18(8):2725. https://pubmed.ncbi.nlm.nih.gov/30126235/ ."/> <meta name="citation_reference" content="Lu K, Yang L, Seoane F, Abtahi F, Forsman M, Lindecrantz K. Fusion of heart rate, respiration and motion measurements from a wearable sensor system to enhance energy expenditure estimation. Sensors (Basel). 2018;18(9):3092. https://pubmed.ncbi.nlm.nih.gov/30223429/ ."/> <meta name="citation_reference" content="Cheung YK, Hsueh PS, Ensari I, Willey JZ, Diaz KM. Quantile coarsening analysis of high-volume wearable activity data in a longitudinal observational study. Sensors (Basel). 2018;18(9):3056. https://pubmed.ncbi.nlm.nih.gov/30213093/ ."/> <meta name="citation_reference" content="citation_journal_title=Nat Med.; citation_title=Cardiologist-level arrhythmia detection and classification in ambulatory electrocardiograms using a deep neural network; citation_author=AY Hannun, P Rajpurkar, M Haghpanahi, GH Tison, C Bourn, MP Turakhia, AY Ng; citation_volume=25; citation_issue=1; citation_publication_date=2019; citation_pages=65-69; citation_doi=10.1038/s41591-018-0268-3; citation_id=CR38"/> <meta name="citation_reference" content="citation_journal_title=Digit Biomark.; citation_title=Depression screening from voice samples of patients affected by Parkinson's disease; citation_author=Y Ozkanca, MG Ozturk, MN Ekmekci, DC Atkins, C Demiroglu, RH Ghomi; citation_volume=3; citation_issue=2; citation_publication_date=2019; citation_pages=72-82; citation_doi=10.1159/000500354; citation_id=CR39"/> <meta name="citation_reference" content="citation_journal_title=IEEE J Biomed Health Inform.; citation_title=Detection of nocturnal scratching movements in patients with atopic dermatitis using accelerometers and recurrent neural networks; citation_author=A Moreau, P Anderer, M Ross, A Cerny, TH Almazan, B Peterson, A Moreau, P Anderer, M Ross, A Cerny, TH Almazan, B Peterson; citation_volume=22; citation_issue=4; citation_publication_date=2018; citation_pages=1011-1018; citation_doi=10.1109/JBHI.2017.2710798; citation_id=CR40"/> <meta name="citation_reference" content="citation_journal_title=Nat Med.; citation_title=Deep learning models for electrocardiograms are susceptible to adversarial attack; citation_author=X Han, Y Hu, L Foschini, L Chinitz, L Jankelson, R Ranganath; citation_volume=26; citation_issue=3; citation_publication_date=2020; citation_pages=360-363; citation_doi=10.1038/s41591-020-0791-x; citation_id=CR41"/> <meta name="citation_reference" content="citation_journal_title=JMIR Mhealth Uhealth.; citation_title=Formative evaluation of participant experience with mobile econsent in the app-mediated Parkinson mPower study: a mixed methods study; citation_author=M Doerr, A Maguire Truong, BM Bot, J Wilbanks, C Suver, LM Mangravite; citation_volume=5; citation_issue=2; citation_publication_date=2017; citation_doi=10.2196/mhealth.6521; citation_id=CR42"/> <meta name="citation_reference" content="citation_journal_title=Cancer Res.; citation_title=Use of natural language processing to extract clinical cancer phenotypes from electronic medical records; citation_author=GK Savova, I Danciu, F Alamudun, T Miller, C Lin, DS Bitterman, G Tourassi, JL Warner; citation_volume=79; citation_issue=21; citation_publication_date=2019; citation_pages=5463-5470; citation_doi=10.1158/0008-5472.CAN-19-0579; citation_id=CR43"/> <meta name="citation_reference" content="citation_journal_title=JCO Clin Cancer Inform.; citation_title=Enhancing case capture, quality, and completeness of primary melanoma pathology records via natural language processing; citation_author=JC Malke, S Jin, SP Camp, B Lari, T Kell, JM Simon, VG Prieto, JE Gershenwald, LE Haydu; citation_volume=3; citation_publication_date=2019; citation_pages=1-11; citation_doi=10.1200/CCI.19.00006; citation_id=CR44"/> <meta name="citation_reference" content="citation_journal_title=Evid Based Ment Health.; citation_title=Natural language processing for structuring clinical text data on depression using UK-CRIS; citation_author=N Vaci, Q Liu, A Kormilitzin, F Crescenzo, A Kurtulmus, J Harvey; citation_volume=23; citation_issue=1; citation_publication_date=2020; citation_pages=21-26; citation_doi=10.1136/ebmental-2019-300134; citation_id=CR45"/> <meta name="citation_reference" content="citation_journal_title=Comput Methods Programs Biomed.; citation_title=An automated data verification approach for improving data quality in a clinical registry; citation_author=Q Tian, M Liu, L Min, J An, X Lu, H Duan; citation_volume=181; citation_publication_date=2019; citation_pages=104840; citation_doi=10.1016/j.cmpb.2019.01.012; citation_id=CR46"/> <meta name="citation_reference" content="citation_journal_title=Comput Methods Programs Biomed.; citation_title=Semi-supervised encoding for outlier detection in clinical observation data; citation_author=H Estiri, SN Murphy; citation_volume=181; citation_publication_date=2019; citation_pages=104830; citation_doi=10.1016/j.cmpb.2019.01.002; citation_id=CR47"/> <meta name="citation_reference" content="citation_title=AI in clinical development: improving safety and accelerating results. [White paper]; citation_publication_date=2019; citation_id=CR48; citation_author=LMSG Glass; citation_author=R Patil"/> <meta name="citation_reference" content="citation_journal_title=Circulation.; citation_title=2017 Cardiovascular and stroke endpoint definitions for clinical trials; citation_author=KA Hicks, KW Mahaffey, R Mehran, SE Nissen, SD Wiviott, B Dunn, SD Solomon, JR Marler, JR Teerlink, A Farb, DA Morrow, SL Targum, CA Sila, MTT Hai, MR Jaff, HV Joffe, DE Cutlip, AS Desai, EF Lewis, CM Gibson, MJ Landray, AM Lincoff, CJ White, SS Brooks, K Rosenfield, MJ Domanski, AJ Lansky, J McMurray, JE Tcheng, SR Steinhubl, P Burton, L Mauri, CM O'Connor, MA Pfeffer, HMJ Hung, NL Stockbridge, BR Chaitman, RJ Temple; citation_volume=137; citation_issue=9; citation_publication_date=2018; citation_pages=961-972; citation_doi=10.1161/CIRCULATIONAHA.117.033502; citation_id=CR49"/> <meta name="citation_reference" content="Liu Y, Gopalakrishnan V. An overview and evaluation of recent machine learning imputation methods using cardiac imaging data. Data (Basel). 2017;2(1):8. https://pubmed.ncbi.nlm.nih.gov/28243594/ ."/> <meta name="citation_reference" content="citation_journal_title=Conf Proc IEEE Eng Med Biol Soc.; citation_title=A deep learning technique for imputing missing healthcare data; citation_author=S Phung, A Kumar, J Kim; citation_volume=2019; citation_publication_date=2019; citation_pages=6513-6516; citation_doi=10.1109/EMBC.2019.8856760; citation_id=CR51"/> <meta name="citation_reference" content="citation_title=A deep learning framework for imputing missing values in genomic data; citation_publication_date=2018; citation_id=CR52; citation_author=YL Qiu; citation_author=H Zheng; citation_author=OJ Gevaert"/> <meta name="citation_reference" content="citation_title=Imputing missing data in large-scale multivariate biomedical wearable recordings using bidirectional recurrent neural networks with temporal activation regularization; citation_inbook_title=2019 41st Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC); citation_publication_date=2019; citation_id=CR53; citation_author=T Feng; citation_author=S Narayanan"/> <meta name="citation_reference" content="citation_journal_title=J Am Med Inform Assoc.; citation_title=3D-MICE: integration of cross-sectional and longitudinal imputation for multi-analyte longitudinal clinical data; citation_author=Y Luo, P Szolovits, AS Dighe, JM Baron; citation_volume=25; citation_issue=6; citation_publication_date=2018; citation_pages=645-653; citation_doi=10.1093/jamia/ocx133; citation_id=CR54"/> <meta name="citation_reference" content="citation_journal_title=AMIA Annu Symp Proc.; citation_title=Identification of Clinically meaningful plasma transfusion subgroups using unsupervised random forest clustering; citation_author=C Ngufor, MA Warner, DH Murphree, H Liu, R Carter, CB Storlie; citation_volume=2017; citation_publication_date=2017; citation_pages=1332-1341; citation_id=CR55"/> <meta name="citation_reference" content="citation_journal_title=J Immunol.; citation_title=SIMON, an automated machine learning system, reveals immune signatures of influenza vaccine responses; citation_author=A Tomic, I Tomic, Y Rosenberg-Hasson, CL Dekker, HT Maecker, MM Davis; citation_volume=203; citation_issue=3; citation_publication_date=2019; citation_pages=749-759; citation_doi=10.4049/jimmunol.1900033; citation_id=CR56"/> <meta name="citation_reference" content="citation_journal_title=Trials.; citation_title=Machine learning analysis plans for randomised controlled trials: detecting treatment effect heterogeneity with strict control of type I error; citation_author=JA Watson, CC Holmes; citation_volume=21; citation_issue=1; citation_publication_date=2020; citation_pages=156; citation_doi=10.1186/s13063-020-4076-y; citation_id=CR57"/> <meta name="citation_reference" content="citation_journal_title=Trials.; citation_title=Preventing false discovery of heterogeneous treatment effect subgroups in randomized trials; citation_author=J Rigdon, M Baiocchi, S Basu; citation_volume=19; citation_issue=1; citation_publication_date=2018; citation_pages=382; citation_doi=10.1186/s13063-018-2774-5; citation_id=CR58"/> <meta name="citation_reference" content="citation_journal_title=Circ Arrhythm Electrophysiol.; citation_title=Machine learning algorithm predicts cardiac resynchronization therapy outcomes: lessons from the companion trial; citation_author=MM Kalscheur, RT Kipp, MC Tattersall, C Mei, KA Buhr, DL DeMets, ME Field, LL Eckhardt, CD Page; citation_volume=11; citation_issue=1; citation_publication_date=2018; citation_doi=10.1161/CIRCEP.117.005499; citation_id=CR59"/> <meta name="citation_reference" content="citation_journal_title=J Eval Clin Pract.; citation_title=Combining machine learning and propensity score weighting to estimate causal effects in multivalued treatments; citation_author=A Linden, PR Yarnold; citation_volume=22; citation_issue=6; citation_publication_date=2016; citation_pages=871-881; citation_doi=10.1111/jep.12610; citation_id=CR60"/> <meta name="citation_reference" content="citation_journal_title=Am J Epidemiol.; citation_title=Targeted maximum likelihood estimation for causal inference in observational studies; citation_author=MS Schuler, S Rose; citation_volume=185; citation_issue=1; citation_publication_date=2017; citation_pages=65-73; citation_doi=10.1093/aje/kww165; citation_id=CR61"/> <meta name="citation_reference" content="citation_journal_title=Stat Med.; citation_title=Comparing methods for estimation of heterogeneous treatment effects using observational data from health care databases; citation_author=T Wendling, K Jung, A Callahan, A Schuler, NH Shah, B Gallego; citation_volume=37; citation_issue=23; citation_publication_date=2018; citation_pages=3309-3324; citation_doi=10.1002/sim.7820; citation_id=CR62"/> <meta name="citation_reference" content="citation_journal_title=Stat Med.; citation_title=Using longitudinal targeted maximum likelihood estimation in complex settings with dynamic interventions; citation_author=M Schomaker, MA Luque-Fernandez, V Leroy, MA Davies; citation_volume=38; citation_issue=24; citation_publication_date=2019; citation_pages=4888-4911; citation_doi=10.1002/sim.8340; citation_id=CR63"/> <meta name="citation_reference" content="citation_journal_title=Am J Epidemiol.; citation_title=Improving propensity score estimators’ robustness to model misspecification using super learner; citation_author=R Pirracchio, ML Petersen, M Laan; citation_volume=181; citation_issue=2; citation_publication_date=2015; citation_pages=108-119; citation_doi=10.1093/aje/kwu253; citation_id=CR64"/> <meta name="citation_reference" content="citation_journal_title=Nat Med.; citation_title=Guidelines for reinforcement learning in healthcare; citation_author=O Gottesman, F Johansson, M Komorowski, A Faisal, D Sontag, F Doshi-Velez, LA Celi; citation_volume=25; citation_issue=1; citation_publication_date=2019; citation_pages=16-18; citation_doi=10.1038/s41591-018-0310-5; citation_id=CR65"/> <meta name="citation_reference" content="citation_journal_title=PLoS One.; citation_title=Personalized survival predictions via trees of predictors: an application to cardiac transplantation; citation_author=J Yoon, WR Zame, A Banerjee, M Cadeiras, AM Alaa, M Schaar; citation_volume=13; citation_issue=3; citation_publication_date=2018; citation_doi=10.1371/journal.pone.0194985; citation_id=CR66"/> <meta name="citation_reference" content="citation_journal_title=Nat Med.; citation_title=The artificial intelligence clinician learns optimal treatment strategies for sepsis in intensive care; citation_author=M Komorowski, LA Celi, O Badawi, AC Gordon, AA Faisal; citation_volume=24; citation_issue=11; citation_publication_date=2018; citation_pages=1716-1720; citation_doi=10.1038/s41591-018-0213-5; citation_id=CR67"/> <meta name="citation_reference" content="citation_journal_title=Lancet Digit Health.; citation_title=Practical guidance on artificial intelligence for health-care data; citation_author=M Ghassemi, T Naumann, P Schulam, AL Beam, IY Chen, R Ranganath; citation_volume=1; citation_issue=4; citation_publication_date=2019; citation_pages=e157-e159; citation_doi=10.1016/S2589-7500(19)30084-6; citation_id=CR68"/> <meta name="citation_reference" content="citation_journal_title=Nat Med.; citation_title=Do no harm: a roadmap for responsible machine learning for health care; citation_author=J Wiens, S Saria, M Sendak, M Ghassemi, VX Liu, F Doshi-Velez, K Jung, K Heller, D Kale, M Saeed, PN Ossorio, S Thadaney-Israni, A Goldenberg; citation_volume=25; citation_issue=9; citation_publication_date=2019; citation_pages=1337-1340; citation_doi=10.1038/s41591-019-0548-6; citation_id=CR69"/> <meta name="citation_reference" content="Nestor B, McDermott M, Chauhan G, et al. Rethinking clinical prediction: why machine learning must consider year of care and feature aggregation. arXiv preprint 2018;arXiv:181112583."/> <meta name="citation_reference" content="citation_journal_title=Sci Data.; citation_title=MIMIC-III, a freely accessible critical care database; citation_author=AE Johnson, TJ Pollard, L Shen, LW Lehman, M Feng, M Ghassemi; citation_volume=3; citation_issue=1; citation_publication_date=2016; citation_pages=160035; citation_doi=10.1038/sdata.2016.35; citation_id=CR71"/> <meta name="citation_reference" content="citation_journal_title=Sci Data.; citation_title=The eICU Collaborative Research Database, a freely available multi-center database for critical care research; citation_author=TJ Pollard, AEW Johnson, JD Raffa, LA Celi, RG Mark, O Badawi; citation_volume=5; citation_issue=1; citation_publication_date=2018; citation_pages=180178; citation_doi=10.1038/sdata.2018.178; citation_id=CR72"/> <meta name="citation_reference" content="UK Biobank. www.ukbiobank.ac.uk . Accessed 22 Mar 2021."/> <meta name="citation_reference" content="citation_title=Predicting clinical outcomes across changing electronic health record systems; citation_inbook_title=Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining; citation_publication_date=2017; citation_pages=1497-1505; citation_id=CR74; citation_author=JJ Gong; citation_author=T Naumann; citation_author=P Szolovits; citation_author=JV Guttag; citation_publisher=Association for Computing Machinery"/> <meta name="citation_reference" content="Beam AL, Manrai AK, Ghassemi M. Challenges to the reproducibility of machine learning models in health care. JAMA. 2020;323(4):305–6. https://doi.org/10.1001/jama.2019.20866 ."/> <meta name="citation_reference" content="citation_title=Sanity checks for saliency maps; citation_inbook_title=Proceedings of the 32nd International Conference on Neural Information Processing Systems; citation_publication_date=2018; citation_pages=9525-9536; citation_id=CR76; citation_author=J Adebayo; citation_author=J Gilmer; citation_author=M Muelly; citation_author=I Goodfellow; citation_author=M Hardt; citation_author=B Kim; citation_publisher=Curran Associates Inc."/> <meta name="citation_reference" content="citation_title=Attention is not not explanation. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP); citation_publication_date=2019; citation_id=CR77; citation_author=S Wiegreffe; citation_author=Y Pinter; citation_publisher=Association for Computational Linguistics"/> <meta name="citation_reference" content="citation_title=Attention is not explanation; citation_publication_date=2019; citation_id=CR78; citation_author=S Jain; citation_author=BC Wallace; citation_publisher=NAACL-HLT"/> <meta name="citation_reference" content="Serrano S, Smith NA. Is attention interpretable? In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 2931–2951, Stroudsburg, PA, USA, 2019. Association for Computational Linguistics."/> <meta name="citation_reference" content="citation_title=“The human body is a black box”: supporting clinical decision-making with deep learning; citation_inbook_title=Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency; citation_publication_date=2020; citation_pages=99-109; citation_id=CR80; citation_author=M Sendak; citation_author=MC Elish; citation_author=M Gao; citation_author=J Futoma; citation_author=W Ratliff; citation_author=M Nichols; citation_publisher=Association for Computing Machinery"/> <meta name="citation_reference" content="citation_title=Machine bias. ProPublica; citation_publication_date=2016; citation_id=CR81; citation_author=JLJ Angwin; citation_author=S Mattu; citation_author=L Kirchner"/> <meta name="citation_reference" content="citation_journal_title=EGEMS (Wash DC); citation_title=Evaluating foundational data quality in the National Patient-Centered Clinical Research Network (PCORnet(R)); citation_author=LG Qualls, TA Phillips, BG Hammill, J Topping, DM Louzao, JS Brown; citation_volume=6; citation_issue=1; citation_publication_date=2018; citation_pages=3; citation_id=CR82"/> <meta name="citation_reference" content="citation_journal_title=Stud Health Technol Inform.; citation_title=Combining archetypes with fast health interoperability resources in future-proof health information systems; citation_author=D Bosca, D Moner, JA Maldonado, M Robles; citation_volume=210; citation_publication_date=2015; citation_pages=180-184; citation_id=CR83"/> <meta name="citation_reference" content="citation_journal_title=J Am Med Inform Assoc.; citation_title=Data interchange using i2b2; citation_author=JG Klann, A Abend, VA Raghavan, KD Mandl, SN Murphy; citation_volume=23; citation_issue=5; citation_publication_date=2016; citation_pages=909-915; citation_doi=10.1093/jamia/ocv188; citation_id=CR84"/> <meta name="citation_reference" content="citation_journal_title=J Am Med Inform Assoc.; citation_title=Validation of a common data model for active safety surveillance research; citation_author=JM Overhage, PB Ryan, CG Reich, AG Hartzema, PE Stang; citation_volume=19; citation_issue=1; citation_publication_date=2012; citation_pages=54-60; citation_doi=10.1136/amiajnl-2011-000376; citation_id=CR85"/> <meta name="citation_reference" content="21st Century Cures Act: Interoperability, information blocking, and the ONC Health IT Certification Program [updated 1 May 2020]. Available from: https://www.federalregister.gov/documents/2020/05/01/2020-07419/21st-century-cures-act-interoperability-information-blocking-and-the-onc-health-it-certification . Accessed 16 May 2020."/> <meta name="citation_reference" content="Oh M, Park S, Kim S, Chae H. Machine learning-based analysis of multi-omics data on the cloud for investigating gene regulations. Brief Bioinform. 2020. Epub 2020/04/01. https://doi.org/10.1093/bib/bbaa032 ."/> <meta name="citation_reference" content="citation_journal_title=Phys Med.; citation_title=Using federated data sources and Varian Learning Portal framework to train a neural network model for automatic organ segmentation; citation_author=E Czeizler, W Wiessler, T Koester, M Hakala, S Basiri, P Jordan, E Kuusela; citation_volume=72; citation_publication_date=2020; citation_pages=39-45; citation_doi=10.1016/j.ejmp.2020.03.011; citation_id=CR88"/> <meta name="citation_reference" content="citation_journal_title=JCO Clin Cancer Inform.; citation_title=Systematic review of privacy-preserving distributed machine learning from federated databases in health care; citation_author=F Zerka, S Barakat, S Walsh, M Bogowicz, RTH Leijenaar, A Jochems, B Miraglio, D Townend, P Lambin; citation_volume=4; citation_publication_date=2020; citation_pages=184-200; citation_doi=10.1200/CCI.19.00047; citation_id=CR89"/> <meta name="citation_reference" content="citation_journal_title=BMC Med Genomics.; citation_title=The eMERGE Network: a consortium of biorepositories linked to electronic medical records data for conducting genomic studies; citation_author=CA McCarty, RL Chisholm, CG Chute, IJ Kullo, GP Jarvik, EB Larson; citation_volume=4; citation_issue=1; citation_publication_date=2011; citation_pages=13; citation_doi=10.1186/1755-8794-4-13; citation_id=CR90"/> <meta name="citation_reference" content="citation_journal_title=Drug Saf.; citation_title=Bridging islands of information to establish an integrated knowledge base of drugs and health outcomes of interest; citation_author=RD Boyce, PB Ryan, GN Noren, MJ Schuemie, C Reich, J Duke; citation_volume=37; citation_issue=8; citation_publication_date=2014; citation_pages=557-567; citation_doi=10.1007/s40264-014-0189-0; citation_id=CR91"/> <meta name="citation_reference" content="citation_journal_title=J Clin Epidemiol.; citation_title=The proposed ‘concordance-statistic for benefit’ provided a useful metric when modeling heterogeneous treatment effects; citation_author=D Klaveren, EW Steyerberg, PW Serruys, DM Kent; citation_volume=94; citation_publication_date=2018; citation_pages=59-68; citation_doi=10.1016/j.jclinepi.2017.10.021; citation_id=CR92"/> <meta name="citation_reference" content="citation_title=An invisible hand: patients aren’t being told about the AI systems advising their care. STAT; citation_publication_date=2020; citation_id=CR93; citation_author=RBE Robbins"/> <meta name="citation_reference" content="citation_journal_title=Med Health Care Philos.; citation_title=“You hoped we would sleep walk into accepting the collection of our data”: controversies surrounding the UK care.data scheme and their wider relevance for biomedical research; citation_author=S Sterckx, V Rakic, J Cockbain, P Borry; citation_volume=19; citation_issue=2; citation_publication_date=2016; citation_pages=177-190; citation_doi=10.1007/s11019-015-9661-6; citation_id=CR94"/> <meta name="citation_reference" content="citation_title=Confronting racial and ethnic disparities in health care; citation_publication_date=2003; citation_id=CR95; citation_publisher=National Academies Press"/> <meta name="citation_reference" content="citation_title=Invisible women; citation_publication_date=2019; citation_id=CR96; citation_author=PC Criado; citation_publisher=Harry N. Abrams"/> <meta name="citation_reference" content="citation_title=Hurtful words: quantifying biases in clinical contextual word embeddings; citation_inbook_title=Proceedings of the ACM Conference on Health, Inference, and Learning; citation_publication_date=2020; citation_pages=110-120; citation_id=CR97; citation_author=H Zhang; citation_author=AX Lu; citation_author=M Abdalla; citation_author=M McDermott; citation_author=M Ghassemi; citation_publisher=Association for Computing Machinery"/> <meta name="citation_reference" content="citation_journal_title=Nat Med.; citation_title=Treating health disparities with artificial intelligence; citation_author=IY Chen, S Joshi, M Ghassemi; citation_volume=26; citation_issue=1; citation_publication_date=2020; citation_pages=16-17; citation_doi=10.1038/s41591-019-0649-2; citation_id=CR98"/> <meta name="citation_reference" content="citation_title=Man is to computer programmer as woman is to homemaker? debiasing word embeddings; citation_inbook_title=Proceedings of the 30th International Conference on Neural Information Processing Systems; citation_publication_date=2016; citation_pages=4356-4364; citation_id=CR99; citation_author=T Bolukbasi; citation_author=K-W Chang; citation_author=J Zou; citation_author=V Saligrama; citation_author=A Kalai; citation_publisher=Curran Associates Inc."/> <meta name="citation_reference" content="Kusner, Matt, Loftus, Joshua, Russell, Chris and Silva, Ricardo. Counterfactual fairness Conference. Proceedings of the 31st International Conference on Neural Information Processing Systems Conference. Long Beach, California, USA Publisher: Curran Associates Inc; 2017:4069–4079."/> <meta name="citation_reference" content="citation_title=Equality of opportunity in supervised learning; citation_inbook_title=Proceedings of the 30th International Conference on Neural Information Processing Systems; citation_publication_date=2016; citation_pages=3323-3331; citation_id=CR101; citation_author=M Hardt; citation_author=E Price; citation_author=N Srebro; citation_publisher=Curran Associates Inc."/> <meta name="citation_reference" content="citation_title=Fairness without harm: decoupled classifiers with preference guarantees; citation_inbook_title=Proceedings of the 36th International Conference on Machine Learning; Proceedings of Machine Learning Research: PMLR %J Proceedings of Machine Learning Research; citation_publication_date=2019; citation_pages=6373-6382; citation_id=CR102; citation_author=B Ustun; citation_author=Y Liu; citation_author=D Parkes"/> <meta name="citation_reference" content="citation_journal_title=Circ Arrhythm Electrophysiol.; citation_title=Assessing and Mitigating bias in medical artificial intelligence: the effects of race and ethnicity on a deep learning model for ECG analysis; citation_author=PA Noseworthy, ZI Attia, LC Brewer, SN Hayes, X Yao, S Kapa, PA Friedman, F Lopez-Jimenez; citation_volume=13; citation_issue=3; citation_publication_date=2020; citation_doi=10.1161/CIRCEP.119.007988; citation_id=CR103"/> <meta name="citation_author" content="Weissler, E. Hope"/> <meta name="citation_author_institution" content="Duke Clinical Research Institute, Duke University School of Medicine, Durham, USA"/> <meta name="citation_author" content="Naumann, Tristan"/> <meta name="citation_author_institution" content="Microsoft Research, Cambridge, USA"/> <meta name="citation_author" content="Andersson, Tomas"/> <meta name="citation_author_institution" content="AstraZeneca, Gothenburg, Sweden"/> <meta name="citation_author" content="Ranganath, Rajesh"/> <meta name="citation_author_institution" content="Courant Institute of Mathematical Science, New York University, New York, USA"/> <meta name="citation_author" content="Elemento, Olivier"/> <meta name="citation_author_institution" content="Englander Institute for Precision Medicine, Weill Cornell Medical College, New York, USA"/> <meta name="citation_author" content="Luo, Yuan"/> <meta name="citation_author_institution" content="Northwestern University Clinical and Translational Sciences Institute, Northwestern University, Chicago, USA"/> <meta name="citation_author" content="Freitag, Daniel F."/> <meta name="citation_author_institution" content="Division Pharmaceuticals, Open Innovation and Digital Technologies, Bayer AG, Wuppertal, Germany"/> <meta name="citation_author" content="Benoit, James"/> <meta name="citation_author_institution" content="University of Alberta, Edmonton, Canada"/> <meta name="citation_author" content="Hughes, Michael C."/> <meta name="citation_author_institution" content="Department of Computer Science, Tufts University, Medford, USA"/> <meta name="citation_author" content="Khan, Faisal"/> <meta name="citation_author_institution" content="AstraZeneca, Gothenburg, Sweden"/> <meta name="citation_author" content="Slater, Paul"/> <meta name="citation_author_institution" content="Billion Minds, Inc., Seattle, USA"/> <meta name="citation_author" content="Shameer, Khader"/> <meta name="citation_author_institution" content="AstraZeneca, Gothenburg, Sweden"/> <meta name="citation_author" content="Roe, Matthew"/> <meta name="citation_author_institution" content="Verana Health, San Francisco, USA"/> <meta name="citation_author" content="Hutchison, Emmette"/> <meta name="citation_author_institution" content="AstraZeneca, Gothenburg, Sweden"/> <meta name="citation_author" content="Kollins, Scott H."/> <meta name="citation_author_institution" content="Duke Clinical Research Institute, Duke University School of Medicine, Durham, USA"/> <meta name="citation_author" content="Broedl, Uli"/> <meta name="citation_author_institution" content="Boehringer-Ingelheim, Burlington, Canada"/> <meta name="citation_author" content="Meng, Zhaoling"/> <meta name="citation_author_institution" content="Sanofi, Cambridge, USA"/> <meta name="citation_author" content="Wong, Jennifer L."/> <meta name="citation_author_institution" content="Sanofi, Washington, USA"/> <meta name="citation_author" content="Curtis, Lesley"/> <meta name="citation_author_institution" content="Duke Clinical Research Institute, Duke University School of Medicine, Durham, USA"/> <meta name="citation_author" content="Huang, Erich"/> <meta name="citation_author_institution" content="Duke Clinical Research Institute, Duke University School of Medicine, Durham, USA"/> <meta name="citation_author_institution" content="Duke Forge, Durham, USA"/> <meta name="citation_author" content="Ghassemi, Marzyeh"/> <meta name="citation_author_institution" content="Vector Institute, University of Toronto, Toronto, Canada"/> <meta name="citation_author_institution" content="Department of Electrical Engineering and Computer Science, Massachusetts Institute of Technology, Cambridge, USA"/> <meta name="citation_author_institution" content="Institute for Medical Engineering and Science, Massachusetts Institute of Technology, Cambridge, USA"/> <meta name="citation_author_institution" content="CIFAR AI Chair, Vector Institute, Ontario, Canada"/> <meta name="format-detection" content="telephone=no"> <link rel="apple-touch-icon" sizes="180x180" href=/static/img/favicons/bmc/apple-touch-icon-582ef1d0f5.png> <link rel="icon" type="image/png" sizes="192x192" href=/static/img/favicons/bmc/android-chrome-192x192-9625b7cdba.png> <link rel="icon" type="image/png" sizes="32x32" href=/static/img/favicons/bmc/favicon-32x32-5d7879efe1.png> <link rel="icon" type="image/png" sizes="16x16" href=/static/img/favicons/bmc/favicon-16x16-c241ac1a2f.png> <link rel="shortcut icon" data-test="shortcut-icon" href=/static/img/favicons/bmc/favicon-1886637b78.ico> <link rel="manifest" href=/static/app-bmc/manifest-3bb9ad383b.json> <meta name="msapplication-config" content=/static/app-bmc/browserconfig-ecd9aa8205.xml> <meta name="msapplication-TileColor" content="#1b3051"> <meta name="msapplication-TileImage" content=/static/img/favicons/bmc/app-icon-144x144.png> <meta name="theme-color" content="#1b3051"> <script>(function(H){H.className=H.className.replace(/\bno-js\b/,'js')})(document.documentElement)</script> <link rel="stylesheet" media="screen" href=/static/app-bmc/css/core-article-f3872e738d.css> <link rel="stylesheet" media="screen" href=/static/app-bmc/css/core-eac3097aa4.css> <link rel="stylesheet" media="print" href=/static/app-bmc/css/print-b8af42253b.css> <!-- This template is only used by BMC for now --> <style>@media only print, only all and (prefers-color-scheme: no-preference), only all and (prefers-color-scheme: light), only all and (prefers-color-scheme: dark) { button{line-height:inherit}html,label{font-family:-apple-system,BlinkMacSystemFont,Segoe UI,Roboto,Oxygen-Sans,Ubuntu,Cantarell,Helvetica Neue,sans-serif}html{-webkit-font-smoothing:subpixel-antialiased;box-sizing:border-box;color:#333;font-size:100%;height:100%;line-height:1.61803;overflow-y:scroll}*{box-sizing:inherit}body{background:#fff;margin:0;max-width:100%;min-height:100%}button,div,form,input,p{margin:0;padding:0}body{padding:0}a{color:#004b83;text-decoration:underline;text-decoration-skip-ink:auto}a>img{vertical-align:middle}h1,h2{color:#1b3051;font-style:normal;font-weight:700}h1{font-family:-apple-system,BlinkMacSystemFont,Segoe UI,Roboto,Oxygen-Sans,Ubuntu,Cantarell,Helvetica Neue,sans-serif;font-size:28px;line-height:1.4}html.webfonts-loaded h1,html.webfonts-loaded h2,html.webfonts-loaded h3{font-family:Europa,Trebuchet MS}h2{font-size:26px;margin-bottom:.5em}h2,h3{font-family:-apple-system,BlinkMacSystemFont,Segoe UI,Roboto,Oxygen-Sans,Ubuntu,Cantarell,Helvetica Neue,sans-serif;line-height:1.4}h3{color:#1b3051;font-size:18px;font-style:normal;margin-bottom:.7em}.c-navbar{background:#f2f2f2;border-bottom:1px solid #d9d9d9;border-top:1px solid #d9d9d9;font-family:-apple-system,BlinkMacSystemFont,Segoe UI,Roboto,Oxygen-Sans,Ubuntu,Cantarell,Helvetica Neue,sans-serif;font-size:14px;line-height:1.61803;padding:16px 0}.c-navbar--with-submit-button{padding-bottom:24px}@media only screen and (min-width:540px){.c-navbar--with-submit-button{padding-bottom:16px}}.c-navbar__container{display:flex;flex-wrap:wrap;justify-content:space-between;margin:0 auto;max-width:1280px;padding:0 16px}.c-navbar__content{display:flex;flex:0 1 auto}.c-navbar__nav{align-items:center;display:flex;flex-wrap:wrap;gap:16px 16px;list-style:none;margin:0;padding:0}.c-navbar__item{flex:0 0 auto}.c-navbar__link{background:0 0;border:0;color:currentcolor;display:block;text-decoration:none;text-transform:capitalize}.c-navbar__link--is-shown{text-decoration:underline}.c-ad{text-align:center}@media only screen and (min-width:320px){.c-ad{padding:8px}}.c-ad--728x90{background-color:#ccc;display:none}.c-ad--728x90 .c-ad__inner{min-height:calc(1.5em + 94px)}.c-ad--728x90 iframe{height:90px;max-width:970px}@media only screen and (min-width:768px){.js .c-ad--728x90{display:none}.js .u-show-following-ad+.c-ad--728x90{display:block}}.c-ad iframe{border:0;overflow:auto;vertical-align:top}.c-ad__label{color:#333;font-weight:400;line-height:1.5;margin-bottom:4px}.c-ad__label,.c-skip-link{font-family:-apple-system,BlinkMacSystemFont,Segoe UI,Roboto,Oxygen-Sans,Ubuntu,Cantarell,Helvetica Neue,sans-serif;font-size:.875rem}.c-skip-link{background:#dceaf6;bottom:auto;color:#004b83;padding:8px;position:absolute;text-align:center;transform:translateY(-100%);z-index:9999}@media (prefers-reduced-motion:reduce){.c-skip-link{transition:top .3s ease-in-out 0s}}@media print{.c-skip-link{display:none}}.c-skip-link:link{color:#004b83}.c-dropdown__button:after{border-color:transparent transparent transparent #fff;border-style:solid;border-width:4px 0 4px 14px;content:"";display:block;height:0;margin-left:3px;width:0}.c-dropdown{display:inline-block;position:relative}.c-dropdown__button{background-color:transparent;border:0;display:inline-block;padding:0;white-space:nowrap}.c-dropdown__button:after{border-color:currentcolor transparent transparent;border-width:5px 4px 0 5px;display:inline-block;margin-left:8px;vertical-align:middle}.c-dropdown__menu{background-color:#fff;border:1px solid #d9d9d9;border-radius:3px;box-shadow:0 2px 6px rgba(0,0,0,.1);font-family:-apple-system,BlinkMacSystemFont,Segoe UI,Roboto,Oxygen-Sans,Ubuntu,Cantarell,Helvetica Neue,sans-serif;font-size:14px;line-height:1.4;list-style:none;margin:0;padding:8px 0;position:absolute;top:100%;transform:translateY(8px);width:180px;z-index:100}.c-dropdown__menu:after,.c-dropdown__menu:before{border-style:solid;bottom:100%;content:"";display:block;height:0;left:16px;position:absolute;width:0}.c-dropdown__menu:before{border-color:transparent transparent #d9d9d9;border-width:0 9px 9px;transform:translateX(-1px)}.c-dropdown__menu:after{border-color:transparent transparent #fff;border-width:0 8px 8px}.c-dropdown__menu--right{left:auto;right:0}.c-dropdown__menu--right:after,.c-dropdown__menu--right:before{left:auto;right:16px}.c-dropdown__menu--right:before{transform:translateX(1px)}.c-dropdown__link{background-color:transparent;color:#004b83;display:block;padding:4px 16px}.c-header{background-color:#1b3051;border-bottom:4px solid #1b3051;font-family:-apple-system,BlinkMacSystemFont,Segoe UI,Roboto,Oxygen-Sans,Ubuntu,Cantarell,Helvetica Neue,sans-serif;font-size:14px;padding:16px 0}.c-header__container,.c-header__menu{align-items:center;display:flex;flex-wrap:wrap}@supports (gap:2em){.c-header__container,.c-header__menu{gap:2em 2em}}.c-header__menu{list-style:none;margin:0;padding:0}.c-header__item{color:#fff}@supports not (gap:2em){.c-header__item{margin-left:24px}}.c-header__container{justify-content:space-between;margin:0 auto;max-width:1280px;padding:0 16px}@supports not (gap:2em){.c-header__brand{margin-right:48px}}.c-header__brand a{display:block;text-decoration:none}.c-header__link{color:#fff}.c-journal-title a{text-decoration:none}html.webfonts-loaded .c-journal-title{font-family:Europa,Trebuchet MS}.c-form-field{margin-bottom:1em}.c-form-field__label{color:#666;display:block;font-size:14px;margin-bottom:.4em}.c-form-field__input{border:1px solid #a6a6a6;border-radius:3px;box-shadow:inset 0 1px 3px 0 rgba(0,0,0,.21);font-size:14px;line-height:1.28571;padding:.75em 1em;vertical-align:middle;width:100%}.c-popup-search{background-color:#25426f;box-shadow:0 3px 3px -3px rgba(0,0,0,.21);padding:16px 0;position:relative;z-index:10}@media only screen and (min-width:1024px){.js .c-popup-search{position:absolute;top:100%;width:100%}.c-popup-search__container{margin:auto;max-width:70%}}.c-logo img{display:block}.ctx-search .c-form-field{margin-bottom:0}.ctx-search .c-form-field__label{color:#fff}.ctx-search .c-form-field__input{border-bottom-right-radius:0;border-top-right-radius:0;margin-right:0}.c-journal-header{overflow:hidden}.c-journal-header__inner{padding-bottom:8px;padding-top:8px}@media only screen and (min-width:540px){.c-journal-header__inner{display:flex;justify-content:space-between}}.c-journal-header__identity{background-size:auto 80px;min-height:16px;padding:6px 0}@media only screen and (min-width:540px){.c-journal-header__identity{background-position:0}}.c-journal-header__identity--default{background:url(/static/images/bmc/identities/patterns/default-7e3a6b1388.svg) 0 0 no-repeat #04caa8}.c-journal-title{color:#1b3051;display:flex;font-family:-apple-system,BlinkMacSystemFont,Segoe UI,Roboto,Oxygen-Sans,Ubuntu,Cantarell,Helvetica Neue,sans-serif;font-size:26px;font-style:normal;font-weight:700;line-height:1.4;margin-bottom:0}.c-journal-title>a{color:#0061a9;color:#1b3051;display:flex;text-decoration:none}.c-journal-title__text{align-self:center;display:block;flex:0 1 auto}.c-logo,.c-logo>a{align-items:baseline;display:flex}.c-logo__strapline{display:none}@media only screen and (min-width:540px){.c-logo__strapline{display:block;margin:0 0 0 16px;transform:translateY(1px)}}.c-logo{display:block}html.webfonts-loaded .u-h3,html.webfonts-loaded .u-h4{font-family:Europa,Trebuchet MS}.u-button{align-items:center;background-color:#f2f2f2;background-image:linear-gradient(#fff,#f2f2f2);border:1px solid #ccc;border-radius:2px;cursor:pointer;display:inline-flex;font-family:-apple-system,BlinkMacSystemFont,Segoe UI,Roboto,Oxygen-Sans,Ubuntu,Cantarell,Helvetica Neue,sans-serif;font-size:1rem;justify-content:center;line-height:1.3;margin:0;padding:8px;position:relative;text-decoration:none;transition:all .25s ease 0s,color .25s ease 0s,border-color .25s ease 0s;width:auto}.u-button svg,.u-button--primary svg,.u-button--tertiary svg{fill:currentcolor}.u-button{color:#004b83}.u-button--primary,.u-button--tertiary{background-color:#33629d;background-image:linear-gradient(#4d76a9,#33629d);border:1px solid rgba(0,59,132,.5);color:#fff}.u-button--tertiary{font-weight:400}.u-button--full-width{display:flex;width:100%}.u-clearfix:after,.u-clearfix:before{content:"";display:table}.u-clearfix:after{clear:both}.u-color-open-access{color:#b74616}.u-container{margin:0 auto;max-width:1280px;padding:0 16px}.u-display-flex{display:flex;width:100%}.u-align-items-center{align-items:center}.u-justify-content-space-between{justify-content:space-between}.u-flex-static{flex:0 0 auto}.u-display-none{display:none}.js .u-js-hide{display:none;visibility:hidden}@media print{.u-hide-print{display:none}}.u-icon{fill:currentcolor;display:inline-block;height:1em;transform:translate(0);vertical-align:text-top;width:1em}.u-list-reset{list-style:none;margin:0;padding:0}.u-position-relative{position:relative}.u-mt-32{margin-top:32px}.u-mr-24{margin-right:24px}.u-mr-48{margin-right:48px}.u-mb-32{margin-bottom:32px}.u-ml-8{margin-left:8px}.u-button-reset{background-color:transparent;border:0;padding:0}.u-text-sm{font-size:14px}.u-h3,.u-h4{font-family:-apple-system,BlinkMacSystemFont,Segoe UI,Roboto,Oxygen-Sans,Ubuntu,Cantarell,Helvetica Neue,sans-serif;line-height:1.4}.u-h3{color:#1b3051;font-size:18px;font-style:normal;margin-bottom:.7em}.u-h4{color:#1b3051;font-size:18px;font-style:italic;font-weight:700;margin-bottom:.7em}.u-vh-full{min-height:100vh}.u-hide{display:none;visibility:hidden}.u-hide:first-child+*{margin-block-start:0}@media only screen and (min-width:1024px){.u-hide-at-lg{display:none;visibility:hidden}}@media only screen and (max-width:1023px){.u-hide-at-lt-lg{display:none;visibility:hidden}.u-hide-at-lt-lg:first-child+*{margin-block-start:0}}.u-visually-hidden{clip:rect(0,0,0,0);border:0;height:1px;margin:-100%;overflow:hidden;padding:0;position:absolute!important;width:1px}.u-button--tertiary{font-size:.875rem;padding:8px 16px}@media only screen and (max-width:539px){.u-button--alt-colour-on-mobile{background-color:#f2f2f2;background-image:linear-gradient(#fff,#f2f2f2);border:1px solid #ccc;color:#004b83}}body{font-size:1.125rem}.c-header__navigation{display:flex;gap:.5rem .5rem} }</style> <style>@media only print, only all and (prefers-color-scheme: no-preference), only all and (prefers-color-scheme: light), only all and (prefers-color-scheme: dark) { button{line-height:inherit}html,label{font-family:-apple-system,BlinkMacSystemFont,Segoe UI,Roboto,Oxygen-Sans,Ubuntu,Cantarell,Helvetica Neue,sans-serif}html{-webkit-font-smoothing:subpixel-antialiased;box-sizing:border-box;color:#333;font-size:100%;height:100%;line-height:1.61803;overflow-y:scroll}*{box-sizing:inherit}body{background:#fff;margin:0;max-width:100%;min-height:100%}button,div,form,input,p{margin:0;padding:0}body{padding:0}a{color:#004b83;overflow-wrap:break-word;text-decoration:underline;text-decoration-skip-ink:auto;word-break:break-word}a>img{vertical-align:middle}h1,h2{color:#1b3051;font-style:normal;font-weight:700}h1{font-family:-apple-system,BlinkMacSystemFont,Segoe UI,Roboto,Oxygen-Sans,Ubuntu,Cantarell,Helvetica Neue,sans-serif;font-size:28px;line-height:1.4}html.webfonts-loaded h1,html.webfonts-loaded h2,html.webfonts-loaded h3{font-family:Europa,Trebuchet MS}h2{font-size:26px;margin-bottom:.5em}h2,h3{font-family:-apple-system,BlinkMacSystemFont,Segoe UI,Roboto,Oxygen-Sans,Ubuntu,Cantarell,Helvetica Neue,sans-serif;line-height:1.4}h3{color:#1b3051;font-size:18px;font-style:normal;margin-bottom:.7em}p{overflow-wrap:break-word;word-break:break-word}.u-h3{font-size:1.5rem}.u-h3,.u-h4{font-weight:700}.u-h4{font-size:1.25rem}.c-reading-companion__figure-title{font-size:1.25rem;font-weight:700}body{font-size:1.125rem}.c-article-header{font-family:-apple-system,BlinkMacSystemFont,Segoe UI,Roboto,Oxygen-Sans,Ubuntu,Cantarell,Helvetica Neue,sans-serif;margin-bottom:40px}.c-article-identifiers{color:#6f6f6f;display:flex;flex-wrap:wrap;font-size:1rem;line-height:1.3;list-style:none;margin:0 0 8px;padding:0}.c-article-identifiers__item{border-right:1px solid #6f6f6f;list-style:none;margin-right:8px;padding-right:8px}.c-article-identifiers__item:last-child{border-right:0;margin-right:0;padding-right:0}.c-article-title{font-size:1.5rem;line-height:1.25;margin-bottom:16px}@media only screen and (min-width:768px){.c-article-title{font-size:1.875rem;line-height:1.2}}.c-article-author-list{display:inline;font-size:1rem;list-style:none;margin:0 8px 0 0;padding:0;width:100%}.c-article-author-list__item{display:inline;padding-right:0}.c-article-author-list svg{margin-left:4px}.c-article-author-list__show-more{display:none;margin-right:4px}.c-article-author-list__button,.js .c-article-author-list__item--hide,.js .c-article-author-list__show-more{display:none}.js .c-article-author-list--long .c-article-author-list__show-more,.js .c-article-author-list--long+.c-article-author-list__button{display:inline}@media only screen and (max-width:539px){.js .c-article-author-list__item--hide-small-screen{display:none}.js .c-article-author-list--short .c-article-author-list__show-more,.js .c-article-author-list--short+.c-article-author-list__button{display:inline}}#uptodate-client,.js .c-article-author-list--expanded .c-article-author-list__show-more{display:none!important}.js .c-article-author-list--expanded .c-article-author-list__item--hide-small-screen{display:inline!important}.c-article-author-list__button,.c-button-author-list{background:#ebf1f5;border:4px solid #ebf1f5;border-radius:20px;color:#666;font-size:.875rem;line-height:1.4;padding:2px 11px 2px 8px;text-decoration:none}.c-article-author-list__button svg,.c-button-author-list svg{margin:1px 4px 0 0}.c-article-author-list__button:hover,.c-button-author-list:hover{background:#173962;border-color:transparent;color:#fff}.c-article-info-details{font-size:1rem;margin-bottom:8px;margin-top:16px}.c-article-info-details__cite-as{border-left:1px solid #6f6f6f;margin-left:8px;padding-left:8px}.c-article-metrics-bar{display:flex;flex-wrap:wrap;font-size:1rem;line-height:1.3}.c-article-metrics-bar__wrapper{margin:0 0 16px}.c-article-metrics-bar__item{align-items:baseline;border-right:1px solid #6f6f6f;margin-right:8px}.c-article-metrics-bar__item:last-child{border-right:0}.c-article-metrics-bar__count{font-weight:700;margin:0}.c-article-metrics-bar__label{color:#626262;font-style:normal;font-weight:400;margin:0 10px 0 5px}.c-article-metrics-bar__details{margin:0}.c-article-main-column{font-family:Georgia,Palatino,serif;margin-right:8.6%;width:60.2%}@media only screen and (max-width:1023px){.c-article-main-column{margin-right:0;width:100%}}.c-article-extras{float:left;font-family:-apple-system,BlinkMacSystemFont,Segoe UI,Roboto,Oxygen-Sans,Ubuntu,Cantarell,Helvetica Neue,sans-serif;width:31.2%}@media only screen and (max-width:1023px){.c-article-extras{display:none}}.c-article-associated-content__container .c-article-associated-content__title,.c-article-section__title{border-bottom:2px solid #d5d5d5;font-size:1.25rem;margin:0;padding-bottom:8px}@media only screen and (min-width:768px){.c-article-associated-content__container .c-article-associated-content__title,.c-article-section__title{font-size:1.5rem;line-height:1.24}}.c-article-associated-content__container .c-article-associated-content__title{margin-bottom:8px}.c-article-section{clear:both}.c-article-section__content{margin-bottom:40px;margin-top:0;padding-top:8px}@media only screen and (max-width:1023px){.c-article-section__content{padding-left:0}}.c-article__sub-heading{color:#222;font-family:-apple-system,BlinkMacSystemFont,Segoe UI,Roboto,Oxygen-Sans,Ubuntu,Cantarell,Helvetica Neue,sans-serif;font-size:1.125rem;font-style:normal;font-weight:400;line-height:1.3;margin:24px 0 8px}@media only screen and (min-width:768px){.c-article__sub-heading{font-size:1.5rem;line-height:1.24}}.c-article__sub-heading:first-child{margin-top:0}.c-article-authors-search{margin-bottom:24px;margin-top:0}.c-article-authors-search__item,.c-article-authors-search__title{font-family:-apple-system,BlinkMacSystemFont,Segoe UI,Roboto,Oxygen-Sans,Ubuntu,Cantarell,Helvetica Neue,sans-serif}.c-article-authors-search__title{color:#626262;font-size:1.05rem;font-weight:700;margin:0;padding:0}.c-article-authors-search__item{font-size:1rem}.c-article-authors-search__text{margin:0}.c-article-share-box__no-sharelink-info{font-size:.813rem;font-weight:700;margin-bottom:24px;padding-top:4px}.c-article-share-box__only-read-input{border:1px solid #d5d5d5;box-sizing:content-box;display:inline-block;font-size:.875rem;font-weight:700;height:24px;margin-bottom:8px;padding:8px 10px}.c-article-share-box__button--link-like{background-color:transparent;border:0;color:#0067c5;cursor:pointer;font-size:.875rem;margin-bottom:8px;margin-left:10px}.c-article-associated-content__container .c-article-associated-content__collection-label{font-size:.875rem;line-height:1.4}.c-article-associated-content__container .c-article-associated-content__collection-title{line-height:1.3}.c-context-bar{box-shadow:0 0 10px 0 rgba(51,51,51,.2);position:relative;width:100%}.c-context-bar__title{display:none}.c-reading-companion{clear:both;min-height:389px}.c-reading-companion__sticky{max-width:389px}.c-reading-companion__scroll-pane{margin:0;min-height:200px;overflow:hidden auto}.c-reading-companion__tabs{display:flex;flex-flow:row nowrap;font-size:1rem;list-style:none;margin:0 0 8px;padding:0}.c-reading-companion__tabs>li{flex-grow:1}.c-reading-companion__tab{background-color:#eee;border:1px solid #d5d5d5;border-image:initial;border-left-width:0;color:#0067c5;font-size:1rem;padding:8px 8px 8px 15px;text-align:left;width:100%}.c-reading-companion__tabs li:first-child .c-reading-companion__tab{border-left-width:1px}.c-reading-companion__tab--active{background-color:#fff;border-bottom:1px solid #fff;color:#222;font-weight:700}.c-reading-companion__sections-list{list-style:none;padding:0}.c-reading-companion__figures-list,.c-reading-companion__references-list{list-style:none;min-height:389px;padding:0}.c-reading-companion__references-list--numeric{list-style:decimal inside}.c-reading-companion__sections-list{margin:0 0 8px;min-height:50px}.c-reading-companion__section-item{font-size:1rem;padding:0}.c-reading-companion__section-item a{display:block;line-height:1.5;overflow:hidden;padding:8px 0 8px 16px;text-overflow:ellipsis;white-space:nowrap}.c-reading-companion__figure-item{border-top:1px solid #d5d5d5;font-size:1rem;padding:16px 8px 16px 0}.c-reading-companion__figure-item:first-child{border-top:none;padding-top:8px}.c-reading-companion__reference-item{border-top:1px solid #d5d5d5;font-size:1rem;padding:8px 8px 8px 16px}.c-reading-companion__reference-item:first-child{border-top:none}.c-reading-companion__reference-item a{word-break:break-word}.c-reading-companion__reference-citation{display:inline}.c-reading-companion__reference-links{font-size:.813rem;font-weight:700;list-style:none;margin:8px 0 0;padding:0;text-align:right}.c-reading-companion__reference-links>a{display:inline-block;padding-left:8px}.c-reading-companion__reference-links>a:first-child{display:inline-block;padding-left:0}.c-reading-companion__figure-title{display:block;margin:0 0 8px}.c-reading-companion__figure-links{display:flex;justify-content:space-between;margin:8px 0 0}.c-reading-companion__figure-links>a{align-items:center;display:flex}.c-reading-companion__figure-full-link svg{height:.8em;margin-left:2px}.c-reading-companion__panel{border-top:none;display:none;margin-top:0;padding-top:0}.c-reading-companion__panel--active{display:block}.c-pdf-download__link .u-icon{padding-top:2px}.c-pdf-download{display:flex;margin-bottom:16px;max-height:48px}@media only screen and (min-width:540px){.c-pdf-download{max-height:none}}@media only screen and (min-width:1024px){.c-pdf-download{max-height:48px}}.c-pdf-download__link{display:flex;flex:1 1 0%;padding:13px 24px!important}.c-pdf-download__text{padding-right:4px}@media only screen and (max-width:539px){.c-pdf-download__text{text-transform:capitalize}}@media only screen and (min-width:540px){.c-pdf-download__text{padding-right:8px}}.c-pdf-container{display:flex;justify-content:flex-end}@media only screen and (max-width:539px){.c-pdf-container .c-pdf-download{display:flex;flex-basis:100%}}.c-article-associated-content__container a,.c-card__summary a{text-decoration:underline}.u-display-none{display:none}.js .u-js-hide,.u-hide{display:none;visibility:hidden}.u-hide:first-child+*{margin-block-start:0}.u-visually-hidden{clip:rect(0,0,0,0);border:0;height:1px;margin:-100%;overflow:hidden;padding:0;position:absolute!important;width:1px}@media print{.u-hide-print{display:none}}@media only screen and (min-width:1024px){.u-hide-at-lg{display:none;visibility:hidden}}.u-icon{fill:currentcolor;display:inline-block;height:1em;transform:translate(0);vertical-align:text-top;width:1em}.u-list-reset{list-style:none;margin:0;padding:0}.hide{display:none;visibility:hidden}.c-article-associated-content__container .c-article-associated-content__collection.collection~.c-article-associated-content__collection.collection .c-article-associated-content__collection-label,.c-article-associated-content__container .c-article-associated-content__collection.section~.c-article-associated-content__collection.section .c-article-associated-content__collection-label,.c-article-associated-content__container .c-article-associated-content__title{display:none}.c-article-associated-content__container .c-article-associated-content__collection.collection .c-article-associated-content__collection-label,.c-article-associated-content__container .c-article-associated-content__collection.section .c-article-associated-content__collection-label{display:block}.c-article-associated-content__container .c-article-associated-content__collection.collection,.c-article-associated-content__container .c-article-associated-content__collection.section{margin-bottom:5px}.c-article-associated-content__container .c-article-associated-content__collection.section~.c-article-associated-content__collection.collection{margin-top:28px}.c-article-associated-content__container .c-article-associated-content__collection:first-child{margin-top:0}.c-article-associated-content__container .c-article-associated-content__collection:last-child{margin-bottom:2.4rem}.c-article-associated-content__container .c-article-associated-content__collection-label{color:#1b3051}.c-article-associated-content__container .c-article-associated-content__collection-title{font-size:1.063rem;font-weight:400}.webfonts-loaded .c-article__sub-heading{font-family:-apple-system,BlinkMacSystemFont,Segoe UI,Roboto,Oxygen-Sans,Ubuntu,Cantarell,Helvetica Neue,sans-serif} }</style> <link rel="stylesheet" data-test="critical-css-handler" data-inline-css-source="critical-css" href="/static/app-bmc/css/enhanced-32c9abc865.css" media="print" onload="this.media='only print, only all and (prefers-color-scheme: no-preference), only all and (prefers-color-scheme: light), only all and (prefers-color-scheme: dark)';this.onload=null"> <link rel="stylesheet" data-test="critical-css-handler" data-inline-css-source="critical-css" href="/static/app-bmc/css/enhanced-article-215af16b37.css" media="print" onload="this.media='only print, only all and (prefers-color-scheme: no-preference), only all and (prefers-color-scheme: light), only all and (prefers-color-scheme: dark)';this.onload=null"> <script type="text/javascript"> config = { env: 'live', site: 'trialsjournal.biomedcentral.com', siteWithPath: 'trialsjournal.biomedcentral.com' + window.location.pathname, twitterHashtag: 'Trialsjournal', cmsPrefix: 'https://studio-cms.springernature.com/studio/', doi: '10.1186/s13063-021-05489-x', figshareScriptUrl: 'https://widgets.figshare.com/static/figshare.js', hasFigshareInvoked: false, publisherBrand: 'BioMed Central', mustardcut: false }; </script> <script type="text/javascript" data-test="dataLayer"> window.dataLayer = [{"content":{"article":{"doi":"10.1186/s13063-021-05489-x","articleType":"Commentary","peerReviewType":"Open","supplement":null,"keywords":"Clinical trials as topic; Machine learning;Artificial intelligence;Research design;Research ethics"},"contentInfo":{"imprint":"BioMed Central","title":"The role of machine learning in clinical research: transforming the future of evidence generation","publishedAt":1629072000000,"publishedAtDate":"2021-08-16","author":["E. Hope Weissler","Tristan Naumann","Tomas Andersson","Rajesh Ranganath","Olivier Elemento","Yuan Luo","Daniel F. Freitag","James Benoit","Michael C. Hughes","Faisal Khan","Paul Slater","Khader Shameer","Matthew Roe","Emmette Hutchison","Scott H. Kollins","Uli Broedl","Zhaoling Meng","Jennifer L. Wong","Lesley Curtis","Erich Huang","Marzyeh Ghassemi"],"collection":[]},"attributes":{"deliveryPlatform":"oscar","template":"rebrand","cms":null,"copyright":{"creativeCommonsType":"CC BY + CC0","openAccess":true},"environment":"live"},"journal":{"siteKey":"trialsjournal.biomedcentral.com","volume":"22","issue":"1","title":"Trials","type":"BMC AJs","journalID":13063,"section":[]},"category":{"pmc":{"primarySubject":"Medicine & Public Health"},"contentType":"Commentary","publishingSegment":"BMC AJ4","snt":["Public Health","Biomedical Research","Biostatistics","Clinical Research","Clinical Medicine","Health Care"]}},"session":{"authentication":{"authenticationID":[]}},"version":"1.0.0","page":{"category":{"pageType":"article"},"attributes":{"featureFlags":[],"environment":"live","darwin":false}},"japan":false,"event":"dataLayerCreated","collection":null,"publisherBrand":"BioMed Central"}]; </script> <script> window.dataLayer = window.dataLayer || []; window.dataLayer.push({ ga4MeasurementId: 'G-PJCTJWPV25', ga360TrackingId: 'UA-54492316-9', twitterId: 'o47a2', baiduId: 'd1b4bc3b5ada4eb7290535e72899bac1', ga4ServerUrl: 'https://collect.biomedcentral.com', imprint: 'biomedcentral' }); </script> <script> (function(w, d) { w.config = w.config || {}; w.config.mustardcut = false; if (w.matchMedia && w.matchMedia('only print, only all and (prefers-color-scheme: no-preference), only all and (prefers-color-scheme: light), only all and (prefers-color-scheme: dark)').matches) { w.config.mustardcut = true; d.classList.add('js'); d.classList.remove('grade-c'); d.classList.remove('no-js'); } })(window, document.documentElement); </script> <script> (function () { if ( typeof window.CustomEvent === "function" ) return false; function CustomEvent ( event, params ) { params = params || { bubbles: false, cancelable: false, detail: null }; var evt = document.createEvent( 'CustomEvent' ); evt.initCustomEvent( event, params.bubbles, params.cancelable, params.detail ); return evt; } CustomEvent.prototype = window.Event.prototype; window.CustomEvent = CustomEvent; })(); </script> <script class="js-entry"> if (window.config.mustardcut) { (function(w, d) { window.Component = {}; window.suppressShareButton = true; window.onArticlePage = true; var currentScript = d.currentScript || d.head.querySelector('script.js-entry'); function catchNoModuleSupport() { var scriptEl = d.createElement('script'); return (!('noModule' in scriptEl) && 'onbeforeload' in scriptEl) } var headScripts = [ {'src': '/static/js/polyfill-es5-bundle-572d4fec60.js', 'async': false} ]; var bodyScripts = [ {'src': '/static/js/app-es5-bundle-d0ac94c97e.js', 'async': false, 'module': false}, {'src': '/static/js/app-es6-bundle-5ee1a6879c.js', 'async': false, 'module': true} , {'src': '/static/js/global-article-es5-bundle-ae3b685a1c.js', 'async': false, 'module': false}, {'src': '/static/js/global-article-es6-bundle-f72e3cd2ca.js', 'async': false, 'module': true} ]; function createScript(script) { var scriptEl = d.createElement('script'); scriptEl.src = script.src; scriptEl.async = script.async; if (script.module === true) { scriptEl.type = "module"; if (catchNoModuleSupport()) { scriptEl.src = ''; } } else if (script.module === false) { scriptEl.setAttribute('nomodule', true) } if (script.charset) { scriptEl.setAttribute('charset', script.charset); } return scriptEl; } for (var i = 0; i < headScripts.length; ++i) { var scriptEl = createScript(headScripts[i]); currentScript.parentNode.insertBefore(scriptEl, currentScript.nextSibling); } d.addEventListener('DOMContentLoaded', function() { for (var i = 0; i < bodyScripts.length; ++i) { var scriptEl = createScript(bodyScripts[i]); d.body.appendChild(scriptEl); } }); // Webfont repeat view var config = w.config; if (config && config.publisherBrand && sessionStorage.fontsLoaded === 'true') { d.documentElement.className += ' webfonts-loaded'; } })(window, document); } </script> <script data-src="https://cdn.optimizely.com/js/27195530232.js" data-cc-script="C03"></script> <script data-test="gtm-head"> window.initGTM = function() { (function (w, d, s, l, i) { w[l] = w[l] || []; w[l].push({'gtm.start': new Date().getTime(), event: 'gtm.js'}); var f = d.getElementsByTagName(s)[0], j = d.createElement(s), dl = l != 'dataLayer' ? '&l=' + l : ''; j.async = true; j.src = 'https://www.googletagmanager.com/gtm.js?id=' + i + dl; f.parentNode.insertBefore(j, f); })(window, document, 'script', 'dataLayer', 'GTM-MRVXSHQ'); } </script> <meta name="360-site-verification" content="2e1d87196f82c9ac5454a21aede69eda" /> <script> (function (w, d, t) { function cc() { var h = w.location.hostname; var e = d.createElement(t), s = d.getElementsByTagName(t)[0]; if (h.indexOf('springer.com') > -1 && h.indexOf('biomedcentral.com') === -1 && h.indexOf('springeropen.com') === -1) { if (h.indexOf('link-qa.springer.com') > -1 || h.indexOf('test-www.springer.com') > -1) { e.src = 'https://cmp.springer.com/production_live/en/consent-bundle-17-52.js'; e.setAttribute('onload', "initGTM(window,document,'script','dataLayer','GTM-MRVXSHQ')"); } else { e.src = 'https://cmp.springer.com/production_live/en/consent-bundle-17-52.js'; e.setAttribute('onload', "initGTM(window,document,'script','dataLayer','GTM-MRVXSHQ')"); } } else if (h.indexOf('biomedcentral.com') > -1) { if (h.indexOf('biomedcentral.com.qa') > -1) { e.src = 'https://cmp.biomedcentral.com/production_live/en/consent-bundle-15-36.js'; e.setAttribute('onload', "initGTM(window,document,'script','dataLayer','GTM-MRVXSHQ')"); } else { e.src = 'https://cmp.biomedcentral.com/production_live/en/consent-bundle-15-36.js'; e.setAttribute('onload', "initGTM(window,document,'script','dataLayer','GTM-MRVXSHQ')"); } } else if (h.indexOf('springeropen.com') > -1) { if (h.indexOf('springeropen.com.qa') > -1) { e.src = 'https://cmp.springernature.com/production_live/en/consent-bundle-16-34.js'; e.setAttribute('onload', "initGTM(window,document,'script','dataLayer','GTM-MRVXSHQ')"); } else { e.src = 'https://cmp.springernature.com/production_live/en/consent-bundle-16-34.js'; e.setAttribute('onload', "initGTM(window,document,'script','dataLayer','GTM-MRVXSHQ')"); } } else if (h.indexOf('springernature.com') > -1) { if (h.indexOf('beta-qa.springernature.com') > -1) { e.src = 'https://cmp.springernature.com/production_live/en/consent-bundle-49-43.js'; e.setAttribute('onload', "initGTM(window,document,'script','dataLayer','GTM-NK22KLS')"); } else { e.src = 'https://cmp.springernature.com/production_live/en/consent-bundle-49-43.js'; e.setAttribute('onload', "initGTM(window,document,'script','dataLayer','GTM-NK22KLS')"); } } else { e.src = '/static/js/cookie-consent-es5-bundle-cb57c2c98a.js'; e.setAttribute('data-consent', h); } s.insertAdjacentElement('afterend', e); } cc(); })(window, document, 'script'); </script> <link rel="canonical" href="https://trialsjournal.biomedcentral.com/articles/10.1186/s13063-021-05489-x"/> <meta property="og:url" content="https://trialsjournal.biomedcentral.com/articles/10.1186/s13063-021-05489-x"/> <meta property="og:type" content="article"/> <meta property="og:site_name" content="BioMed Central"/> <meta property="og:title" content="The role of machine learning in clinical research: transforming the future of evidence generation - Trials"/> <meta property="og:description" content="Background Interest in the application of machine learning (ML) to the design, conduct, and analysis of clinical trials has grown, but the evidence base for such applications has not been surveyed. This manuscript reviews the proceedings of a multi-stakeholder conference to discuss the current and future state of ML for clinical research. Key areas of clinical trial methodology in which ML holds particular promise and priority areas for further investigation are presented alongside a narrative review of evidence supporting the use of ML across the clinical trial spectrum. Results Conference attendees included stakeholders, such as biomedical and ML researchers, representatives from the US Food and Drug Administration (FDA), artificial intelligence technology and data analytics companies, non-profit organizations, patient advocacy groups, and pharmaceutical companies. ML contributions to clinical research were highlighted in the pre-trial phase, cohort selection and participant management, and data collection and analysis. A particular focus was paid to the operational and philosophical barriers to ML in clinical research. Peer-reviewed evidence was noted to be lacking in several areas. Conclusions ML holds great promise for improving the efficiency and quality of clinical research, but substantial barriers remain, the surmounting of which will require addressing significant gaps in evidence."/> <meta property="og:image" content="https://static-content.springer.com/image/art%3A10.1186%2Fs13063-021-05489-x/MediaObjects/13063_2021_5489_Fig1_HTML.png"/> <script type="application/ld+json">{"mainEntity":{"headline":"The role of machine learning in clinical research: transforming the future of evidence generation","description":"Interest in the application of machine learning (ML) to the design, conduct, and analysis of clinical trials has grown, but the evidence base for such applications has not been surveyed. This manuscript reviews the proceedings of a multi-stakeholder conference to discuss the current and future state of ML for clinical research. Key areas of clinical trial methodology in which ML holds particular promise and priority areas for further investigation are presented alongside a narrative review of evidence supporting the use of ML across the clinical trial spectrum. Conference attendees included stakeholders, such as biomedical and ML researchers, representatives from the US Food and Drug Administration (FDA), artificial intelligence technology and data analytics companies, non-profit organizations, patient advocacy groups, and pharmaceutical companies. ML contributions to clinical research were highlighted in the pre-trial phase, cohort selection and participant management, and data collection and analysis. A particular focus was paid to the operational and philosophical barriers to ML in clinical research. Peer-reviewed evidence was noted to be lacking in several areas. ML holds great promise for improving the efficiency and quality of clinical research, but substantial barriers remain, the surmounting of which will require addressing significant gaps in evidence.","datePublished":"2021-08-16T00:00:00Z","dateModified":"2021-09-06T00:00:00Z","pageStart":"1","pageEnd":"15","license":"http://creativecommons.org/publicdomain/zero/1.0/","sameAs":"https://doi.org/10.1186/s13063-021-05489-x","keywords":["Clinical trials as topic; Machine learning","Artificial intelligence","Research design","Research ethics","Medicine/Public Health","general","Biomedicine","Statistics for Life Sciences","Medicine","Health Sciences"],"image":["https://media.springernature.com/lw1200/springer-static/image/art%3A10.1186%2Fs13063-021-05489-x/MediaObjects/13063_2021_5489_Fig1_HTML.png","https://media.springernature.com/lw1200/springer-static/image/art%3A10.1186%2Fs13063-021-05489-x/MediaObjects/13063_2021_5489_Fig2_HTML.png","https://media.springernature.com/lw1200/springer-static/image/art%3A10.1186%2Fs13063-021-05489-x/MediaObjects/13063_2021_5489_Fig3_HTML.png"],"isPartOf":{"name":"Trials","issn":["1745-6215"],"volumeNumber":"22","@type":["Periodical","PublicationVolume"]},"publisher":{"name":"BioMed Central","logo":{"url":"https://www.springernature.com/app-sn/public/images/logo-springernature.png","@type":"ImageObject"},"@type":"Organization"},"author":[{"name":"E. Hope Weissler","url":"http://orcid.org/0000-0002-8442-6150","affiliation":[{"name":"Duke University School of Medicine","address":{"name":"Duke Clinical Research Institute, Duke University School of Medicine, Durham, USA","@type":"PostalAddress"},"@type":"Organization"}],"email":"Hope.weissler@duke.edu","@type":"Person"},{"name":"Tristan Naumann","affiliation":[{"name":"Microsoft Research","address":{"name":"Microsoft Research, Cambridge, USA","@type":"PostalAddress"},"@type":"Organization"}],"@type":"Person"},{"name":"Tomas Andersson","affiliation":[{"name":"AstraZeneca","address":{"name":"AstraZeneca, Gothenburg, Sweden","@type":"PostalAddress"},"@type":"Organization"}],"@type":"Person"},{"name":"Rajesh Ranganath","affiliation":[{"name":"New York University","address":{"name":"Courant Institute of Mathematical Science, New York University, New York, USA","@type":"PostalAddress"},"@type":"Organization"}],"@type":"Person"},{"name":"Olivier Elemento","affiliation":[{"name":"Weill Cornell Medical College","address":{"name":"Englander Institute for Precision Medicine, Weill Cornell Medical College, New York, USA","@type":"PostalAddress"},"@type":"Organization"}],"@type":"Person"},{"name":"Yuan Luo","affiliation":[{"name":"Northwestern University","address":{"name":"Northwestern University Clinical and Translational Sciences Institute, Northwestern University, Chicago, USA","@type":"PostalAddress"},"@type":"Organization"}],"@type":"Person"},{"name":"Daniel F. Freitag","affiliation":[{"name":"Bayer AG","address":{"name":"Division Pharmaceuticals, Open Innovation and Digital Technologies, Bayer AG, Wuppertal, Germany","@type":"PostalAddress"},"@type":"Organization"}],"@type":"Person"},{"name":"James Benoit","affiliation":[{"name":"University of Alberta","address":{"name":"University of Alberta, Edmonton, Canada","@type":"PostalAddress"},"@type":"Organization"}],"@type":"Person"},{"name":"Michael C. Hughes","affiliation":[{"name":"Tufts University","address":{"name":"Department of Computer Science, Tufts University, Medford, USA","@type":"PostalAddress"},"@type":"Organization"}],"@type":"Person"},{"name":"Faisal Khan","affiliation":[{"name":"AstraZeneca","address":{"name":"AstraZeneca, Gothenburg, Sweden","@type":"PostalAddress"},"@type":"Organization"}],"@type":"Person"},{"name":"Paul Slater","affiliation":[{"name":"Billion Minds, Inc.","address":{"name":"Billion Minds, Inc., Seattle, USA","@type":"PostalAddress"},"@type":"Organization"}],"@type":"Person"},{"name":"Khader Shameer","affiliation":[{"name":"AstraZeneca","address":{"name":"AstraZeneca, Gothenburg, Sweden","@type":"PostalAddress"},"@type":"Organization"}],"@type":"Person"},{"name":"Matthew Roe","affiliation":[{"name":"Verana Health","address":{"name":"Verana Health, San Francisco, USA","@type":"PostalAddress"},"@type":"Organization"}],"@type":"Person"},{"name":"Emmette Hutchison","affiliation":[{"name":"AstraZeneca","address":{"name":"AstraZeneca, Gothenburg, Sweden","@type":"PostalAddress"},"@type":"Organization"}],"@type":"Person"},{"name":"Scott H. Kollins","affiliation":[{"name":"Duke University School of Medicine","address":{"name":"Duke Clinical Research Institute, Duke University School of Medicine, Durham, USA","@type":"PostalAddress"},"@type":"Organization"}],"@type":"Person"},{"name":"Uli Broedl","affiliation":[{"name":"Boehringer-Ingelheim","address":{"name":"Boehringer-Ingelheim, Burlington, Canada","@type":"PostalAddress"},"@type":"Organization"}],"@type":"Person"},{"name":"Zhaoling Meng","affiliation":[{"name":"Sanofi","address":{"name":"Sanofi, Cambridge, USA","@type":"PostalAddress"},"@type":"Organization"}],"@type":"Person"},{"name":"Jennifer L. Wong","affiliation":[{"name":"Sanofi","address":{"name":"Sanofi, Washington, USA","@type":"PostalAddress"},"@type":"Organization"}],"@type":"Person"},{"name":"Lesley Curtis","affiliation":[{"name":"Duke University School of Medicine","address":{"name":"Duke Clinical Research Institute, Duke University School of Medicine, Durham, USA","@type":"PostalAddress"},"@type":"Organization"}],"@type":"Person"},{"name":"Erich Huang","affiliation":[{"name":"Duke University School of Medicine","address":{"name":"Duke Clinical Research Institute, Duke University School of Medicine, Durham, USA","@type":"PostalAddress"},"@type":"Organization"},{"name":"Duke Forge","address":{"name":"Duke Forge, Durham, USA","@type":"PostalAddress"},"@type":"Organization"}],"@type":"Person"},{"name":"Marzyeh Ghassemi","affiliation":[{"name":"University of Toronto","address":{"name":"Vector Institute, University of Toronto, Toronto, Canada","@type":"PostalAddress"},"@type":"Organization"},{"name":"Massachusetts Institute of Technology","address":{"name":"Department of Electrical Engineering and Computer Science, Massachusetts Institute of Technology, Cambridge, USA","@type":"PostalAddress"},"@type":"Organization"},{"name":"Massachusetts Institute of Technology","address":{"name":"Institute for Medical Engineering and Science, Massachusetts Institute of Technology, Cambridge, USA","@type":"PostalAddress"},"@type":"Organization"},{"name":"Vector Institute","address":{"name":"CIFAR AI Chair, Vector Institute, Ontario, Canada","@type":"PostalAddress"},"@type":"Organization"}],"@type":"Person"}],"isAccessibleForFree":true,"@type":"ScholarlyArticle"},"@context":"https://schema.org","@type":"WebPage"}</script> </head> <body class="journal journal-fulltext" > <div class="ctm"></div> <!-- Google Tag Manager (noscript) --> <noscript> <iframe src="https://www.googletagmanager.com/ns.html?id=GTM-MRVXSHQ" height="0" width="0" style="display:none;visibility:hidden"></iframe> </noscript> <!-- End Google Tag Manager (noscript) --> <div class="u-visually-hidden" aria-hidden="true"> <?xml version="1.0" encoding="UTF-8"?><!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN" "http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd"><svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink"><defs><path id="a" d="M0 .74h56.72v55.24H0z"/></defs><symbol id="icon-access" viewBox="0 0 18 18"><path d="m14 8c.5522847 0 1 .44771525 1 1v7h2.5c.2761424 0 .5.2238576.5.5v1.5h-18v-1.5c0-.2761424.22385763-.5.5-.5h2.5v-7c0-.55228475.44771525-1 1-1s1 .44771525 1 1v6.9996556h8v-6.9996556c0-.55228475.4477153-1 1-1zm-8 0 2 1v5l-2 1zm6 0v7l-2-1v-5zm-2.42653766-7.59857636 7.03554716 4.92488299c.4162533.29137735.5174853.86502537.226108 1.28127873-.1721584.24594054-.4534847.39241464-.7536934.39241464h-14.16284822c-.50810197 0-.92-.41189803-.92-.92 0-.30020869.1464741-.58153499.39241464-.75369337l7.03554714-4.92488299c.34432015-.2410241.80260453-.2410241 1.14692468 0zm-.57346234 2.03988748-3.65526982 2.55868888h7.31053962z" fill-rule="evenodd"/></symbol><symbol id="icon-account" viewBox="0 0 18 18"><path d="m10.2379028 16.9048051c1.3083556-.2032362 2.5118471-.7235183 3.5294683-1.4798399-.8731327-2.5141501-2.0638925-3.935978-3.7673711-4.3188248v-1.27684611c1.1651924-.41183641 2-1.52307546 2-2.82929429 0-1.65685425-1.3431458-3-3-3-1.65685425 0-3 1.34314575-3 3 0 1.30621883.83480763 2.41745788 2 2.82929429v1.27684611c-1.70347856.3828468-2.89423845 1.8046747-3.76737114 4.3188248 1.01762123.7563216 2.22111275 1.2766037 3.52946833 1.4798399.40563808.0629726.81921174.0951949 1.23790281.0951949s.83226473-.0322223 1.2379028-.0951949zm4.3421782-2.1721994c1.4927655-1.4532925 2.419919-3.484675 2.419919-5.7326057 0-4.418278-3.581722-8-8-8s-8 3.581722-8 8c0 2.2479307.92715352 4.2793132 2.41991895 5.7326057.75688473-2.0164459 1.83949951-3.6071894 3.48926591-4.3218837-1.14534283-.70360829-1.90918486-1.96796271-1.90918486-3.410722 0-2.209139 1.790861-4 4-4s4 1.790861 4 4c0 1.44275929-.763842 2.70711371-1.9091849 3.410722 1.6497664.7146943 2.7323812 2.3054378 3.4892659 4.3218837zm-5.580081 3.2673943c-4.97056275 0-9-4.0294373-9-9 0-4.97056275 4.02943725-9 9-9 4.9705627 0 9 4.02943725 9 9 0 4.9705627-4.0294373 9-9 9z" fill-rule="evenodd"/></symbol><symbol id="icon-alert" viewBox="0 0 18 18"><path d="m4 10h2.5c.27614237 0 .5.2238576.5.5s-.22385763.5-.5.5h-3.08578644l-1.12132034 1.1213203c-.18753638.1875364-.29289322.4418903-.29289322.7071068v.1715729h14v-.1715729c0-.2652165-.1053568-.5195704-.2928932-.7071068l-1.7071068-1.7071067v-3.4142136c0-2.76142375-2.2385763-5-5-5-2.76142375 0-5 2.23857625-5 5zm3 4c0 1.1045695.8954305 2 2 2s2-.8954305 2-2zm-5 0c-.55228475 0-1-.4477153-1-1v-.1715729c0-.530433.21071368-1.0391408.58578644-1.4142135l1.41421356-1.4142136v-3c0-3.3137085 2.6862915-6 6-6s6 2.6862915 6 6v3l1.4142136 1.4142136c.3750727.3750727.5857864.8837805.5857864 1.4142135v.1715729c0 .5522847-.4477153 1-1 1h-4c0 1.6568542-1.3431458 3-3 3-1.65685425 0-3-1.3431458-3-3z" fill-rule="evenodd"/></symbol><symbol id="icon-arrow-broad" viewBox="0 0 16 16"><path d="m6.10307866 2.97190702v7.69043288l2.44965196-2.44676915c.38776071-.38730439 1.0088052-.39493524 1.38498697-.01919617.38609051.38563612.38643641 1.01053024-.00013864 1.39665039l-4.12239817 4.11754683c-.38616704.3857126-1.01187344.3861062-1.39846576-.0000311l-4.12258206-4.11773056c-.38618426-.38572979-.39254614-1.00476697-.01636437-1.38050605.38609047-.38563611 1.01018509-.38751562 1.4012233.00306241l2.44985644 2.4469734v-8.67638639c0-.54139983.43698413-.98042709.98493125-.98159081l7.89910522-.0043627c.5451687 0 .9871152.44142642.9871152.98595351s-.4419465.98595351-.9871152.98595351z" fill-rule="evenodd" transform="matrix(-1 0 0 -1 14 15)"/></symbol><symbol id="icon-arrow-down" viewBox="0 0 16 16"><path d="m3.28337502 11.5302405 4.03074001 4.176208c.37758093.3912076.98937525.3916069 1.367372-.0000316l4.03091977-4.1763942c.3775978-.3912252.3838182-1.0190815.0160006-1.4001736-.3775061-.39113013-.9877245-.39303641-1.3700683.003106l-2.39538585 2.4818345v-11.6147896l-.00649339-.11662112c-.055753-.49733869-.46370161-.88337888-.95867408-.88337888-.49497246 0-.90292107.38604019-.95867408.88337888l-.00649338.11662112v11.6147896l-2.39518594-2.4816273c-.37913917-.39282218-.98637524-.40056175-1.35419292-.0194697-.37750607.3911302-.37784433 1.0249269.00013556 1.4165479z" fill-rule="evenodd"/></symbol><symbol id="icon-arrow-left" viewBox="0 0 16 16"><path d="m4.46975946 3.28337502-4.17620792 4.03074001c-.39120768.37758093-.39160691.98937525.0000316 1.367372l4.1763942 4.03091977c.39122514.3775978 1.01908149.3838182 1.40017357.0160006.39113012-.3775061.3930364-.9877245-.00310603-1.3700683l-2.48183446-2.39538585h11.61478958l.1166211-.00649339c.4973387-.055753.8833789-.46370161.8833789-.95867408 0-.49497246-.3860402-.90292107-.8833789-.95867408l-.1166211-.00649338h-11.61478958l2.4816273-2.39518594c.39282216-.37913917.40056173-.98637524.01946965-1.35419292-.39113012-.37750607-1.02492687-.37784433-1.41654791.00013556z" fill-rule="evenodd"/></symbol><symbol id="icon-arrow-right" viewBox="0 0 16 16"><path d="m11.5302405 12.716625 4.176208-4.03074003c.3912076-.37758093.3916069-.98937525-.0000316-1.367372l-4.1763942-4.03091981c-.3912252-.37759778-1.0190815-.38381821-1.4001736-.01600053-.39113013.37750607-.39303641.98772445.003106 1.37006824l2.4818345 2.39538588h-11.6147896l-.11662112.00649339c-.49733869.055753-.88337888.46370161-.88337888.95867408 0 .49497246.38604019.90292107.88337888.95867408l.11662112.00649338h11.6147896l-2.4816273 2.39518592c-.39282218.3791392-.40056175.9863753-.0194697 1.3541929.3911302.3775061 1.0249269.3778444 1.4165479-.0001355z" fill-rule="evenodd"/></symbol><symbol id="icon-arrow-sub" viewBox="0 0 16 16"><path d="m7.89692134 4.97190702v7.69043288l-2.44965196-2.4467692c-.38776071-.38730434-1.0088052-.39493519-1.38498697-.0191961-.38609047.3856361-.38643643 1.0105302.00013864 1.3966504l4.12239817 4.1175468c.38616704.3857126 1.01187344.3861062 1.39846576-.0000311l4.12258202-4.1177306c.3861843-.3857298.3925462-1.0047669.0163644-1.380506-.3860905-.38563612-1.0101851-.38751563-1.4012233.0030624l-2.44985643 2.4469734v-8.67638639c0-.54139983-.43698413-.98042709-.98493125-.98159081l-7.89910525-.0043627c-.54516866 0-.98711517.44142642-.98711517.98595351s.44194651.98595351.98711517.98595351z" fill-rule="evenodd"/></symbol><symbol id="icon-arrow-up" viewBox="0 0 16 16"><path d="m12.716625 4.46975946-4.03074003-4.17620792c-.37758093-.39120768-.98937525-.39160691-1.367372.0000316l-4.03091981 4.1763942c-.37759778.39122514-.38381821 1.01908149-.01600053 1.40017357.37750607.39113012.98772445.3930364 1.37006824-.00310603l2.39538588-2.48183446v11.61478958l.00649339.1166211c.055753.4973387.46370161.8833789.95867408.8833789.49497246 0 .90292107-.3860402.95867408-.8833789l.00649338-.1166211v-11.61478958l2.39518592 2.4816273c.3791392.39282216.9863753.40056173 1.3541929.01946965.3775061-.39113012.3778444-1.02492687-.0001355-1.41654791z" fill-rule="evenodd"/></symbol><symbol id="icon-article" viewBox="0 0 18 18"><path d="m13 15v-12.9906311c0-.0073595-.0019884-.0093689.0014977-.0093689l-11.00158888.00087166v13.00506804c0 .5482678.44615281.9940603.99415146.9940603h10.27350412c-.1701701-.2941734-.2675644-.6357129-.2675644-1zm-12 .0059397v-13.00506804c0-.5562408.44704472-1.00087166.99850233-1.00087166h11.00299537c.5510129 0 .9985023.45190985.9985023 1.0093689v2.9906311h3v9.9914698c0 1.1065798-.8927712 2.0085302-1.9940603 2.0085302h-12.01187942c-1.09954652 0-1.99406028-.8927712-1.99406028-1.9940603zm13-9.0059397v9c0 .5522847.4477153 1 1 1s1-.4477153 1-1v-9zm-10-2h7v4h-7zm1 1v2h5v-2zm-1 4h7v1h-7zm0 2h7v1h-7zm0 2h7v1h-7z" fill-rule="evenodd"/></symbol><symbol id="icon-audio" viewBox="0 0 18 18"><path d="m13.0957477 13.5588459c-.195279.1937043-.5119137.193729-.7072234.0000551-.1953098-.193674-.1953346-.5077061-.0000556-.7014104 1.0251004-1.0168342 1.6108711-2.3905226 1.6108711-3.85745208 0-1.46604976-.5850634-2.83898246-1.6090736-3.85566829-.1951894-.19379323-.1950192-.50782531.0003802-.70141028.1953993-.19358497.512034-.19341614.7072234.00037709 1.2094886 1.20083761 1.901635 2.8250555 1.901635 4.55670148 0 1.73268608-.6929822 3.35779608-1.9037571 4.55880738zm2.1233994 2.1025159c-.195234.193749-.5118687.1938462-.7072235.0002171-.1953548-.1936292-.1954528-.5076613-.0002189-.7014104 1.5832215-1.5711805 2.4881302-3.6939808 2.4881302-5.96012998 0-2.26581266-.9046382-4.3883241-2.487443-5.95944795-.1952117-.19377107-.1950777-.50780316.0002993-.70141031s.5120117-.19347426.7072234.00029682c1.7683321 1.75528196 2.7800854 4.12911258 2.7800854 6.66056144 0 2.53182498-1.0120556 4.90597838-2.7808529 6.66132328zm-14.21898205-3.6854911c-.5523759 0-1.00016505-.4441085-1.00016505-.991944v-3.96777631c0-.54783558.44778915-.99194407 1.00016505-.99194407h2.0003301l5.41965617-3.8393633c.44948677-.31842296 1.07413994-.21516983 1.39520191.23062232.12116339.16823446.18629727.36981184.18629727.57655577v12.01603479c0 .5478356-.44778914.9919441-1.00016505.9919441-.20845738 0-.41170538-.0645985-.58133413-.184766l-5.41965617-3.8393633zm0-.991944h2.32084805l5.68047235 4.0241292v-12.01603479l-5.68047235 4.02412928h-2.32084805z" fill-rule="evenodd"/></symbol><symbol id="icon-block" viewBox="0 0 24 24"><path d="m0 0h24v24h-24z" fill-rule="evenodd"/></symbol><symbol id="icon-book" viewBox="0 0 18 18"><path d="m4 13v-11h1v11h11v-11h-13c-.55228475 0-1 .44771525-1 1v10.2675644c.29417337-.1701701.63571286-.2675644 1-.2675644zm12 1h-13c-.55228475 0-1 .4477153-1 1s.44771525 1 1 1h13zm0 3h-13c-1.1045695 0-2-.8954305-2-2v-12c0-1.1045695.8954305-2 2-2h13c.5522847 0 1 .44771525 1 1v14c0 .5522847-.4477153 1-1 1zm-8.5-13h6c.2761424 0 .5.22385763.5.5s-.2238576.5-.5.5h-6c-.27614237 0-.5-.22385763-.5-.5s.22385763-.5.5-.5zm1 2h4c.2761424 0 .5.22385763.5.5s-.2238576.5-.5.5h-4c-.27614237 0-.5-.22385763-.5-.5s.22385763-.5.5-.5z" fill-rule="evenodd"/></symbol><symbol id="icon-broad" viewBox="0 0 24 24"><path d="m9.18274226 7.81v7.7999954l2.48162734-2.4816273c.3928221-.3928221 1.0219731-.4005617 1.4030652-.0194696.3911301.3911301.3914806 1.0249268-.0001404 1.4165479l-4.17620796 4.1762079c-.39120769.3912077-1.02508144.3916069-1.41671995-.0000316l-4.1763942-4.1763942c-.39122514-.3912251-.39767006-1.0190815-.01657798-1.4001736.39113012-.3911301 1.02337106-.3930364 1.41951349.0031061l2.48183446 2.4818344v-8.7999954c0-.54911294.4426881-.99439484.99778758-.99557515l8.00221246-.00442485c.5522847 0 1 .44771525 1 1s-.4477153 1-1 1z" fill-rule="evenodd" transform="matrix(-1 0 0 -1 20.182742 24.805206)"/></symbol><symbol id="icon-calendar" viewBox="0 0 18 18"><path d="m12.5 0c.2761424 0 .5.21505737.5.49047852v.50952148h2c1.1072288 0 2 .89451376 2 2v12c0 1.1072288-.8945138 2-2 2h-12c-1.1072288 0-2-.8945138-2-2v-12c0-1.1072288.89451376-2 2-2h1v1h-1c-.55393837 0-1 .44579254-1 1v3h14v-3c0-.55393837-.4457925-1-1-1h-2v1.50952148c0 .27088381-.2319336.49047852-.5.49047852-.2761424 0-.5-.21505737-.5-.49047852v-3.01904296c0-.27088381.2319336-.49047852.5-.49047852zm3.5 7h-14v8c0 .5539384.44579254 1 1 1h12c.5539384 0 1-.4457925 1-1zm-11 6v1h-1v-1zm3 0v1h-1v-1zm3 0v1h-1v-1zm-6-2v1h-1v-1zm3 0v1h-1v-1zm6 0v1h-1v-1zm-3 0v1h-1v-1zm-3-2v1h-1v-1zm6 0v1h-1v-1zm-3 0v1h-1v-1zm-5.5-9c.27614237 0 .5.21505737.5.49047852v.50952148h5v1h-5v1.50952148c0 .27088381-.23193359.49047852-.5.49047852-.27614237 0-.5-.21505737-.5-.49047852v-3.01904296c0-.27088381.23193359-.49047852.5-.49047852z" fill-rule="evenodd"/></symbol><symbol id="icon-cart" viewBox="0 0 18 18"><path d="m5 14c1.1045695 0 2 .8954305 2 2s-.8954305 2-2 2-2-.8954305-2-2 .8954305-2 2-2zm10 0c1.1045695 0 2 .8954305 2 2s-.8954305 2-2 2-2-.8954305-2-2 .8954305-2 2-2zm-10 1c-.55228475 0-1 .4477153-1 1s.44771525 1 1 1 1-.4477153 1-1-.44771525-1-1-1zm10 0c-.5522847 0-1 .4477153-1 1s.4477153 1 1 1 1-.4477153 1-1-.4477153-1-1-1zm-12.82032249-15c.47691417 0 .88746157.33678127.98070211.80449199l.23823144 1.19501025 13.36277974.00045554c.5522847.00001882.9999659.44774934.9999659 1.00004222 0 .07084994-.0075361.14150708-.022474.2107727l-1.2908094 5.98534344c-.1007861.46742419-.5432548.80388386-1.0571651.80388386h-10.24805106c-.59173366 0-1.07142857.4477153-1.07142857 1 0 .5128358.41361449.9355072.94647737.9932723l.1249512.0067277h10.35933776c.2749512 0 .4979349.2228539.4979349.4978051 0 .2749417-.2227336.4978951-.4976753.4980063l-10.35959736.0041886c-1.18346732 0-2.14285714-.8954305-2.14285714-2 0-.6625717.34520317-1.24989198.87690425-1.61383592l-1.63768102-8.19004794c-.01312273-.06561364-.01950005-.131011-.0196107-.19547395l-1.71961253-.00064219c-.27614237 0-.5-.22385762-.5-.5 0-.27614237.22385763-.5.5-.5zm14.53193359 2.99950224h-13.11300004l1.20580469 6.02530174c.11024034-.0163252.22327998-.02480398.33844139-.02480398h10.27064786z"/></symbol><symbol id="icon-chevron-less" viewBox="0 0 10 10"><path d="m5.58578644 4-3.29289322-3.29289322c-.39052429-.39052429-.39052429-1.02368927 0-1.41421356s1.02368927-.39052429 1.41421356 0l4 4c.39052429.39052429.39052429 1.02368927 0 1.41421356l-4 4c-.39052429.39052429-1.02368927.39052429-1.41421356 0s-.39052429-1.02368927 0-1.41421356z" fill-rule="evenodd" transform="matrix(0 -1 -1 0 9 9)"/></symbol><symbol id="icon-chevron-more" viewBox="0 0 10 10"><path d="m5.58578644 6-3.29289322-3.29289322c-.39052429-.39052429-.39052429-1.02368927 0-1.41421356s1.02368927-.39052429 1.41421356 0l4 4c.39052429.39052429.39052429 1.02368927 0 1.41421356l-4 4.00000002c-.39052429.3905243-1.02368927.3905243-1.41421356 0s-.39052429-1.02368929 0-1.41421358z" fill-rule="evenodd" transform="matrix(0 1 -1 0 11 1)"/></symbol><symbol id="icon-chevron-right" viewBox="0 0 10 10"><path d="m5.96738168 4.70639573 2.39518594-2.41447274c.37913917-.38219212.98637524-.38972225 1.35419292-.01894278.37750606.38054586.37784436.99719163-.00013556 1.37821513l-4.03074001 4.06319683c-.37758093.38062133-.98937525.38100976-1.367372-.00003075l-4.03091981-4.06337806c-.37759778-.38063832-.38381821-.99150444-.01600053-1.3622839.37750607-.38054587.98772445-.38240057 1.37006824.00302197l2.39538588 2.4146743.96295325.98624457z" fill-rule="evenodd" transform="matrix(0 -1 1 0 0 10)"/></symbol><symbol id="icon-circle-fill" viewBox="0 0 16 16"><path d="m8 14c-3.3137085 0-6-2.6862915-6-6s2.6862915-6 6-6 6 2.6862915 6 6-2.6862915 6-6 6z" fill-rule="evenodd"/></symbol><symbol id="icon-circle" viewBox="0 0 16 16"><path d="m8 12c2.209139 0 4-1.790861 4-4s-1.790861-4-4-4-4 1.790861-4 4 1.790861 4 4 4zm0 2c-3.3137085 0-6-2.6862915-6-6s2.6862915-6 6-6 6 2.6862915 6 6-2.6862915 6-6 6z" fill-rule="evenodd"/></symbol><symbol id="icon-citation" viewBox="0 0 18 18"><path d="m8.63593473 5.99995183c2.20913897 0 3.99999997 1.79084375 3.99999997 3.99996146 0 1.40730761-.7267788 2.64486871-1.8254829 3.35783281 1.6240224.6764218 2.8754442 2.0093871 3.4610603 3.6412466l-1.0763845.000006c-.5310008-1.2078237-1.5108121-2.1940153-2.7691712-2.7181346l-.79002167-.329052v-1.023992l.63016577-.4089232c.8482885-.5504661 1.3698342-1.4895187 1.3698342-2.51898361 0-1.65683828-1.3431457-2.99996146-2.99999997-2.99996146-1.65685425 0-3 1.34312318-3 2.99996146 0 1.02946491.52154569 1.96851751 1.36983419 2.51898361l.63016581.4089232v1.023992l-.79002171.329052c-1.25835905.5241193-2.23817037 1.5103109-2.76917113 2.7181346l-1.07638453-.000006c.58561612-1.6318595 1.8370379-2.9648248 3.46106024-3.6412466-1.09870405-.7129641-1.82548287-1.9505252-1.82548287-3.35783281 0-2.20911771 1.790861-3.99996146 4-3.99996146zm7.36897597-4.99995183c1.1018574 0 1.9950893.89353404 1.9950893 2.00274083v5.994422c0 1.10608317-.8926228 2.00274087-1.9950893 2.00274087l-3.0049107-.0009037v-1l3.0049107.00091329c.5490631 0 .9950893-.44783123.9950893-1.00275046v-5.994422c0-.55646537-.4450595-1.00275046-.9950893-1.00275046h-14.00982141c-.54906309 0-.99508929.44783123-.99508929 1.00275046v5.9971821c0 .66666024.33333333.99999036 1 .99999036l2-.00091329v1l-2 .0009037c-1 0-2-.99999041-2-1.99998077v-5.9971821c0-1.10608322.8926228-2.00274083 1.99508929-2.00274083zm-8.5049107 2.9999711c.27614237 0 .5.22385547.5.5 0 .2761349-.22385763.5-.5.5h-4c-.27614237 0-.5-.2238651-.5-.5 0-.27614453.22385763-.5.5-.5zm3 0c.2761424 0 .5.22385547.5.5 0 .2761349-.2238576.5-.5.5h-1c-.27614237 0-.5-.2238651-.5-.5 0-.27614453.22385763-.5.5-.5zm4 0c.2761424 0 .5.22385547.5.5 0 .2761349-.2238576.5-.5.5h-2c-.2761424 0-.5-.2238651-.5-.5 0-.27614453.2238576-.5.5-.5z" fill-rule="evenodd"/></symbol><symbol id="icon-close" viewBox="0 0 16 16"><path d="m2.29679575 12.2772478c-.39658757.3965876-.39438847 1.0328109-.00062148 1.4265779.39651227.3965123 1.03246768.3934888 1.42657791-.0006214l4.27724782-4.27724787 4.2772478 4.27724787c.3965876.3965875 1.0328109.3943884 1.4265779.0006214.3965123-.3965122.3934888-1.0324677-.0006214-1.4265779l-4.27724787-4.2772478 4.27724787-4.27724782c.3965875-.39658757.3943884-1.03281091.0006214-1.42657791-.3965122-.39651226-1.0324677-.39348875-1.4265779.00062148l-4.2772478 4.27724782-4.27724782-4.27724782c-.39658757-.39658757-1.03281091-.39438847-1.42657791-.00062148-.39651226.39651227-.39348875 1.03246768.00062148 1.42657791l4.27724782 4.27724782z" fill-rule="evenodd"/></symbol><symbol id="icon-collections" viewBox="0 0 18 18"><path d="m15 4c1.1045695 0 2 .8954305 2 2v9c0 1.1045695-.8954305 2-2 2h-8c-1.1045695 0-2-.8954305-2-2h1c0 .5128358.38604019.9355072.88337887.9932723l.11662113.0067277h8c.5128358 0 .9355072-.3860402.9932723-.8833789l.0067277-.1166211v-9c0-.51283584-.3860402-.93550716-.8833789-.99327227l-.1166211-.00672773h-1v-1zm-4-3c1.1045695 0 2 .8954305 2 2v9c0 1.1045695-.8954305 2-2 2h-8c-1.1045695 0-2-.8954305-2-2v-9c0-1.1045695.8954305-2 2-2zm0 1h-8c-.51283584 0-.93550716.38604019-.99327227.88337887l-.00672773.11662113v9c0 .5128358.38604019.9355072.88337887.9932723l.11662113.0067277h8c.5128358 0 .9355072-.3860402.9932723-.8833789l.0067277-.1166211v-9c0-.51283584-.3860402-.93550716-.8833789-.99327227zm-1.5 7c.27614237 0 .5.22385763.5.5s-.22385763.5-.5.5h-5c-.27614237 0-.5-.22385763-.5-.5s.22385763-.5.5-.5zm0-2c.27614237 0 .5.22385763.5.5s-.22385763.5-.5.5h-5c-.27614237 0-.5-.22385763-.5-.5s.22385763-.5.5-.5zm0-2c.27614237 0 .5.22385763.5.5s-.22385763.5-.5.5h-5c-.27614237 0-.5-.22385763-.5-.5s.22385763-.5.5-.5z" fill-rule="evenodd"/></symbol><symbol id="icon-compare" viewBox="0 0 18 18"><path d="m12 3c3.3137085 0 6 2.6862915 6 6s-2.6862915 6-6 6c-1.0928452 0-2.11744941-.2921742-2.99996061-.8026704-.88181407.5102749-1.90678042.8026704-3.00003939.8026704-3.3137085 0-6-2.6862915-6-6s2.6862915-6 6-6c1.09325897 0 2.11822532.29239547 3.00096303.80325037.88158756-.51107621 1.90619177-.80325037 2.99903697-.80325037zm-6 1c-2.76142375 0-5 2.23857625-5 5 0 2.7614237 2.23857625 5 5 5 .74397391 0 1.44999672-.162488 2.08451611-.4539116-1.27652344-1.1000812-2.08451611-2.7287264-2.08451611-4.5460884s.80799267-3.44600721 2.08434391-4.5463015c-.63434719-.29121054-1.34037-.4536985-2.08434391-.4536985zm6 0c-.7439739 0-1.4499967.16248796-2.08451611.45391156 1.27652341 1.10008123 2.08451611 2.72872644 2.08451611 4.54608844s-.8079927 3.4460072-2.08434391 4.5463015c.63434721.2912105 1.34037001.4536985 2.08434391.4536985 2.7614237 0 5-2.2385763 5-5 0-2.76142375-2.2385763-5-5-5zm-1.4162763 7.0005324h-3.16744736c.15614659.3572676.35283837.6927622.58425872 1.0006671h1.99892988c.23142036-.3079049.42811216-.6433995.58425876-1.0006671zm.4162763-2.0005324h-4c0 .34288501.0345146.67770871.10025909 1.0011864h3.79948181c.0657445-.32347769.1002591-.65830139.1002591-1.0011864zm-.4158423-1.99953894h-3.16831543c-.13859957.31730812-.24521946.651783-.31578599.99935097h3.79988742c-.0705665-.34756797-.1771864-.68204285-.315786-.99935097zm-1.58295822-1.999926-.08316107.06199199c-.34550042.27081213-.65446126.58611297-.91825862.93727862h2.00044041c-.28418626-.37830727-.6207872-.71499149-.99902072-.99927061z" fill-rule="evenodd"/></symbol><symbol id="icon-download-file" viewBox="0 0 18 18"><path d="m10.0046024 0c.5497429 0 1.3179837.32258606 1.707238.71184039l4.5763192 4.57631922c.3931386.39313859.7118404 1.16760135.7118404 1.71431368v8.98899651c0 1.1092806-.8945138 2.0085302-1.9940603 2.0085302h-12.01187942c-1.10128908 0-1.99406028-.8926228-1.99406028-1.9950893v-14.00982141c0-1.10185739.88743329-1.99508929 1.99961498-1.99508929zm0 1h-7.00498742c-.55709576 0-.99961498.44271433-.99961498.99508929v14.00982141c0 .5500396.44491393.9950893.99406028.9950893h12.01187942c.5463747 0 .9940603-.4506622.9940603-1.0085302v-8.98899651c0-.28393444-.2150684-.80332809-.4189472-1.0072069l-4.5763192-4.57631922c-.2038461-.20384606-.718603-.41894717-1.0001312-.41894717zm-1.5046024 4c.27614237 0 .5.21637201.5.49209595v6.14827645l1.7462789-1.77990922c.1933927-.1971171.5125222-.19455839.7001689-.0069117.1932998.19329992.1910058.50899492-.0027774.70277812l-2.59089271 2.5908927c-.19483374.1948337-.51177825.1937771-.70556873-.0000133l-2.59099079-2.5909908c-.19484111-.1948411-.19043735-.5151448-.00279066-.70279146.19329987-.19329987.50465175-.19237083.70018565.00692852l1.74638684 1.78001764v-6.14827695c0-.27177709.23193359-.49209595.5-.49209595z" fill-rule="evenodd"/></symbol><symbol id="icon-download" viewBox="0 0 16 16"><path d="m12.9975267 12.999368c.5467123 0 1.0024733.4478567 1.0024733 1.000316 0 .5563109-.4488226 1.000316-1.0024733 1.000316h-9.99505341c-.54671233 0-1.00247329-.4478567-1.00247329-1.000316 0-.5563109.44882258-1.000316 1.00247329-1.000316zm-4.9975267-11.999368c.55228475 0 1 .44497754 1 .99589209v6.80214418l2.4816273-2.48241149c.3928222-.39294628 1.0219732-.4006883 1.4030652-.01947579.3911302.39125371.3914806 1.02525073-.0001404 1.41699553l-4.17620792 4.17752758c-.39120769.3913313-1.02508144.3917306-1.41671995-.0000316l-4.17639421-4.17771394c-.39122513-.39134876-.39767006-1.01940351-.01657797-1.40061601.39113012-.39125372 1.02337105-.3931606 1.41951349.00310701l2.48183446 2.48261871v-6.80214418c0-.55001601.44386482-.99589209 1-.99589209z" fill-rule="evenodd"/></symbol><symbol id="icon-editors" viewBox="0 0 18 18"><path d="m8.72592184 2.54588137c-.48811714-.34391207-1.08343326-.54588137-1.72592184-.54588137-1.65685425 0-3 1.34314575-3 3 0 1.02947485.5215457 1.96853646 1.3698342 2.51900785l.6301658.40892721v1.02400182l-.79002171.32905522c-1.93395773.8055207-3.20997829 2.7024791-3.20997829 4.8180274v.9009805h-1v-.9009805c0-2.5479714 1.54557359-4.79153984 3.82548288-5.7411543-1.09870406-.71297106-1.82548288-1.95054399-1.82548288-3.3578652 0-2.209139 1.790861-4 4-4 1.09079823 0 2.07961816.43662103 2.80122451 1.1446278-.37707584.09278571-.7373238.22835063-1.07530267.40125357zm-2.72592184 14.45411863h-1v-.9009805c0-2.5479714 1.54557359-4.7915398 3.82548288-5.7411543-1.09870406-.71297106-1.82548288-1.95054399-1.82548288-3.3578652 0-2.209139 1.790861-4 4-4s4 1.790861 4 4c0 1.40732121-.7267788 2.64489414-1.8254829 3.3578652 2.2799093.9496145 3.8254829 3.1931829 3.8254829 5.7411543v.9009805h-1v-.9009805c0-2.1155483-1.2760206-4.0125067-3.2099783-4.8180274l-.7900217-.3290552v-1.02400184l.6301658-.40892721c.8482885-.55047139 1.3698342-1.489533 1.3698342-2.51900785 0-1.65685425-1.3431458-3-3-3-1.65685425 0-3 1.34314575-3 3 0 1.02947485.5215457 1.96853646 1.3698342 2.51900785l.6301658.40892721v1.02400184l-.79002171.3290552c-1.93395773.8055207-3.20997829 2.7024791-3.20997829 4.8180274z" fill-rule="evenodd"/></symbol><symbol id="icon-email" viewBox="0 0 18 18"><path d="m16.0049107 2c1.1018574 0 1.9950893.89706013 1.9950893 2.00585866v9.98828264c0 1.1078052-.8926228 2.0058587-1.9950893 2.0058587h-14.00982141c-1.10185739 0-1.99508929-.8970601-1.99508929-2.0058587v-9.98828264c0-1.10780515.8926228-2.00585866 1.99508929-2.00585866zm0 1h-14.00982141c-.54871518 0-.99508929.44887827-.99508929 1.00585866v9.98828264c0 .5572961.44630695 1.0058587.99508929 1.0058587h14.00982141c.5487152 0 .9950893-.4488783.9950893-1.0058587v-9.98828264c0-.55729607-.446307-1.00585866-.9950893-1.00585866zm-.0049107 2.55749512v1.44250488l-7 4-7-4v-1.44250488l7 4z" fill-rule="evenodd"/></symbol><symbol id="icon-error" viewBox="0 0 18 18"><path d="m9 0c4.9705627 0 9 4.02943725 9 9 0 4.9705627-4.0294373 9-9 9-4.97056275 0-9-4.0294373-9-9 0-4.97056275 4.02943725-9 9-9zm2.8630343 4.71100931-2.8630343 2.86303426-2.86303426-2.86303426c-.39658757-.39658757-1.03281091-.39438847-1.4265779-.00062147-.39651227.39651226-.39348876 1.03246767.00062147 1.4265779l2.86303426 2.86303426-2.86303426 2.8630343c-.39658757.3965875-.39438847 1.0328109-.00062147 1.4265779.39651226.3965122 1.03246767.3934887 1.4265779-.0006215l2.86303426-2.8630343 2.8630343 2.8630343c.3965875.3965876 1.0328109.3943885 1.4265779.0006215.3965122-.3965123.3934887-1.0324677-.0006215-1.4265779l-2.8630343-2.8630343 2.8630343-2.86303426c.3965876-.39658757.3943885-1.03281091.0006215-1.4265779-.3965123-.39651227-1.0324677-.39348876-1.4265779.00062147z" fill-rule="evenodd"/></symbol><symbol id="icon-ethics" viewBox="0 0 18 18"><path d="m6.76384967 1.41421356.83301651-.8330165c.77492941-.77492941 2.03133823-.77492941 2.80626762 0l.8330165.8330165c.3750728.37507276.8837806.58578644 1.4142136.58578644h1.3496361c1.1045695 0 2 .8954305 2 2v1.34963611c0 .53043298.2107137 1.03914081.5857864 1.41421356l.8330165.83301651c.7749295.77492941.7749295 2.03133823 0 2.80626762l-.8330165.8330165c-.3750727.3750728-.5857864.8837806-.5857864 1.4142136v1.3496361c0 1.1045695-.8954305 2-2 2h-1.3496361c-.530433 0-1.0391408.2107137-1.4142136.5857864l-.8330165.8330165c-.77492939.7749295-2.03133821.7749295-2.80626762 0l-.83301651-.8330165c-.37507275-.3750727-.88378058-.5857864-1.41421356-.5857864h-1.34963611c-1.1045695 0-2-.8954305-2-2v-1.3496361c0-.530433-.21071368-1.0391408-.58578644-1.4142136l-.8330165-.8330165c-.77492941-.77492939-.77492941-2.03133821 0-2.80626762l.8330165-.83301651c.37507276-.37507275.58578644-.88378058.58578644-1.41421356v-1.34963611c0-1.1045695.8954305-2 2-2h1.34963611c.53043298 0 1.03914081-.21071368 1.41421356-.58578644zm-1.41421356 1.58578644h-1.34963611c-.55228475 0-1 .44771525-1 1v1.34963611c0 .79564947-.31607052 1.55871121-.87867966 2.12132034l-.8330165.83301651c-.38440512.38440512-.38440512 1.00764896 0 1.39205408l.8330165.83301646c.56260914.5626092.87867966 1.3256709.87867966 2.1213204v1.3496361c0 .5522847.44771525 1 1 1h1.34963611c.79564947 0 1.55871121.3160705 2.12132034.8786797l.83301651.8330165c.38440512.3844051 1.00764896.3844051 1.39205408 0l.83301646-.8330165c.5626092-.5626092 1.3256709-.8786797 2.1213204-.8786797h1.3496361c.5522847 0 1-.4477153 1-1v-1.3496361c0-.7956495.3160705-1.5587112.8786797-2.1213204l.8330165-.83301646c.3844051-.38440512.3844051-1.00764896 0-1.39205408l-.8330165-.83301651c-.5626092-.56260913-.8786797-1.32567087-.8786797-2.12132034v-1.34963611c0-.55228475-.4477153-1-1-1h-1.3496361c-.7956495 0-1.5587112-.31607052-2.1213204-.87867966l-.83301646-.8330165c-.38440512-.38440512-1.00764896-.38440512-1.39205408 0l-.83301651.8330165c-.56260913.56260914-1.32567087.87867966-2.12132034.87867966zm3.58698944 11.4960218c-.02081224.002155-.04199226.0030286-.06345763.002542-.98766446-.0223875-1.93408568-.3063547-2.75885125-.8155622-.23496767-.1450683-.30784554-.4531483-.16277726-.688116.14506827-.2349677.45314827-.3078455.68811595-.1627773.67447084.4164161 1.44758575.6483839 2.25617384.6667123.01759529.0003988.03495764.0017019.05204365.0038639.01713363-.0017748.03452416-.0026845.05212715-.0026845 2.4852814 0 4.5-2.0147186 4.5-4.5 0-1.04888973-.3593547-2.04134635-1.0074477-2.83787157-.1742817-.21419731-.1419238-.5291218.0722736-.70340353.2141973-.17428173.5291218-.14192375.7034035.07227357.7919032.97327203 1.2317706 2.18808682 1.2317706 3.46900153 0 3.0375661-2.4624339 5.5-5.5 5.5-.02146768 0-.04261937-.0013529-.06337445-.0039782zm1.57975095-10.78419583c.2654788.07599731.419084.35281842.3430867.61829728-.0759973.26547885-.3528185.419084-.6182973.3430867-.37560116-.10752146-.76586237-.16587951-1.15568824-.17249193-2.5587807-.00064534-4.58547766 2.00216524-4.58547766 4.49928198 0 .62691557.12797645 1.23496.37274865 1.7964426.11035133.2531347-.0053975.5477984-.25853224.6581497-.25313473.1103514-.54779841-.0053975-.65814974-.2585322-.29947131-.6869568-.45606667-1.43097603-.45606667-2.1960601 0-3.05211432 2.47714695-5.50006595 5.59399617-5.49921198.48576182.00815502.96289603.0795037 1.42238033.21103795zm-1.9766658 6.41091303 2.69835-2.94655317c.1788432-.21040373.4943901-.23598862.7047939-.05714545.2104037.17884318.2359886.49439014.0571454.70479387l-3.01637681 3.34277395c-.18039088.1999106-.48669547.2210637-.69285412.0478478l-1.93095347-1.62240047c-.21213845-.17678204-.24080048-.49206439-.06401844-.70420284.17678204-.21213844.49206439-.24080048.70420284-.06401844z" fill-rule="evenodd"/></symbol><symbol id="icon-expand"><path d="M7.498 11.918a.997.997 0 0 0-.003-1.411.995.995 0 0 0-1.412-.003l-4.102 4.102v-3.51A1 1 0 0 0 .98 10.09.992.992 0 0 0 0 11.092V17c0 .554.448 1.002 1.002 1.002h5.907c.554 0 1.002-.45 1.002-1.003 0-.539-.45-.978-1.006-.978h-3.51zm3.005-5.835a.997.997 0 0 0 .003 1.412.995.995 0 0 0 1.411.003l4.103-4.103v3.51a1 1 0 0 0 1.001 1.006A.992.992 0 0 0 18 6.91V1.002A1 1 0 0 0 17 0h-5.907a1.003 1.003 0 0 0-1.002 1.003c0 .539.45.978 1.006.978h3.51z" fill-rule="evenodd"/></symbol><symbol id="icon-explore" viewBox="0 0 18 18"><path d="m9 17c4.418278 0 8-3.581722 8-8s-3.581722-8-8-8-8 3.581722-8 8 3.581722 8 8 8zm0 1c-4.97056275 0-9-4.0294373-9-9 0-4.97056275 4.02943725-9 9-9 4.9705627 0 9 4.02943725 9 9 0 4.9705627-4.0294373 9-9 9zm0-2.5c-.27614237 0-.5-.2238576-.5-.5s.22385763-.5.5-.5c2.969509 0 5.400504-2.3575119 5.497023-5.31714844.0090007-.27599565.2400359-.49243782.5160315-.48343711.2759957.0090007.4924378.2400359.4834371.51603155-.114093 3.4985237-2.9869632 6.284554-6.4964916 6.284554zm-.29090657-12.99359748c.27587424-.01216621.50937715.20161139.52154336.47748563.01216621.27587423-.20161139.50937715-.47748563.52154336-2.93195733.12930094-5.25315116 2.54886451-5.25315116 5.49456849 0 .27614237-.22385763.5-.5.5s-.5-.22385763-.5-.5c0-3.48142406 2.74307146-6.34074398 6.20909343-6.49359748zm1.13784138 8.04763908-1.2004882-1.20048821c-.19526215-.19526215-.19526215-.51184463 0-.70710678s.51184463-.19526215.70710678 0l1.20048821 1.2004882 1.6006509-4.00162734-4.50670359 1.80268144-1.80268144 4.50670359zm4.10281269-6.50378907-2.6692597 6.67314927c-.1016411.2541026-.3029834.4554449-.557086.557086l-6.67314927 2.6692597 2.66925969-6.67314926c.10164107-.25410266.30298336-.45544495.55708602-.55708602z" fill-rule="evenodd"/></symbol><symbol id="icon-filter" viewBox="0 0 16 16"><path d="m14.9738641 0c.5667192 0 1.0261359.4477136 1.0261359 1 0 .24221858-.0902161.47620768-.2538899.65849851l-5.6938314 6.34147206v5.49997973c0 .3147562-.1520673.6111434-.4104543.7999971l-2.05227171 1.4999945c-.45337535.3313696-1.09655869.2418269-1.4365902-.1999993-.13321514-.1730955-.20522717-.3836284-.20522717-.5999978v-6.99997423l-5.69383133-6.34147206c-.3731872-.41563511-.32996891-1.0473954.09653074-1.41107611.18705584-.15950448.42716133-.2474224.67571519-.2474224zm-5.9218641 8.5h-2.105v6.491l.01238459.0070843.02053271.0015705.01955278-.0070558 2.0532976-1.4990996zm-8.02585008-7.5-.01564945.00240169 5.83249953 6.49759831h2.313l5.836-6.499z"/></symbol><symbol id="icon-home" viewBox="0 0 18 18"><path d="m9 5-6 6v5h4v-4h4v4h4v-5zm7 6.5857864v4.4142136c0 .5522847-.4477153 1-1 1h-5v-4h-2v4h-5c-.55228475 0-1-.4477153-1-1v-4.4142136c-.25592232 0-.51184464-.097631-.70710678-.2928932l-.58578644-.5857864c-.39052429-.3905243-.39052429-1.02368929 0-1.41421358l8.29289322-8.29289322 8.2928932 8.29289322c.3905243.39052429.3905243 1.02368928 0 1.41421358l-.5857864.5857864c-.1952622.1952622-.4511845.2928932-.7071068.2928932zm-7-9.17157284-7.58578644 7.58578644.58578644.5857864 7-6.99999996 7 6.99999996.5857864-.5857864z" fill-rule="evenodd"/></symbol><symbol id="icon-image" viewBox="0 0 18 18"><path d="m10.0046024 0c.5497429 0 1.3179837.32258606 1.707238.71184039l4.5763192 4.57631922c.3931386.39313859.7118404 1.16760135.7118404 1.71431368v8.98899651c0 1.1092806-.8945138 2.0085302-1.9940603 2.0085302h-12.01187942c-1.10128908 0-1.99406028-.8926228-1.99406028-1.9950893v-14.00982141c0-1.10185739.88743329-1.99508929 1.99961498-1.99508929zm-3.49645283 10.1752453-3.89407257 6.7495552c.11705545.048464.24538859.0751995.37998328.0751995h10.60290092l-2.4329715-4.2154691-1.57494129 2.7288098zm8.49779013 6.8247547c.5463747 0 .9940603-.4506622.9940603-1.0085302v-8.98899651c0-.28393444-.2150684-.80332809-.4189472-1.0072069l-4.5763192-4.57631922c-.2038461-.20384606-.718603-.41894717-1.0001312-.41894717h-7.00498742c-.55709576 0-.99961498.44271433-.99961498.99508929v13.98991071l4.50814957-7.81026689 3.08089884 5.33809539 1.57494129-2.7288097 3.5875735 6.2159812zm-3.0059397-11c1.1045695 0 2 .8954305 2 2s-.8954305 2-2 2-2-.8954305-2-2 .8954305-2 2-2zm0 1c-.5522847 0-1 .44771525-1 1s.4477153 1 1 1 1-.44771525 1-1-.4477153-1-1-1z" fill-rule="evenodd"/></symbol><symbol id="icon-info" viewBox="0 0 18 18"><path d="m9 0c4.9705627 0 9 4.02943725 9 9 0 4.9705627-4.0294373 9-9 9-4.97056275 0-9-4.0294373-9-9 0-4.97056275 4.02943725-9 9-9zm0 7h-1.5l-.11662113.00672773c-.49733868.05776511-.88337887.48043643-.88337887.99327227 0 .47338693.32893365.86994729.77070917.97358929l.1126697.01968298.11662113.00672773h.5v3h-.5l-.11662113.0067277c-.42082504.0488782-.76196299.3590206-.85696816.7639815l-.01968298.1126697-.00672773.1166211.00672773.1166211c.04887817.4208251.35902055.761963.76398144.8569682l.1126697.019683.11662113.0067277h3l.1166211-.0067277c.4973387-.0577651.8833789-.4804365.8833789-.9932723 0-.4733869-.3289337-.8699473-.7707092-.9735893l-.1126697-.019683-.1166211-.0067277h-.5v-4l-.00672773-.11662113c-.04887817-.42082504-.35902055-.76196299-.76398144-.85696816l-.1126697-.01968298zm0-3.25c-.69035594 0-1.25.55964406-1.25 1.25s.55964406 1.25 1.25 1.25 1.25-.55964406 1.25-1.25-.55964406-1.25-1.25-1.25z" fill-rule="evenodd"/></symbol><symbol id="icon-institution" viewBox="0 0 18 18"><path d="m7 16.9998189v-2.0003623h4v2.0003623h2v-3.0005434h-8v3.0005434zm-3-10.00181122h-1.52632364c-.27614237 0-.5-.22389817-.5-.50009056 0-.13995446.05863589-.27350497.16166338-.36820841l1.23156713-1.13206327h-2.36690687v12.00217346h3v-2.0003623h-3v-1.0001811h3v-1.0001811h1v-4.00072448h-1zm10 0v2.00036224h-1v4.00072448h1v1.0001811h3v1.0001811h-3v2.0003623h3v-12.00217346h-2.3695309l1.2315671 1.13206327c.2033191.186892.2166633.50325042.0298051.70660631-.0946863.10304615-.2282126.16169266-.3681417.16169266zm3-3.00054336c.5522847 0 1 .44779634 1 1.00018112v13.00235456h-18v-13.00235456c0-.55238478.44771525-1.00018112 1-1.00018112h3.45499992l4.20535144-3.86558216c.19129876-.17584288.48537447-.17584288.67667324 0l4.2053514 3.86558216zm-4 3.00054336h-8v1.00018112h8zm-2 6.00108672h1v-4.00072448h-1zm-1 0v-4.00072448h-2v4.00072448zm-3 0v-4.00072448h-1v4.00072448zm8-4.00072448c.5522847 0 1 .44779634 1 1.00018112v2.00036226h-2v-2.00036226c0-.55238478.4477153-1.00018112 1-1.00018112zm-12 0c.55228475 0 1 .44779634 1 1.00018112v2.00036226h-2v-2.00036226c0-.55238478.44771525-1.00018112 1-1.00018112zm5.99868798-7.81907007-5.24205601 4.81852671h10.48411203zm.00131202 3.81834559c-.55228475 0-1-.44779634-1-1.00018112s.44771525-1.00018112 1-1.00018112 1 .44779634 1 1.00018112-.44771525 1.00018112-1 1.00018112zm-1 11.00199236v1.0001811h2v-1.0001811z" fill-rule="evenodd"/></symbol><symbol id="icon-location" viewBox="0 0 18 18"><path d="m9.39521328 16.2688008c.79596342-.7770119 1.59208152-1.6299956 2.33285652-2.5295081 1.4020032-1.7024324 2.4323601-3.3624519 2.9354918-4.871847.2228715-.66861448.3364384-1.29323246.3364384-1.8674457 0-3.3137085-2.6862915-6-6-6-3.36356866 0-6 2.60156856-6 6 0 .57421324.11356691 1.19883122.3364384 1.8674457.50313169 1.5093951 1.53348863 3.1694146 2.93549184 4.871847.74077492.8995125 1.53689309 1.7524962 2.33285648 2.5295081.13694479.1336842.26895677.2602648.39521328.3793207.12625651-.1190559.25826849-.2456365.39521328-.3793207zm-.39521328 1.7311992s-7-6-7-11c0-4 3.13400675-7 7-7 3.8659932 0 7 3.13400675 7 7 0 5-7 11-7 11zm0-8c-1.65685425 0-3-1.34314575-3-3s1.34314575-3 3-3c1.6568542 0 3 1.34314575 3 3s-1.3431458 3-3 3zm0-1c1.1045695 0 2-.8954305 2-2s-.8954305-2-2-2-2 .8954305-2 2 .8954305 2 2 2z" fill-rule="evenodd"/></symbol><symbol id="icon-minus" viewBox="0 0 16 16"><path d="m2.00087166 7h11.99825664c.5527662 0 1.0008717.44386482 1.0008717 1 0 .55228475-.4446309 1-1.0008717 1h-11.99825664c-.55276616 0-1.00087166-.44386482-1.00087166-1 0-.55228475.44463086-1 1.00087166-1z" fill-rule="evenodd"/></symbol><symbol id="icon-newsletter" viewBox="0 0 18 18"><path d="m9 11.8482489 2-1.1428571v-1.7053918h-4v1.7053918zm-3-1.7142857v-2.1339632h6v2.1339632l3-1.71428574v-6.41967746h-12v6.41967746zm10-5.3839632 1.5299989.95624934c.2923814.18273835.4700011.50320827.4700011.8479983v8.44575236c0 1.1045695-.8954305 2-2 2h-14c-1.1045695 0-2-.8954305-2-2v-8.44575236c0-.34479003.1776197-.66525995.47000106-.8479983l1.52999894-.95624934v-2.75c0-.55228475.44771525-1 1-1h12c.5522847 0 1 .44771525 1 1zm0 1.17924764v3.07075236l-7 4-7-4v-3.07075236l-1 .625v8.44575236c0 .5522847.44771525 1 1 1h14c.5522847 0 1-.4477153 1-1v-8.44575236zm-10-1.92924764h6v1h-6zm-1 2h8v1h-8z" fill-rule="evenodd"/></symbol><symbol id="icon-orcid" viewBox="0 0 18 18"><path d="m9 1c4.418278 0 8 3.581722 8 8s-3.581722 8-8 8-8-3.581722-8-8 3.581722-8 8-8zm-2.90107518 5.2732337h-1.41865256v7.1712107h1.41865256zm4.55867178.02508949h-2.99247027v7.14612121h2.91062487c.7673039 0 1.4476365-.1483432 2.0410182-.445034s1.0511995-.7152915 1.3734671-1.2558144c.3222677-.540523.4833991-1.1603247.4833991-1.85942385 0-.68545815-.1602789-1.30270225-.4808414-1.85175082-.3205625-.54904856-.7707074-.97532211-1.3504481-1.27883343-.5797408-.30351132-1.2413173-.45526471-1.9847495-.45526471zm-.1892674 1.07933542c.7877654 0 1.4143875.22336734 1.8798852.67010873.4654977.44674138.698243 1.05546001.698243 1.82617415 0 .74343221-.2310402 1.34447791-.6931277 1.80315511-.4620874.4586773-1.0750688.6880124-1.8389625.6880124h-1.46810075v-4.98745039zm-5.08652545-3.71099194c-.21825533 0-.410525.08444276-.57681478.25333081-.16628977.16888806-.24943341.36245684-.24943341.58071218 0 .22345188.08314364.41961891.24943341.58850696.16628978.16888806.35855945.25333082.57681478.25333082.233845 0 .43390938-.08314364.60019916-.24943342.16628978-.16628977.24943342-.36375592.24943342-.59240436 0-.233845-.08314364-.43131115-.24943342-.59240437s-.36635416-.24163862-.60019916-.24163862z" fill-rule="evenodd"/></symbol><symbol id="icon-plus" viewBox="0 0 16 16"><path d="m2.00087166 7h4.99912834v-4.99912834c0-.55276616.44386482-1.00087166 1-1.00087166.55228475 0 1 .44463086 1 1.00087166v4.99912834h4.9991283c.5527662 0 1.0008717.44386482 1.0008717 1 0 .55228475-.4446309 1-1.0008717 1h-4.9991283v4.9991283c0 .5527662-.44386482 1.0008717-1 1.0008717-.55228475 0-1-.4446309-1-1.0008717v-4.9991283h-4.99912834c-.55276616 0-1.00087166-.44386482-1.00087166-1 0-.55228475.44463086-1 1.00087166-1z" fill-rule="evenodd"/></symbol><symbol id="icon-print" viewBox="0 0 18 18"><path d="m16.0049107 5h-14.00982141c-.54941618 0-.99508929.4467783-.99508929.99961498v6.00077002c0 .5570958.44271433.999615.99508929.999615h1.00491071v-3h12v3h1.0049107c.5494162 0 .9950893-.4467783.9950893-.999615v-6.00077002c0-.55709576-.4427143-.99961498-.9950893-.99961498zm-2.0049107-1v-2.00208688c0-.54777062-.4519464-.99791312-1.0085302-.99791312h-7.9829396c-.55661731 0-1.0085302.44910695-1.0085302.99791312v2.00208688zm1 10v2.0018986c0 1.103521-.9019504 1.9981014-2.0085302 1.9981014h-7.9829396c-1.1092806 0-2.0085302-.8867064-2.0085302-1.9981014v-2.0018986h-1.00491071c-1.10185739 0-1.99508929-.8874333-1.99508929-1.999615v-6.00077002c0-1.10435686.8926228-1.99961498 1.99508929-1.99961498h1.00491071v-2.00208688c0-1.10341695.90195036-1.99791312 2.0085302-1.99791312h7.9829396c1.1092806 0 2.0085302.89826062 2.0085302 1.99791312v2.00208688h1.0049107c1.1018574 0 1.9950893.88743329 1.9950893 1.99961498v6.00077002c0 1.1043569-.8926228 1.999615-1.9950893 1.999615zm-1-3h-10v5.0018986c0 .5546075.44702548.9981014 1.0085302.9981014h7.9829396c.5565964 0 1.0085302-.4491701 1.0085302-.9981014zm-9 1h8v1h-8zm0 2h5v1h-5zm9-5c-.5522847 0-1-.44771525-1-1s.4477153-1 1-1 1 .44771525 1 1-.4477153 1-1 1z" fill-rule="evenodd"/></symbol><symbol id="icon-search" viewBox="0 0 22 22"><path d="M21.697 20.261a1.028 1.028 0 01.01 1.448 1.034 1.034 0 01-1.448-.01l-4.267-4.267A9.812 9.811 0 010 9.812a9.812 9.811 0 1117.43 6.182zM9.812 18.222A8.41 8.41 0 109.81 1.403a8.41 8.41 0 000 16.82z" fill-rule="evenodd"/></symbol><symbol id="icon-social-facebook" viewBox="0 0 24 24"><path d="m6.00368507 20c-1.10660471 0-2.00368507-.8945138-2.00368507-1.9940603v-12.01187942c0-1.10128908.89451376-1.99406028 1.99406028-1.99406028h12.01187942c1.1012891 0 1.9940603.89451376 1.9940603 1.99406028v12.01187942c0 1.1012891-.88679 1.9940603-2.0032184 1.9940603h-2.9570132v-6.1960818h2.0797387l.3114113-2.414723h-2.39115v-1.54164807c0-.69911803.1941355-1.1755439 1.1966615-1.1755439l1.2786739-.00055875v-2.15974763l-.2339477-.02492088c-.3441234-.03134957-.9500153-.07025255-1.6293054-.07025255-1.8435726 0-3.1057323 1.12531866-3.1057323 3.19187953v1.78079225h-2.0850778v2.414723h2.0850778v6.1960818z" fill-rule="evenodd"/></symbol><symbol id="icon-social-twitter" viewBox="0 0 24 24"><path d="m18.8767135 6.87445248c.7638174-.46908424 1.351611-1.21167363 1.6250764-2.09636345-.7135248.43394112-1.50406.74870123-2.3464594.91677702-.6695189-.73342162-1.6297913-1.19486605-2.6922204-1.19486605-2.0399895 0-3.6933555 1.69603749-3.6933555 3.78628909 0 .29642457.0314329.58673729.0942985.8617704-3.06469922-.15890802-5.78835241-1.66547825-7.60988389-3.9574208-.3174714.56076194-.49978171 1.21167363-.49978171 1.90536824 0 1.31404706.65223085 2.47224203 1.64236444 3.15218497-.60350999-.0198635-1.17401554-.1925232-1.67222562-.47366811v.04583885c0 1.83355406 1.27302891 3.36609966 2.96411421 3.71294696-.31118484.0886217-.63651445.1329326-.97441718.1329326-.2357461 0-.47149219-.0229194-.69466516-.0672303.47149219 1.5065703 1.83253297 2.6036468 3.44975116 2.632678-1.2651707 1.0160946-2.85724264 1.6196394-4.5891906 1.6196394-.29861172 0-.59093688-.0152796-.88011875-.0504227 1.63450624 1.0726291 3.57548241 1.6990934 5.66104951 1.6990934 6.79263079 0 10.50641749-5.7711113 10.50641749-10.7751859l-.0094298-.48894775c.7229547-.53478659 1.3516109-1.20250585 1.8419628-1.96190282-.6632323.30100846-1.3751855.50422736-2.1217148.59590507z" fill-rule="evenodd"/></symbol><symbol id="icon-social-youtube" viewBox="0 0 24 24"><path d="m10.1415 14.3973208-.0005625-5.19318431 4.863375 2.60554491zm9.963-7.92753362c-.6845625-.73643756-1.4518125-.73990314-1.803375-.7826454-2.518875-.18714178-6.2971875-.18714178-6.2971875-.18714178-.007875 0-3.7861875 0-6.3050625.18714178-.352125.04274226-1.1188125.04620784-1.8039375.7826454-.5394375.56084773-.7149375 1.8344515-.7149375 1.8344515s-.18 1.49597903-.18 2.99138042v1.4024082c0 1.495979.18 2.9913804.18 2.9913804s.1755 1.2736038.7149375 1.8344515c.685125.7364376 1.5845625.7133337 1.9850625.7901542 1.44.1420891 6.12.1859866 6.12.1859866s3.78225-.005776 6.301125-.1929178c.3515625-.0433198 1.1188125-.0467854 1.803375-.783223.5394375-.5608477.7155-1.8344515.7155-1.8344515s.18-1.4954014.18-2.9913804v-1.4024082c0-1.49540139-.18-2.99138042-.18-2.99138042s-.1760625-1.27360377-.7155-1.8344515z" fill-rule="evenodd"/></symbol><symbol id="icon-subject-medicine" viewBox="0 0 18 18"><path d="m12.5 8h-6.5c-1.65685425 0-3 1.34314575-3 3v1c0 1.6568542 1.34314575 3 3 3h1v-2h-.5c-.82842712 0-1.5-.6715729-1.5-1.5s.67157288-1.5 1.5-1.5h1.5 2 1 2c1.6568542 0 3-1.34314575 3-3v-1c0-1.65685425-1.3431458-3-3-3h-2v2h1.5c.8284271 0 1.5.67157288 1.5 1.5s-.6715729 1.5-1.5 1.5zm-5.5-1v-1h-3.5c-1.38071187 0-2.5-1.11928813-2.5-2.5s1.11928813-2.5 2.5-2.5h1.02786405c.46573528 0 .92507448.10843528 1.34164078.31671843l1.13382424.56691212c.06026365-1.05041141.93116291-1.88363055 1.99667093-1.88363055 1.1045695 0 2 .8954305 2 2h2c2.209139 0 4 1.790861 4 4v1c0 2.209139-1.790861 4-4 4h-2v1h2c1.1045695 0 2 .8954305 2 2s-.8954305 2-2 2h-2c0 1.1045695-.8954305 2-2 2s-2-.8954305-2-2h-1c-2.209139 0-4-1.790861-4-4v-1c0-2.209139 1.790861-4 4-4zm0-2v-2.05652691c-.14564246-.03538148-.28733393-.08714006-.42229124-.15461871l-1.15541752-.57770876c-.27771087-.13885544-.583937-.21114562-.89442719-.21114562h-1.02786405c-.82842712 0-1.5.67157288-1.5 1.5s.67157288 1.5 1.5 1.5zm4 1v1h1.5c.2761424 0 .5-.22385763.5-.5s-.2238576-.5-.5-.5zm-1 1v-5c0-.55228475-.44771525-1-1-1s-1 .44771525-1 1v5zm-2 4v5c0 .5522847.44771525 1 1 1s1-.4477153 1-1v-5zm3 2v2h2c.5522847 0 1-.4477153 1-1s-.4477153-1-1-1zm-4-1v-1h-.5c-.27614237 0-.5.2238576-.5.5s.22385763.5.5.5zm-3.5-9h1c.27614237 0 .5.22385763.5.5s-.22385763.5-.5.5h-1c-.27614237 0-.5-.22385763-.5-.5s.22385763-.5.5-.5z" fill-rule="evenodd"/></symbol><symbol id="icon-success" viewBox="0 0 18 18"><path d="m9 0c4.9705627 0 9 4.02943725 9 9 0 4.9705627-4.0294373 9-9 9-4.97056275 0-9-4.0294373-9-9 0-4.97056275 4.02943725-9 9-9zm3.4860198 4.98163161-4.71802968 5.50657859-2.62834168-2.02300024c-.42862421-.36730544-1.06564993-.30775346-1.42283677.13301307-.35718685.44076653-.29927542 1.0958383.12934879 1.46314377l3.40735508 2.7323063c.42215801.3385221 1.03700951.2798252 1.38749189-.1324571l5.38450527-6.33394549c.3613513-.43716226.3096573-1.09278382-.115462-1.46437175-.4251192-.37158792-1.0626796-.31842941-1.4240309.11873285z" fill-rule="evenodd"/></symbol><symbol id="icon-table" viewBox="0 0 18 18"><path d="m16.0049107 2c1.1018574 0 1.9950893.89706013 1.9950893 2.00585866v9.98828264c0 1.1078052-.8926228 2.0058587-1.9950893 2.0058587l-4.0059107-.001.001.001h-1l-.001-.001h-5l.001.001h-1l-.001-.001-3.00391071.001c-1.10185739 0-1.99508929-.8970601-1.99508929-2.0058587v-9.98828264c0-1.10780515.8926228-2.00585866 1.99508929-2.00585866zm-11.0059107 5h-3.999v6.9941413c0 .5572961.44630695 1.0058587.99508929 1.0058587h3.00391071zm6 0h-5v8h5zm5.0059107-4h-4.0059107v3h5.001v1h-5.001v7.999l4.0059107.001c.5487152 0 .9950893-.4488783.9950893-1.0058587v-9.98828264c0-.55729607-.446307-1.00585866-.9950893-1.00585866zm-12.5049107 9c.27614237 0 .5.2238576.5.5s-.22385763.5-.5.5h-1c-.27614237 0-.5-.2238576-.5-.5s.22385763-.5.5-.5zm12 0c.2761424 0 .5.2238576.5.5s-.2238576.5-.5.5h-2c-.2761424 0-.5-.2238576-.5-.5s.2238576-.5.5-.5zm-6 0c.27614237 0 .5.2238576.5.5s-.22385763.5-.5.5h-2c-.27614237 0-.5-.2238576-.5-.5s.22385763-.5.5-.5zm-6-2c.27614237 0 .5.2238576.5.5s-.22385763.5-.5.5h-1c-.27614237 0-.5-.2238576-.5-.5s.22385763-.5.5-.5zm12 0c.2761424 0 .5.2238576.5.5s-.2238576.5-.5.5h-2c-.2761424 0-.5-.2238576-.5-.5s.2238576-.5.5-.5zm-6 0c.27614237 0 .5.2238576.5.5s-.22385763.5-.5.5h-2c-.27614237 0-.5-.2238576-.5-.5s.22385763-.5.5-.5zm-6-2c.27614237 0 .5.22385763.5.5s-.22385763.5-.5.5h-1c-.27614237 0-.5-.22385763-.5-.5s.22385763-.5.5-.5zm12 0c.2761424 0 .5.22385763.5.5s-.2238576.5-.5.5h-2c-.2761424 0-.5-.22385763-.5-.5s.2238576-.5.5-.5zm-6 0c.27614237 0 .5.22385763.5.5s-.22385763.5-.5.5h-2c-.27614237 0-.5-.22385763-.5-.5s.22385763-.5.5-.5zm1.499-5h-5v3h5zm-6 0h-3.00391071c-.54871518 0-.99508929.44887827-.99508929 1.00585866v1.99414134h3.999z" fill-rule="evenodd"/></symbol><symbol id="icon-tick-circle" viewBox="0 0 24 24"><path d="m12 2c5.5228475 0 10 4.4771525 10 10s-4.4771525 10-10 10-10-4.4771525-10-10 4.4771525-10 10-10zm0 1c-4.97056275 0-9 4.02943725-9 9 0 4.9705627 4.02943725 9 9 9 4.9705627 0 9-4.0294373 9-9 0-4.97056275-4.0294373-9-9-9zm4.2199868 5.36606669c.3613514-.43716226.9989118-.49032077 1.424031-.11873285s.4768133 1.02720949.115462 1.46437175l-6.093335 6.94397871c-.3622945.4128716-.9897871.4562317-1.4054264.0971157l-3.89719065-3.3672071c-.42862421-.3673054-.48653564-1.0223772-.1293488-1.4631437s.99421256-.5003185 1.42283677-.1330131l3.11097438 2.6987741z" fill-rule="evenodd"/></symbol><symbol id="icon-tick" viewBox="0 0 16 16"><path d="m6.76799012 9.21106946-3.1109744-2.58349728c-.42862421-.35161617-1.06564993-.29460792-1.42283677.12733148s-.29927541 1.04903009.1293488 1.40064626l3.91576307 3.23873978c.41034319.3393961 1.01467563.2976897 1.37450571-.0948578l6.10568327-6.660841c.3613513-.41848908.3096572-1.04610608-.115462-1.4018218-.4251192-.35571573-1.0626796-.30482786-1.424031.11366122z" fill-rule="evenodd"/></symbol><symbol id="icon-update" viewBox="0 0 18 18"><path d="m1 13v1c0 .5522847.44771525 1 1 1h14c.5522847 0 1-.4477153 1-1v-1h-1v-10h-14v10zm16-1h1v2c0 1.1045695-.8954305 2-2 2h-14c-1.1045695 0-2-.8954305-2-2v-2h1v-9c0-.55228475.44771525-1 1-1h14c.5522847 0 1 .44771525 1 1zm-1 0v1h-4.5857864l-1 1h-2.82842716l-1-1h-4.58578644v-1h5l1 1h2l1-1zm-13-8h12v7h-12zm1 1v5h10v-5zm1 1h4v1h-4zm0 2h4v1h-4z" fill-rule="evenodd"/></symbol><symbol id="icon-upload" viewBox="0 0 18 18"><path d="m10.0046024 0c.5497429 0 1.3179837.32258606 1.707238.71184039l4.5763192 4.57631922c.3931386.39313859.7118404 1.16760135.7118404 1.71431368v8.98899651c0 1.1092806-.8945138 2.0085302-1.9940603 2.0085302h-12.01187942c-1.10128908 0-1.99406028-.8926228-1.99406028-1.9950893v-14.00982141c0-1.10185739.88743329-1.99508929 1.99961498-1.99508929zm0 1h-7.00498742c-.55709576 0-.99961498.44271433-.99961498.99508929v14.00982141c0 .5500396.44491393.9950893.99406028.9950893h12.01187942c.5463747 0 .9940603-.4506622.9940603-1.0085302v-8.98899651c0-.28393444-.2150684-.80332809-.4189472-1.0072069l-4.5763192-4.57631922c-.2038461-.20384606-.718603-.41894717-1.0001312-.41894717zm-1.85576936 4.14572769c.19483374-.19483375.51177826-.19377714.70556874.00001334l2.59099082 2.59099079c.1948411.19484112.1904373.51514474.0027906.70279143-.1932998.19329987-.5046517.19237083-.7001856-.00692852l-1.74638687-1.7800176v6.14827687c0 .2717771-.23193359.492096-.5.492096-.27614237 0-.5-.216372-.5-.492096v-6.14827641l-1.74627892 1.77990922c-.1933927.1971171-.51252214.19455839-.70016883.0069117-.19329987-.19329988-.19100584-.50899493.00277731-.70277808z" fill-rule="evenodd"/></symbol><symbol id="icon-video" viewBox="0 0 18 18"><path d="m16.0049107 2c1.1018574 0 1.9950893.89706013 1.9950893 2.00585866v9.98828264c0 1.1078052-.8926228 2.0058587-1.9950893 2.0058587h-14.00982141c-1.10185739 0-1.99508929-.8970601-1.99508929-2.0058587v-9.98828264c0-1.10780515.8926228-2.00585866 1.99508929-2.00585866zm0 1h-14.00982141c-.54871518 0-.99508929.44887827-.99508929 1.00585866v9.98828264c0 .5572961.44630695 1.0058587.99508929 1.0058587h14.00982141c.5487152 0 .9950893-.4488783.9950893-1.0058587v-9.98828264c0-.55729607-.446307-1.00585866-.9950893-1.00585866zm-8.30912922 2.24944486 4.60460462 2.73982242c.9365543.55726659.9290753 1.46522435 0 2.01804082l-4.60460462 2.7398224c-.93655425.5572666-1.69578148.1645632-1.69578148-.8937585v-5.71016863c0-1.05087579.76670616-1.446575 1.69578148-.89375851zm-.67492769.96085624v5.5750128c0 .2995102-.10753745.2442517.16578928.0847713l4.58452283-2.67497259c.3050619-.17799716.3051624-.21655446 0-.39461026l-4.58452283-2.67497264c-.26630747-.15538481-.16578928-.20699944-.16578928.08477139z" fill-rule="evenodd"/></symbol><symbol id="icon-warning" viewBox="0 0 18 18"><path d="m9 11.75c.69035594 0 1.25.5596441 1.25 1.25s-.55964406 1.25-1.25 1.25-1.25-.5596441-1.25-1.25.55964406-1.25 1.25-1.25zm.41320045-7.75c.55228475 0 1.00000005.44771525 1.00000005 1l-.0034543.08304548-.3333333 4c-.043191.51829212-.47645714.91695452-.99654578.91695452h-.15973424c-.52008864 0-.95335475-.3986624-.99654576-.91695452l-.33333333-4c-.04586475-.55037702.36312325-1.03372649.91350028-1.07959124l.04148683-.00259031zm-.41320045 14c-4.97056275 0-9-4.0294373-9-9 0-4.97056275 4.02943725-9 9-9 4.9705627 0 9 4.02943725 9 9 0 4.9705627-4.0294373 9-9 9z" fill-rule="evenodd"/></symbol><symbol id="icon-arrow-left-bullet" viewBox="0 0 8 16"><path d="M3 8l5 5v3L0 8l8-8v3L3 8z"/></symbol><symbol id="icon-chevron-down" viewBox="0 0 16 16"><path d="m5.58578644 3-3.29289322-3.29289322c-.39052429-.39052429-.39052429-1.02368927 0-1.41421356s1.02368927-.39052429 1.41421356 0l4 4c.39052429.39052429.39052429 1.02368927 0 1.41421356l-4 4c-.39052429.39052429-1.02368927.39052429-1.41421356 0s-.39052429-1.02368927 0-1.41421356z" fill-rule="evenodd" transform="matrix(0 1 -1 0 11 3)"/></symbol><symbol id="icon-download-rounded"><path d="M0 13c0-.556.449-1 1.002-1h9.996a.999.999 0 110 2H1.002A1.006 1.006 0 010 13zM7 1v6.8l2.482-2.482c.392-.392 1.022-.4 1.403-.02a1.001 1.001 0 010 1.417l-4.177 4.177a1.001 1.001 0 01-1.416 0L1.115 6.715a.991.991 0 01-.016-1.4 1 1 0 011.42.003L5 7.8V1c0-.55.444-.996 1-.996.552 0 1 .445 1 .996z"/></symbol><symbol id="icon-ext-link" viewBox="0 0 16 16"><path d="M12.9 16H3.1C1.4 16 0 14.6 0 12.9V3.2C0 1.4 1.4 0 3.1 0h3.7v1H3.1C2 1 1 2 1 3.2v9.7C1 14 2 15 3.1 15h9.7c1.2 0 2.1-1 2.1-2.1V8.7h1v4.2c.1 1.7-1.3 3.1-3 3.1z"/><path d="M12.8 2.5l.7.7-9 8.9-.7-.7 9-8.9z"/><path d="M9.7 0L16 6.2V0z"/></symbol><symbol id="icon-remove" viewBox="-296 388 18 18"><path d="M-291.7 396.1h9v2h-9z"/><path d="M-287 405.5c-4.7 0-8.5-3.8-8.5-8.5s3.8-8.5 8.5-8.5 8.5 3.8 8.5 8.5-3.8 8.5-8.5 8.5zm0-16c-4.1 0-7.5 3.4-7.5 7.5s3.4 7.5 7.5 7.5 7.5-3.4 7.5-7.5-3.4-7.5-7.5-7.5z"/></symbol><symbol id="icon-rss" viewBox="0 0 18 18"><path d="m.97480857 6.01583891.11675372.00378391c5.75903295.51984988 10.34261021 5.10537458 10.85988231 10.86480098.0494035.5500707-.3564674 1.0360406-.906538 1.0854441-.5500707.0494036-1.0360406-.3564673-1.08544412-.906538-.43079083-4.7965248-4.25151132-8.61886853-9.04770289-9.05180573-.55004837-.04965115-.95570047-.53580366-.90604933-1.08585203.04610464-.5107592.46858035-.89701345.96909831-.90983323zm1.52519143 6.95474179c1.38071187 0 2.5 1.1192881 2.5 2.5s-1.11928813 2.5-2.5 2.5-2.5-1.1192881-2.5-2.5 1.11928813-2.5 2.5-2.5zm-1.43253846-12.96884168c9.09581416.53242539 16.37540296 7.8163886 16.90205336 16.91294558.0319214.5513615-.389168 1.0242056-.9405294 1.056127-.5513615.0319214-1.0242057-.389168-1.0561271-.9405294-.4679958-8.08344784-6.93949306-14.55883389-15.02226722-15.03196077-.55134101-.03227286-.97212889-.50538538-.93985602-1.05672639.03227286-.551341.50538538-.97212888 1.05672638-.93985602z" fill-rule="evenodd"/></symbol><symbol id="icon-springer-arrow-left"><path d="M15 7a1 1 0 000-2H3.385l2.482-2.482a.994.994 0 00.02-1.403 1.001 1.001 0 00-1.417 0L.294 5.292a1.001 1.001 0 000 1.416l4.176 4.177a.991.991 0 001.4.016 1 1 0 00-.003-1.42L3.385 7H15z"/></symbol><symbol id="icon-springer-arrow-right"><path d="M1 7a1 1 0 010-2h11.615l-2.482-2.482a.994.994 0 01-.02-1.403 1.001 1.001 0 011.417 0l4.176 4.177a1.001 1.001 0 010 1.416l-4.176 4.177a.991.991 0 01-1.4.016 1 1 0 01.003-1.42L12.615 7H1z"/></symbol><symbol id="icon-springer-collections" viewBox="3 3 32 32"><path fill-rule="evenodd" d="M25.583333,30.1249997 L25.583333,7.1207574 C25.583333,7.10772495 25.579812,7.10416665 25.5859851,7.10416665 L6.10400517,7.10571021 L6.10400517,30.1355179 C6.10400517,31.1064087 6.89406744,31.8958329 7.86448169,31.8958329 L26.057145,31.8958329 C25.7558021,31.374901 25.583333,30.7700915 25.583333,30.1249997 Z M4.33333333,30.1355179 L4.33333333,7.10571021 C4.33333333,6.12070047 5.12497502,5.33333333 6.10151452,5.33333333 L25.5859851,5.33333333 C26.5617372,5.33333333 27.3541664,6.13359035 27.3541664,7.1207574 L27.3541664,12.4166666 L32.6666663,12.4166666 L32.6666663,30.1098941 C32.6666663,32.0694626 31.0857174,33.6666663 29.1355179,33.6666663 L7.86448169,33.6666663 C5.91736809,33.6666663 4.33333333,32.0857174 4.33333333,30.1355179 Z M27.3541664,14.1874999 L27.3541664,30.1249997 C27.3541664,31.1030039 28.1469954,31.8958329 29.1249997,31.8958329 C30.1030039,31.8958329 30.8958329,31.1030039 30.8958329,30.1249997 L30.8958329,14.1874999 L27.3541664,14.1874999 Z M9.64583326,10.6458333 L22.0416665,10.6458333 L22.0416665,17.7291665 L9.64583326,17.7291665 L9.64583326,10.6458333 Z M11.4166666,12.4166666 L11.4166666,15.9583331 L20.2708331,15.9583331 L20.2708331,12.4166666 L11.4166666,12.4166666 Z M9.64583326,19.4999998 L22.0416665,19.4999998 L22.0416665,21.2708331 L9.64583326,21.2708331 L9.64583326,19.4999998 Z M9.64583326,23.0416665 L22.0416665,23.0416665 L22.0416665,24.8124997 L9.64583326,24.8124997 L9.64583326,23.0416665 Z M9.64583326,26.583333 L22.0416665,26.583333 L22.0416665,28.3541664 L9.64583326,28.3541664 L9.64583326,26.583333 Z"/></symbol><symbol id="icon-springer-download" viewBox="-301 390 9 14"><path d="M-301 395.6l4.5 5.1 4.5-5.1h-3V390h-3v5.6h-3zm0 6.5h9v1.9h-9z"/></symbol><symbol id="icon-springer-info" viewBox="0 0 24 24"><!--Generator: Sketch 63.1 (92452) - https://sketch.com--><g id="V&I" stroke="none" stroke-width="1" fill-rule="evenodd"><g id="info" fill-rule="nonzero"><path d="M12,0 C18.627417,0 24,5.372583 24,12 C24,18.627417 18.627417,24 12,24 C5.372583,24 0,18.627417 0,12 C0,5.372583 5.372583,0 12,0 Z M12.5540543,9.1 L11.5540543,9.1 C11.0017696,9.1 10.5540543,9.54771525 10.5540543,10.1 L10.5540543,10.1 L10.5540543,18.1 C10.5540543,18.6522847 11.0017696,19.1 11.5540543,19.1 L11.5540543,19.1 L12.5540543,19.1 C13.1063391,19.1 13.5540543,18.6522847 13.5540543,18.1 L13.5540543,18.1 L13.5540543,10.1 C13.5540543,9.54771525 13.1063391,9.1 12.5540543,9.1 L12.5540543,9.1 Z M12,5 C11.5356863,5 11.1529412,5.14640523 10.8517647,5.43921569 C10.5505882,5.73202614 10.4,6.11546841 10.4,6.58954248 C10.4,7.06361656 10.5505882,7.45054466 10.8517647,7.7503268 C11.1529412,8.05010893 11.5356863,8.2 12,8.2 C12.4768627,8.2 12.8627451,8.05010893 13.1576471,7.7503268 C13.452549,7.45054466 13.6,7.06361656 13.6,6.58954248 C13.6,6.11546841 13.452549,5.73202614 13.1576471,5.43921569 C12.8627451,5.14640523 12.4768627,5 12,5 Z" id="Combined-Shape"/></g></g></symbol><symbol id="icon-springer-tick-circle" viewBox="0 0 24 24"><g id="Page-1" stroke="none" stroke-width="1" fill-rule="evenodd"><g id="springer-tick-circle" fill-rule="nonzero"><path d="M12,24 C5.372583,24 0,18.627417 0,12 C0,5.372583 5.372583,0 12,0 C18.627417,0 24,5.372583 24,12 C24,18.627417 18.627417,24 12,24 Z M7.657,10.79 C7.45285634,10.6137568 7.18569967,10.5283283 6.91717333,10.5534259 C6.648647,10.5785236 6.40194824,10.7119794 6.234,10.923 C5.87705269,11.3666969 5.93445559,12.0131419 6.364,12.387 L10.261,15.754 C10.6765468,16.112859 11.3037113,16.0695601 11.666,15.657 L17.759,8.713 C18.120307,8.27302248 18.0695334,7.62621189 17.644,7.248 C17.4414817,7.06995024 17.1751516,6.9821166 16.9064461,7.00476032 C16.6377406,7.02740404 16.3898655,7.15856958 16.22,7.368 L10.768,13.489 L7.657,10.79 Z" id="path-1"/></g></g></symbol><symbol id="icon-updates" viewBox="0 0 18 18"><g fill-rule="nonzero"><path d="M16.98 3.484h-.48c-2.52-.058-5.04 1.161-7.44 2.903-2.46-1.8-4.74-2.903-8.04-2.903-.3 0-.54.29-.54.58v9.813c0 .29.24.523.54.581 2.76.348 4.86 1.045 7.62 2.903.24.116.54.116.72 0 2.76-1.858 4.86-2.555 7.62-2.903.3-.058.54-.29.54-.58V4.064c0-.29-.24-.523-.54-.581zm-15.3 1.22c2.34 0 4.86 1.509 6.72 2.786v8.478c-2.34-1.394-4.38-2.09-6.72-2.439V4.703zm14.58 8.767c-2.34.348-4.38 1.045-6.72 2.439V7.374C12 5.632 14.1 4.645 16.26 4.645v8.826z"/><path d="M9 .058c-1.56 0-2.76 1.22-2.76 2.671C6.24 4.181 7.5 5.4 9 5.4c1.5 0 2.76-1.22 2.76-2.671 0-1.452-1.2-2.67-2.76-2.67zm0 4.413c-.96 0-1.8-.755-1.8-1.742C7.2 1.8 7.98.987 9 .987s1.8.755 1.8 1.742c0 .93-.84 1.742-1.8 1.742z"/></g></symbol><symbol id="icon-checklist-banner" viewBox="0 0 56.69 56.69"><path style="fill:none" d="M0 0h56.69v56.69H0z"/><clipPath id="b"><use xlink:href="#a" style="overflow:visible"/></clipPath><path d="M21.14 34.46c0-6.77 5.48-12.26 12.24-12.26s12.24 5.49 12.24 12.26-5.48 12.26-12.24 12.26c-6.76-.01-12.24-5.49-12.24-12.26zm19.33 10.66 10.23 9.22s1.21 1.09 2.3-.12l2.09-2.32s1.09-1.21-.12-2.3l-10.23-9.22m-19.29-5.92c0-4.38 3.55-7.94 7.93-7.94s7.93 3.55 7.93 7.94c0 4.38-3.55 7.94-7.93 7.94-4.38-.01-7.93-3.56-7.93-7.94zm17.58 12.99 4.14-4.81" style="clip-path:url(#b);fill:none;stroke:#01324b;stroke-width:2;stroke-linecap:round"/><path d="M8.26 9.75H28.6M8.26 15.98H28.6m-20.34 6.2h12.5m14.42-5.2V4.86s0-2.93-2.93-2.93H4.13s-2.93 0-2.93 2.93v37.57s0 2.93 2.93 2.93h15.01M8.26 9.75H28.6M8.26 15.98H28.6m-20.34 6.2h12.5" style="clip-path:url(#b);fill:none;stroke:#01324b;stroke-width:2;stroke-linecap:round;stroke-linejoin:round"/></symbol><symbol id="icon-submit-closed" viewBox="0 0 18 18"><path d="m15 0c1.1045695 0 2 .8954305 2 2v4.5c0 .27614237-.2238576.5-.5.5s-.5-.22385763-.5-.5v-4.5c0-.51283584-.3860402-.93550716-.8833789-.99327227l-.1166211-.00672773h-9v3c0 1.1045695-.8954305 2-2 2h-3v10c0 .5128358.38604019.9355072.88337887.9932723l.11662113.0067277h4.5c.27614237 0 .5.2238576.5.5s-.22385763.5-.5.5h-4.5c-1.1045695 0-2-.8954305-2-2v-10.17157288c0-.53043297.21071368-1.0391408.58578644-1.41421356l3.82842712-3.82842712c.37507276-.37507276.88378059-.58578644 1.41421356-.58578644zm-2.5 7c3.0375661 0 5.5 2.46243388 5.5 5.5 0 3.0375661-2.4624339 5.5-5.5 5.5-3.03756612 0-5.5-2.4624339-5.5-5.5 0-3.03756612 2.46243388-5.5 5.5-5.5zm0 1c-2.4852814 0-4.5 2.0147186-4.5 4.5s2.0147186 4.5 4.5 4.5 4.5-2.0147186 4.5-4.5-2.0147186-4.5-4.5-4.5zm2.3087379 2.1912621c.2550161.2550162.2550161.668479 0 .9234952l-1.3859024 1.3845831 1.3859024 1.3859023c.2550161.2550162.2550161.668479 0 .9234952-.2550162.2550161-.668479.2550161-.9234952 0l-1.3859023-1.3859024-1.3845831 1.3859024c-.2550162.2550161-.668479.2550161-.9234952 0-.25501614-.2550162-.25501614-.668479 0-.9234952l1.3845831-1.3859023-1.3845831-1.3845831c-.25501614-.2550162-.25501614-.668479 0-.9234952.2550162-.25501614.668479-.25501614.9234952 0l1.3845831 1.3845831 1.3859023-1.3845831c.2550162-.25501614.668479-.25501614.9234952 0zm-9.8087379-8.7782621-3.587 3.587h2.587c.55228475 0 1-.44771525 1-1z"/></symbol><symbol id="icon-submit-open" viewBox="0 0 18 18"><path d="m15 0c1.1045695 0 2 .8954305 2 2v5.5c0 .27614237-.2238576.5-.5.5s-.5-.22385763-.5-.5v-5.5c0-.51283584-.3860402-.93550716-.8833789-.99327227l-.1166211-.00672773h-9v3c0 1.1045695-.8954305 2-2 2h-3v10c0 .5128358.38604019.9355072.88337887.9932723l.11662113.0067277h7.5c.27614237 0 .5.2238576.5.5s-.22385763.5-.5.5h-7.5c-1.1045695 0-2-.8954305-2-2v-10.17157288c0-.53043297.21071368-1.0391408.58578644-1.41421356l3.82842712-3.82842712c.37507276-.37507276.88378059-.58578644 1.41421356-.58578644zm-.5442863 8.18867991 3.3545404 3.35454039c.2508994.2508994.2538696.6596433.0035959.909917-.2429543.2429542-.6561449.2462671-.9065387-.0089489l-2.2609825-2.3045251.0010427 7.2231989c0 .3569916-.2898381.6371378-.6473715.6371378-.3470771 0-.6473715-.2852563-.6473715-.6371378l-.0010428-7.2231995-2.2611222 2.3046654c-.2531661.2580415-.6562868.2592444-.9065605.0089707-.24295423-.2429542-.24865597-.6576651.0036132-.9099343l3.3546673-3.35466731c.2509089-.25090888.6612706-.25227691.9135302-.00001728zm-.9557137-3.18867991c.2761424 0 .5.22385763.5.5s-.2238576.5-.5.5h-6c-.27614237 0-.5-.22385763-.5-.5s.22385763-.5.5-.5zm-8.5-3.587-3.587 3.587h2.587c.55228475 0 1-.44771525 1-1zm8.5 1.587c.2761424 0 .5.22385763.5.5s-.2238576.5-.5.5h-6c-.27614237 0-.5-.22385763-.5-.5s.22385763-.5.5-.5z"/></symbol><symbol id="icon-submit-upcoming" viewBox="0 0 18 18"><path d="m15 0c1.1045695 0 2 .8954305 2 2v4.5c0 .27614237-.2238576.5-.5.5s-.5-.22385763-.5-.5v-4.5c0-.51283584-.3860402-.93550716-.8833789-.99327227l-.1166211-.00672773h-9v3c0 1.1045695-.8954305 2-2 2h-3v10c0 .5128358.38604019.9355072.88337887.9932723l.11662113.0067277h4.5c.27614237 0 .5.2238576.5.5s-.22385763.5-.5.5h-4.5c-1.1045695 0-2-.8954305-2-2v-10.17157288c0-.53043297.21071368-1.0391408.58578644-1.41421356l3.82842712-3.82842712c.37507276-.37507276.88378059-.58578644 1.41421356-.58578644zm-2.5 7c3.0375661 0 5.5 2.46243388 5.5 5.5 0 3.0375661-2.4624339 5.5-5.5 5.5-3.03756612 0-5.5-2.4624339-5.5-5.5 0-1.6607442.73606908-3.14957021 1.89976608-4.15803695l-1.51549374.02214397c-.27613212.00263356-.49998143-.22483432-.49998143-.49020681 0-.24299316.17766103-.44509007.40961587-.48700057l.08928713-.00797472h2.66407569c.2449213 0 .4486219.17766776.490865.40963137l.008038.08929051v2.6642143c0 .275547-.2296028.4989219-.4949753.4989219-.24299317 0-.44342617-.1744719-.4830969-.4093269l-.00710993-.0906783.01983146-1.46576707c-.96740882.82538117-1.58082193 2.05345007-1.58082193 3.42478927 0 2.4852814 2.0147186 4.5 4.5 4.5s4.5-2.0147186 4.5-4.5-2.0147186-4.5-4.5-4.5c-.7684937 0-.7684937-1 0-1zm0 2.85c.3263501 0 .5965265.2405082.6429523.5539478l.0070477.0960522v1.731l.8096194.8093806c.2284567.2284567.2513024.5846637.068537.8386705l-.068537.0805683c-.2284567.2284567-.5846637.2513024-.8386705.068537l-.0805683-.068537-.9707107-.9707107c-.1125218-.1125218-.1855975-.257116-.2103268-.412296l-.0093431-.1180341v-1.9585786c0-.3589851.2910149-.65.65-.65zm-7.5-8.437-3.587 3.587h2.587c.55228475 0 1-.44771525 1-1z"/></symbol><symbol id="icon-facebook-bordered" viewBox="463.812 263.868 32 32"><path d="M479.812,263.868c-8.837,0-16,7.163-16,16s7.163,16,16,16s16-7.163,16-16S488.649,263.868,479.812,263.868z M479.812,293.868c-7.732,0-14-6.269-14-14s6.268-14,14-14s14,6.269,14,14S487.545,293.868,479.812,293.868z"/><path d="M483.025,280.48l0.32-2.477h-2.453v-1.582c0-0.715,0.199-1.207,1.227-1.207h1.311v-2.213 c-0.227-0.029-1.003-0.098-1.907-0.098c-1.894,0-3.186,1.154-3.186,3.271v1.826h-2.142v2.477h2.142v6.354h2.557v-6.354 L483.025,280.48L483.025,280.48z"/></symbol><symbol id="icon-twitter-bordered" viewBox="463.812 263.868 32 32"><g><path d="M486.416,276.191c-0.483,0.215-1.007,0.357-1.554,0.429c0.558-0.338,0.991-0.868,1.19-1.502 c-0.521,0.308-1.104,0.536-1.72,0.657c-0.494-0.526-1.2-0.854-1.979-0.854c-1.496,0-2.711,1.213-2.711,2.71 c0,0.212,0.023,0.419,0.069,0.616c-2.252-0.111-4.25-1.19-5.586-2.831c-0.231,0.398-0.365,0.866-0.365,1.361 c0,0.94,0.479,1.772,1.204,2.257c-0.441-0.015-0.861-0.138-1.227-0.339v0.031c0,1.314,0.937,2.41,2.174,2.656 c-0.227,0.062-0.47,0.098-0.718,0.098c-0.171,0-0.343-0.018-0.511-0.049c0.35,1.074,1.347,1.859,2.531,1.883 c-0.928,0.726-2.095,1.16-3.366,1.16c-0.22,0-0.433-0.014-0.644-0.037c1.2,0.768,2.621,1.215,4.155,1.215 c4.983,0,7.71-4.129,7.71-7.711c0-0.115-0.004-0.232-0.006-0.351C485.592,277.212,486.054,276.734,486.416,276.191z"/></g><path d="M479.812,263.868c-8.837,0-16,7.163-16,16s7.163,16,16,16s16-7.163,16-16S488.649,263.868,479.812,263.868z M479.812,293.868c-7.732,0-14-6.269-14-14s6.268-14,14-14s14,6.269,14,14S487.545,293.868,479.812,293.868z"/></symbol><symbol id="icon-weibo-bordered" viewBox="463.812 263.868 32 32"><path d="M479.812,263.868c-8.838,0-16,7.163-16,16s7.162,16,16,16c8.837,0,16-7.163,16-16S488.649,263.868,479.812,263.868z M479.812,293.868c-7.732,0-14-6.269-14-14s6.268-14,14-14c7.731,0,14,6.269,14,14S487.545,293.868,479.812,293.868z"/><g><path d="M478.552,285.348c-2.616,0.261-4.876-0.926-5.044-2.649c-0.167-1.722,1.814-3.33,4.433-3.588 c2.609-0.263,4.871,0.926,5.041,2.647C483.147,283.479,481.164,285.089,478.552,285.348 M483.782,279.63 c-0.226-0.065-0.374-0.109-0.259-0.403c0.25-0.639,0.276-1.188,0.005-1.581c-0.515-0.734-1.915-0.693-3.521-0.021 c0,0-0.508,0.224-0.378-0.181c0.247-0.798,0.209-1.468-0.178-1.852c-0.87-0.878-3.194,0.032-5.183,2.027 c-1.489,1.494-2.357,3.082-2.357,4.453c0,2.619,3.354,4.213,6.631,4.213c4.297,0,7.154-2.504,7.154-4.493 C485.697,280.594,484.689,279.911,483.782,279.63"/><path d="M486.637,274.833c-1.039-1.154-2.57-1.592-3.982-1.291l0,0c-0.325,0.068-0.532,0.391-0.465,0.72 c0.068,0.328,0.391,0.537,0.72,0.466c1.005-0.215,2.092,0.104,2.827,0.92c0.736,0.818,0.938,1.939,0.625,2.918l0,0 c-0.102,0.318,0.068,0.661,0.39,0.762c0.32,0.104,0.658-0.069,0.763-0.391v-0.001C487.953,277.558,487.674,275.985,486.637,274.833 "/><path d="M485.041,276.276c-0.504-0.562-1.25-0.774-1.938-0.63c-0.279,0.06-0.461,0.339-0.396,0.621 c0.062,0.281,0.335,0.461,0.617,0.398l0,0c0.336-0.071,0.702,0.03,0.947,0.307s0.312,0.649,0.207,0.979l0,0 c-0.089,0.271,0.062,0.565,0.336,0.654c0.274,0.09,0.564-0.062,0.657-0.336C485.686,277.604,485.549,276.837,485.041,276.276"/><path d="M478.694,282.227c-0.09,0.156-0.293,0.233-0.451,0.166c-0.151-0.062-0.204-0.235-0.115-0.389 c0.093-0.155,0.284-0.229,0.44-0.168C478.725,281.892,478.782,282.071,478.694,282.227 M477.862,283.301 c-0.253,0.405-0.795,0.58-1.202,0.396c-0.403-0.186-0.521-0.655-0.27-1.051c0.248-0.39,0.771-0.566,1.176-0.393 C477.979,282.423,478.109,282.889,477.862,283.301 M478.812,280.437c-1.244-0.326-2.65,0.294-3.19,1.396 c-0.553,1.119-0.021,2.369,1.236,2.775c1.303,0.42,2.84-0.225,3.374-1.436C480.758,281.989,480.1,280.77,478.812,280.437"/></g></symbol></svg> </div> <div class="u-vh-full"> <a class="c-skip-link" href="#main-content">Skip to main content</a> <div class="u-hide u-show-following-ad"></div> <aside class="adsbox c-ad c-ad--728x90" data-component-mpu> <div class="c-ad__inner"> <p class="c-ad__label">Advertisement</p> <div id="div-gpt-ad-LB1" data-ad-type="LB1" data-test="LB1-ad" data-pa11y-ignore data-gpt data-gpt-unitpath="/270604982/bmc/trialsjournal/articles" data-gpt-sizes="728x90,970x90" data-gpt-targeting="pos=LB1;doi=10.1186/s13063-021-05489-x;type=article;kwrd=Clinical trials as topic; Machine learning,Artificial intelligence,Research design,Research ethics;pmc=H00007,B0000X,S17030;" > <noscript> <a href="//pubads.g.doubleclick.net/gampad/jump?iu=/270604982/bmc/trialsjournal/articles&sz=728x90,970x90&pos=LB1&doi=10.1186/s13063-021-05489-x&type=article&kwrd=Clinical trials as topic; Machine learning,Artificial intelligence,Research design,Research ethics&pmc=H00007,B0000X,S17030&"> <img data-test="gpt-advert-fallback-img" src="//pubads.g.doubleclick.net/gampad/ad?iu=/270604982/bmc/trialsjournal/articles&sz=728x90,970x90&pos=LB1&doi=10.1186/s13063-021-05489-x&type=article&kwrd=Clinical trials as topic; Machine learning,Artificial intelligence,Research design,Research ethics&pmc=H00007,B0000X,S17030&" alt="Advertisement" width="728" height="90"> </a> </noscript> </div> </div> </aside> <div id="membership-message-loader-desktop" class="placeholder" data-placeholder="/placeholder/v1/membership/message"></div> <div id="top" class="u-position-relative"> <header class="c-header" data-test="publisher-header"> <div class="c-header__container"> <div class="c-header__brand u-mr-48" itemscope itemtype="http://schema.org/Organization" data-test="navbar-logo-header"> <div class="c-logo"> <a href="https://www.biomedcentral.com" itemprop="url"> <img alt="BMC" itemprop="logo" width="76" height="18" role="img" src=/static/images/bmc/logos/logo-bmc-white-aj-be532aa3f0.svg> <div class="c-logo__strapline"> <img alt="Part of Springer Nature" width="173" height="16" role="img" src=/static/images/bmc/logos/logo-bmc-white-strapline-sn-f224388d67.svg> </div> </a> </div> </div> <div class="c-header__navigation"> <button type="button" class="c-header__link u-button-reset u-mr-24" data-expander data-expander-target="#publisher-header-search" data-expander-autofocus="firstTabbable" data-test="header-search-button" aria-controls="publisher-header-search" aria-expanded="false"> <span class="u-display-flex u-align-items-center"> <span>Search</span> <svg class="u-icon u-flex-static u-ml-8" aria-hidden="true" focusable="false"> <use xlink:href="#icon-search"></use> </svg> </span> </button> <nav> <ul class="c-header__menu" data-enhanced-menu data-test="publisher-navigation"> <li class="c-header__item u-hide-at-lt-lg"> <a class="c-header__link" href="//www.biomedcentral.com/journals"> Explore journals </a> </li> <li class="c-header__item u-hide-at-lt-lg"> <a class="c-header__link" href="//www.biomedcentral.com/getpublished"> Get published </a> </li> <li class="c-header__item u-hide-at-lt-lg"> <a class="c-header__link" href="//www.biomedcentral.com/about"> About BMC </a> </li> <li class="c-header__item"> <a data-enhanced-account class="c-header__link" href="https://www.biomedcentral.com/account" data-test="login-link"> My account </a> </li> </ul> </nav> </div> </div> </header> <div class="c-popup-search u-js-hide" id="publisher-header-search"> <div class="u-container"> <div class="c-popup-search__container"> <div class="ctx-search"> <form role="search" class="c-form-field js-skip-validation" method="GET" action="//www.biomedcentral.com/search" data-track="search" data-track-context="pop out website-wide search in bmc website header" data-track-category="Search and Results" data-track-action="Submit search" data-dynamic-track-label data-track-label="" data-test="global-search"> <label for="publisherSearch" class="c-form-field__label">Search all BMC articles</label> <div class="u-display-flex"> <input id="publisherSearch" class="c-form-field__input" data-search-input autocomplete="off" role="textbox" data-test="search-input" name="query" type="text" value=""/> <div> <button class="u-button" type="submit" data-test="search-submit-button"> <span class="u-visually-hidden">Search</span> <svg class="u-icon u-flex-static" width="16" height="16" aria-hidden="true" focusable="false"> <use xlink:href="#icon-search"></use> </svg> </button> </div> </div> <input type="hidden" name="searchType" value="publisherSearch"/> </form> </div> </div> </div> </div> </div> <header class="c-journal-header c-journal-header--trials ctx-journal-header"> <div class="u-container"> <div class="c-journal-header__inner "> <div class="c-journal-title" id="journalTitle"> <a href="/"> <img class="c-journal-title__logo" src=https://trialsjournal.biomedcentral.com/images/13063/logo.svg alt="Trials logo"> <span class="c-journal-title__text ">Trials</span> </a> </div> </div> </div> <div class="c-navbar c-navbar--with-submit-button"> <div class="c-navbar__container"> <div class="c-navbar__content"> <nav class="c-navbar__nav"> <ul class="c-navbar__nav c-navbar__nav--journal" role="menu" data-test="site-navigation"> <li class="c-navbar__item" role="menuitem"> <a class="c-navbar__link" data-track="click" data-track-category="Home" data-track-action="Clicked journal navigation link" href='/'>Home</a> </li> <li class="c-navbar__item" role="menuitem"> <a class="c-navbar__link" data-track="click" data-track-category="About" data-track-action="Clicked journal navigation link" href='/about'>About</a> </li> <li class="c-navbar__item" role="menuitem"> <a class="c-navbar__link c-navbar__link--is-shown" data-track="click" data-track-category="Articles" data-track-action="Clicked journal navigation link" href='/articles'>Articles</a> </li> <li class="c-navbar__item" role="menuitem"> <a class="c-navbar__link" data-track="click" data-track-category="In Review" data-track-action="Clicked journal navigation link" href='https://www.researchsquare.com/browse/journal/trials'>In Review</a> </li> <li class="c-navbar__item" role="menuitem"> <a class="c-navbar__link" data-track="click" data-track-category="Submission Guidelines" data-track-action="Clicked journal navigation link" href='/submission-guidelines'>Submission Guidelines</a> </li> <li class="c-navbar__item" role="menuitem"> <a class="c-navbar__link" data-track="click" data-track-category="News" data-track-action="Clicked journal navigation link" href='/news'>News</a> </li> <li class="c-navbar__item" role="menuitem"> <a class="c-navbar__link" data-track="click" data-track-category="COVID-19" data-track-action="Clicked journal navigation link" href='/covid-19'>COVID-19</a> </li> <li class="c-navbar__item" role="menuitem" data-test="journal-header-submit-button"> <div class=""> <a class="u-button u-button--tertiary u-button--alt-colour-on-mobile" href="https://www.editorialmanager.com/trls/" data-track="click_submit_manuscript" data-track-action="manuscript submission" data-track-category="article" data-track-label="button in journal nav" data-track-context="journal header on article page" data-track-external data-test="submit-manuscript-button">Submit manuscript<svg class="u-ml-8" width="15" height="16" aria-hidden="true" focusable="false"><use xlink:href="#icon-submit-open"></use></svg></a> </div> </li> </ul> </nav> </div> </div> </div> <div class="c-journal-header__identity c-journal-header__identity--sky-dark-blue"> </div> </header> <div class="u-container u-mt-32 u-mb-32 u-clearfix" id="main-content" data-component="article-container"> <main class="c-article-main-column u-float-left js-main-column" data-track-component="article body"> <div class="c-context-bar u-hide" data-test="context-bar" data-context-bar aria-hidden="true"> <div class="c-context-bar__container u-container" data-track-context="sticky banner"> <div class="c-context-bar__title"> The role of machine learning in clinical research: transforming the future of evidence generation </div> <div class="c-pdf-container"> <div class="c-pdf-download u-clear-both"> <a href="//trialsjournal.biomedcentral.com/counter/pdf/10.1186/s13063-021-05489-x.pdf" class="u-button u-button--full-width u-button--primary u-justify-content-space-between c-pdf-download__link" data-article-pdf="true" data-readcube-pdf-url="true" data-test="pdf-link" data-draft-ignore="true" data-track="content_download" data-track-type="article pdf download" data-track-action="download pdf" data-track-label="link" data-track-external download> <span class="c-pdf-download__text">Download PDF</span> <svg aria-hidden="true" focusable="false" width="16" height="16" class="u-icon"><use xlink:href="#icon-download"/></svg> </a> </div> </div> </div> </div> <div class="c-pdf-button__container u-hide-at-lg js-context-bar-sticky-point-mobile"> <div class="c-pdf-container" data-track-context="article body"> <div class="c-pdf-download u-clear-both"> <a href="//trialsjournal.biomedcentral.com/counter/pdf/10.1186/s13063-021-05489-x.pdf" class="u-button u-button--full-width u-button--primary u-justify-content-space-between c-pdf-download__link" data-article-pdf="true" data-readcube-pdf-url="true" data-test="pdf-link" data-draft-ignore="true" data-track="content_download" data-track-type="article pdf download" data-track-action="download pdf" data-track-label="link" data-track-external download> <span class="c-pdf-download__text">Download PDF</span> <svg aria-hidden="true" focusable="false" width="16" height="16" class="u-icon"><use xlink:href="#icon-download"/></svg> </a> </div> </div> </div> <article lang="en"> <div class="c-article-header"> <ul class="c-article-identifiers" data-test="article-identifier"> <li class="c-article-identifiers__item" data-test="article-category">Commentary</li> <li class="c-article-identifiers__item"> <a href="https://www.springernature.com/gp/open-research/about/the-fundamentals-of-open-access-and-open-research" data-track="click" data-track-action="open access" data-track-label="link" class="u-color-open-access" data-test="open-access">Open access</a> </li> <li class="c-article-identifiers__item">Published: <time datetime="2021-08-16">16 August 2021</time></li> </ul> <h1 class="c-article-title" data-test="article-title" data-article-title="">The role of machine learning in clinical research: transforming the future of evidence generation</h1> <ul class="c-article-author-list c-article-author-list--short" data-test="authors-list" data-component-authors-activator="authors-list"><li class="c-article-author-list__item"><a data-test="author-name" data-track="click" data-track-action="open author" data-track-label="link" href="#auth-E__Hope-Weissler-Aff1" data-author-popup="auth-E__Hope-Weissler-Aff1" data-author-search="Weissler, E. Hope" data-corresp-id="c1">E. Hope Weissler<svg width="16" height="16" focusable="false" role="img" aria-hidden="true" class="u-icon"><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#icon-eds-i-mail-medium"></use></svg></a><span class="u-js-hide"> <a class="js-orcid" href="http://orcid.org/0000-0002-8442-6150"><span class="u-visually-hidden">ORCID: </span>orcid.org/0000-0002-8442-6150</a></span><sup class="u-js-hide"><a href="#Aff1">1</a></sup>, </li><li class="c-article-author-list__item"><a data-test="author-name" data-track="click" data-track-action="open author" data-track-label="link" href="#auth-Tristan-Naumann-Aff2" data-author-popup="auth-Tristan-Naumann-Aff2" data-author-search="Naumann, Tristan">Tristan Naumann</a><sup class="u-js-hide"><a href="#Aff2">2</a></sup>, </li><li class="c-article-author-list__item c-article-author-list__item--hide-small-screen"><a data-test="author-name" data-track="click" data-track-action="open author" data-track-label="link" href="#auth-Tomas-Andersson-Aff3" data-author-popup="auth-Tomas-Andersson-Aff3" data-author-search="Andersson, Tomas">Tomas Andersson</a><sup class="u-js-hide"><a href="#Aff3">3</a></sup>, </li><li class="c-article-author-list__item c-article-author-list__item--hide-small-screen"><a data-test="author-name" data-track="click" data-track-action="open author" data-track-label="link" href="#auth-Rajesh-Ranganath-Aff4" data-author-popup="auth-Rajesh-Ranganath-Aff4" data-author-search="Ranganath, Rajesh">Rajesh Ranganath</a><sup class="u-js-hide"><a href="#Aff4">4</a></sup>, </li><li class="c-article-author-list__item c-article-author-list__item--hide-small-screen"><a data-test="author-name" data-track="click" data-track-action="open author" data-track-label="link" href="#auth-Olivier-Elemento-Aff5" data-author-popup="auth-Olivier-Elemento-Aff5" data-author-search="Elemento, Olivier">Olivier Elemento</a><sup class="u-js-hide"><a href="#Aff5">5</a></sup>, </li><li class="c-article-author-list__item c-article-author-list__item--hide-small-screen"><a data-test="author-name" data-track="click" data-track-action="open author" data-track-label="link" href="#auth-Yuan-Luo-Aff6" data-author-popup="auth-Yuan-Luo-Aff6" data-author-search="Luo, Yuan">Yuan Luo</a><sup class="u-js-hide"><a href="#Aff6">6</a></sup>, </li><li class="c-article-author-list__item c-article-author-list__item--hide-small-screen"><a data-test="author-name" data-track="click" data-track-action="open author" data-track-label="link" href="#auth-Daniel_F_-Freitag-Aff7" data-author-popup="auth-Daniel_F_-Freitag-Aff7" data-author-search="Freitag, Daniel F.">Daniel F. Freitag</a><sup class="u-js-hide"><a href="#Aff7">7</a></sup>, </li><li class="c-article-author-list__item c-article-author-list__item--hide-small-screen"><a data-test="author-name" data-track="click" data-track-action="open author" data-track-label="link" href="#auth-James-Benoit-Aff8" data-author-popup="auth-James-Benoit-Aff8" data-author-search="Benoit, James">James Benoit</a><sup class="u-js-hide"><a href="#Aff8">8</a></sup>, </li><li class="c-article-author-list__item c-article-author-list__item--hide-small-screen"><a data-test="author-name" data-track="click" data-track-action="open author" data-track-label="link" href="#auth-Michael_C_-Hughes-Aff9" data-author-popup="auth-Michael_C_-Hughes-Aff9" data-author-search="Hughes, Michael C.">Michael C. Hughes</a><sup class="u-js-hide"><a href="#Aff9">9</a></sup>, </li><li class="c-article-author-list__item c-article-author-list__item--hide-small-screen"><a data-test="author-name" data-track="click" data-track-action="open author" data-track-label="link" href="#auth-Faisal-Khan-Aff3" data-author-popup="auth-Faisal-Khan-Aff3" data-author-search="Khan, Faisal">Faisal Khan</a><sup class="u-js-hide"><a href="#Aff3">3</a></sup>, </li><li class="c-article-author-list__item c-article-author-list__item--hide-small-screen"><a data-test="author-name" data-track="click" data-track-action="open author" data-track-label="link" href="#auth-Paul-Slater-Aff10" data-author-popup="auth-Paul-Slater-Aff10" data-author-search="Slater, Paul">Paul Slater</a><sup class="u-js-hide"><a href="#Aff10">10</a></sup>, </li><li class="c-article-author-list__item c-article-author-list__item--hide-small-screen"><a data-test="author-name" data-track="click" data-track-action="open author" data-track-label="link" href="#auth-Khader-Shameer-Aff3" data-author-popup="auth-Khader-Shameer-Aff3" data-author-search="Shameer, Khader">Khader Shameer</a><sup class="u-js-hide"><a href="#Aff3">3</a></sup>, </li><li class="c-article-author-list__item c-article-author-list__item--hide-small-screen"><a data-test="author-name" data-track="click" data-track-action="open author" data-track-label="link" href="#auth-Matthew-Roe-Aff11" data-author-popup="auth-Matthew-Roe-Aff11" data-author-search="Roe, Matthew">Matthew Roe</a><sup class="u-js-hide"><a href="#Aff11">11</a></sup>, </li><li class="c-article-author-list__item c-article-author-list__item--hide-small-screen"><a data-test="author-name" data-track="click" data-track-action="open author" data-track-label="link" href="#auth-Emmette-Hutchison-Aff3" data-author-popup="auth-Emmette-Hutchison-Aff3" data-author-search="Hutchison, Emmette">Emmette Hutchison</a><sup class="u-js-hide"><a href="#Aff3">3</a></sup>, </li><li class="c-article-author-list__item c-article-author-list__item--hide-small-screen"><a data-test="author-name" data-track="click" data-track-action="open author" data-track-label="link" href="#auth-Scott_H_-Kollins-Aff1" data-author-popup="auth-Scott_H_-Kollins-Aff1" data-author-search="Kollins, Scott H.">Scott H. Kollins</a><sup class="u-js-hide"><a href="#Aff1">1</a></sup>, </li><li class="c-article-author-list__item c-article-author-list__item--hide-small-screen"><a data-test="author-name" data-track="click" data-track-action="open author" data-track-label="link" href="#auth-Uli-Broedl-Aff12" data-author-popup="auth-Uli-Broedl-Aff12" data-author-search="Broedl, Uli">Uli Broedl</a><sup class="u-js-hide"><a href="#Aff12">12</a></sup>, </li><li class="c-article-author-list__item c-article-author-list__item--hide-small-screen"><a data-test="author-name" data-track="click" data-track-action="open author" data-track-label="link" href="#auth-Zhaoling-Meng-Aff13" data-author-popup="auth-Zhaoling-Meng-Aff13" data-author-search="Meng, Zhaoling">Zhaoling Meng</a><sup class="u-js-hide"><a href="#Aff13">13</a></sup>, </li><li class="c-article-author-list__item c-article-author-list__item--hide-small-screen"><a data-test="author-name" data-track="click" data-track-action="open author" data-track-label="link" href="#auth-Jennifer_L_-Wong-Aff14" data-author-popup="auth-Jennifer_L_-Wong-Aff14" data-author-search="Wong, Jennifer L.">Jennifer L. Wong</a><sup class="u-js-hide"><a href="#Aff14">14</a></sup>, </li><li class="c-article-author-list__item c-article-author-list__item--hide-small-screen"><a data-test="author-name" data-track="click" data-track-action="open author" data-track-label="link" href="#auth-Lesley-Curtis-Aff1" data-author-popup="auth-Lesley-Curtis-Aff1" data-author-search="Curtis, Lesley">Lesley Curtis</a><sup class="u-js-hide"><a href="#Aff1">1</a></sup>, </li><li class="c-article-author-list__item c-article-author-list__item--hide-small-screen"><a data-test="author-name" data-track="click" data-track-action="open author" data-track-label="link" href="#auth-Erich-Huang-Aff1-Aff15" data-author-popup="auth-Erich-Huang-Aff1-Aff15" data-author-search="Huang, Erich">Erich Huang</a><sup class="u-js-hide"><a href="#Aff1">1</a>,<a href="#Aff15">15</a></sup> & </li><li class="c-article-author-list__show-more" aria-label="Show all 21 authors for this article" title="Show all 21 authors for this article">…</li><li class="c-article-author-list__item"><a data-test="author-name" data-track="click" data-track-action="open author" data-track-label="link" href="#auth-Marzyeh-Ghassemi-Aff16-Aff17-Aff18-Aff19" data-author-popup="auth-Marzyeh-Ghassemi-Aff16-Aff17-Aff18-Aff19" data-author-search="Ghassemi, Marzyeh">Marzyeh Ghassemi</a><sup class="u-js-hide"><a href="#Aff16">16</a>,<a href="#Aff17">17</a>,<a href="#Aff18">18</a>,<a href="#Aff19">19</a></sup> </li></ul><button aria-expanded="false" class="c-article-author-list__button"><svg width="16" height="16" focusable="false" role="img" aria-hidden="true" class="u-icon"><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#icon-eds-i-chevron-down-medium"></use></svg><span>Show authors</span></button> <p class="c-article-info-details" data-container-section="info"> <a data-test="journal-link" href="/" data-track="click" data-track-action="journal homepage" data-track-category="article body" data-track-label="link"><i data-test="journal-title">Trials</i></a> <b data-test="journal-volume"><span class="u-visually-hidden">volume</span> 22</b>, Article number: <span data-test="article-number">537</span> (<span data-test="article-publication-year">2021</span>) <a href="#citeas" class="c-article-info-details__cite-as u-hide-print" data-track="click" data-track-action="cite this article" data-track-label="link">Cite this article</a> </p> <div class="c-article-metrics-bar__wrapper u-clear-both"> <ul class="c-article-metrics-bar u-list-reset"> <li class=" c-article-metrics-bar__item" data-test="access-count"> <p class="c-article-metrics-bar__count">48k <span class="c-article-metrics-bar__label">Accesses</span></p> </li> <li class="c-article-metrics-bar__item" data-test="citation-count"> <p class="c-article-metrics-bar__count">105 <span class="c-article-metrics-bar__label">Citations</span></p> </li> <li class="c-article-metrics-bar__item" data-test="altmetric-score"> <p class="c-article-metrics-bar__count">69 <span class="c-article-metrics-bar__label">Altmetric</span></p> </li> <li class="c-article-metrics-bar__item"> <p class="c-article-metrics-bar__details"><a href="/articles/10.1186/s13063-021-05489-x/metrics" data-track="click" data-track-action="view metrics" data-track-label="link" rel="nofollow">Metrics <span class="u-visually-hidden">details</span></a></p> </li> </ul> </div> <div class="u-mb-8 c-status-message c-status-message--boxed c-status-message--info"> <span class="c-status-message__icon"> <svg class="u-icon" width="18" height="18" aria-hidden="true" focusable="false"> <use xlink:href="#icon-eds-i-info-filled-medium"></use> </svg> </span> <p class="u-mt-0">A <a href="/articles/10.1186/s13063-021-05571-4" class="relation-link" data-track="click" data-track-action="view linked article" data-track-label="link">Correction</a> to this article was published on 06 September 2021</p> </div> <div class="u-mb-8 c-status-message c-status-message--boxed c-status-message--info"> <span class="c-status-message__icon"> <svg class="u-icon" width="18" height="18" aria-hidden="true" focusable="false"> <use xlink:href="#icon-eds-i-info-filled-medium"></use> </svg> </span> <p class="u-mt-0">This article has been <a href="#change-history">updated</a></p> </div> </div> <section aria-labelledby="Abs1" data-title="Abstract" lang="en"><div class="c-article-section" id="Abs1-section"><h2 class="c-article-section__title js-section-title js-c-reading-companion-sections-item" id="Abs1">Abstract</h2><div class="c-article-section__content" id="Abs1-content"><h3 class="c-article__sub-heading" data-test="abstract-sub-heading">Background</h3><p>Interest in the application of machine learning (ML) to the design, conduct, and analysis of clinical trials has grown, but the evidence base for such applications has not been surveyed. This manuscript reviews the proceedings of a multi-stakeholder conference to discuss the current and future state of ML for clinical research. Key areas of clinical trial methodology in which ML holds particular promise and priority areas for further investigation are presented alongside a narrative review of evidence supporting the use of ML across the clinical trial spectrum.</p><h3 class="c-article__sub-heading" data-test="abstract-sub-heading">Results</h3><p>Conference attendees included stakeholders, such as biomedical and ML researchers, representatives from the US Food and Drug Administration (FDA), artificial intelligence technology and data analytics companies, non-profit organizations, patient advocacy groups, and pharmaceutical companies. ML contributions to clinical research were highlighted in the pre-trial phase, cohort selection and participant management, and data collection and analysis. A particular focus was paid to the operational and philosophical barriers to ML in clinical research. Peer-reviewed evidence was noted to be lacking in several areas.</p><h3 class="c-article__sub-heading" data-test="abstract-sub-heading">Conclusions</h3><p>ML holds great promise for improving the efficiency and quality of clinical research, but substantial barriers remain, the surmounting of which will require addressing significant gaps in evidence.</p></div></div></section> <p class="c-status-message c-status-message--info c-status-message--boxed u-mb-32"> <svg class="c-status-message__icon" width="24" height="24" aria-hidden="true" focusable="false"><use xlink:href="#icon-eds-i-info-filled-medium"></use></svg> <a href="/articles/10.1186/s13063-021-05489-x/peer-review" data-track="click" data-track-category="article body" data-track-action="open peer review reports" data-track-label="10.1186/s13063-021-05489-x">Peer Review reports</a> </p> <section data-title="Background"><div class="c-article-section" id="Sec1-section"><h2 class="c-article-section__title js-section-title js-c-reading-companion-sections-item" id="Sec1">Background</h2><div class="c-article-section__content" id="Sec1-content"><p>Interest in machine learning (ML) for healthcare has increased rapidly over the last 10 years. Though the academic discipline of ML has existed since the mid-twentieth century, improved computing resources, data availability, novel methods, and increasingly diverse technical talent have accelerated the application of ML to healthcare. Much of this attention has focused on applications of ML in healthcare <i>delivery</i>; however, applications of ML that facilitate clinical <i>research</i> are less frequently discussed in the academic and lay press (Fig. <a data-track="click" data-track-label="link" data-track-action="figure anchor" href="/articles/10.1186/s13063-021-05489-x#Fig1">1</a>). Clinical research is a wide-ranging field, with preclinical investigation and observational analyses leading to traditional trials and trials with pragmatic elements, which in turn spur clinical registries and further implementation work. While indispensable to improving healthcare and outcomes, clinical research as currently conducted is complex, labor intensive, expensive, and may be prone to unexpected errors and biases that can, at times, threaten its successful application, implementation, and acceptance. </p><div class="c-article-section__figure js-c-reading-companion-figures-item" data-test="figure" data-container-section="figure" id="figure-1" data-title="Fig. 1"><figure><figcaption><b id="Fig1" class="c-article-section__figure-caption" data-test="figure-caption-text">Fig. 1</b></figcaption><div class="c-article-section__figure-content"><div class="c-article-section__figure-item"><a class="c-article-section__figure-link" data-test="img-link" data-track="click" data-track-label="image" data-track-action="view figure" href="/articles/10.1186/s13063-021-05489-x/figures/1" rel="nofollow"><picture><source type="image/webp" srcset="//media.springernature.com/lw685/springer-static/image/art%3A10.1186%2Fs13063-021-05489-x/MediaObjects/13063_2021_5489_Fig1_HTML.png?as=webp"><img aria-describedby="Fig1" src="//media.springernature.com/lw685/springer-static/image/art%3A10.1186%2Fs13063-021-05489-x/MediaObjects/13063_2021_5489_Fig1_HTML.png" alt="figure 1" loading="lazy" width="685" height="384"></picture></a></div><div class="c-article-section__figure-description" data-test="bottom-caption" id="figure-1-desc"><p> The number of clinical practice–related publications was determined by searching “(“machine learning” or “artificial intelligence”) and (“healthcare”).” The number of healthcare-related publications was determined by searching “(“machine learning” or “artificial intelligence”) and (“healthcare”)”, and the number of clinical research–related publications was determined by searching “(“machine learning” or “artificial intelligence”) and (“clinical research”).”</p></div></div><div class="u-text-right u-hide-print"><a class="c-article__pill-button" data-test="article-link" data-track="click" data-track-label="button" data-track-action="view figure" href="/articles/10.1186/s13063-021-05489-x/figures/1" data-track-dest="link:Figure1 Full size image" aria-label="Full size image figure 1" rel="nofollow"><span>Full size image</span><svg width="16" height="16" focusable="false" role="img" aria-hidden="true" class="u-icon"><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#icon-eds-i-chevron-right-small"></use></svg></a></div></figure></div><p>Machine learning has the potential to help improve the success, generalizability, patient-centeredness, and efficiency of clinical trials. Various ML approaches are available for managing large and heterogeneous sources of data, identifying intricate and occult patterns, and predicting complex outcomes. As a result, ML has value to add across the spectrum of clinical trials, from preclinical drug discovery to pre-trial planning through study execution to data management and analysis (Fig. <a data-track="click" data-track-label="link" data-track-action="figure anchor" href="/articles/10.1186/s13063-021-05489-x#Fig2">2</a>). Despite the relative lack of academic and lay publications focused on ML-enabled clinical research (vìs-a-vìs the attention to ML in care delivery), the profusion of established and start-up companies devoting significant resources to the area indicates a high level of interest in, and burgeoning attempts to make use of, ML application to clinical research, and specifically clinical trials. </p><div class="c-article-section__figure js-c-reading-companion-figures-item" data-test="figure" data-container-section="figure" id="figure-2" data-title="Fig. 2"><figure><figcaption><b id="Fig2" class="c-article-section__figure-caption" data-test="figure-caption-text">Fig. 2</b></figcaption><div class="c-article-section__figure-content"><div class="c-article-section__figure-item"><a class="c-article-section__figure-link" data-test="img-link" data-track="click" data-track-label="image" data-track-action="view figure" href="/articles/10.1186/s13063-021-05489-x/figures/2" rel="nofollow"><picture><source type="image/webp" srcset="//media.springernature.com/lw685/springer-static/image/art%3A10.1186%2Fs13063-021-05489-x/MediaObjects/13063_2021_5489_Fig2_HTML.png?as=webp"><img aria-describedby="Fig2" src="//media.springernature.com/lw685/springer-static/image/art%3A10.1186%2Fs13063-021-05489-x/MediaObjects/13063_2021_5489_Fig2_HTML.png" alt="figure 2" loading="lazy" width="685" height="818"></picture></a></div><div class="c-article-section__figure-description" data-test="bottom-caption" id="figure-2-desc"><p>Areas of machine learning contribution to clinical research. Machine learning has the potential to contribute to clinical research through increasing the power and efficiency of pre-trial basic/translational research and enhancing the planning, conduct, and analysis of clinical trials</p></div></div><div class="u-text-right u-hide-print"><a class="c-article__pill-button" data-test="article-link" data-track="click" data-track-label="button" data-track-action="view figure" href="/articles/10.1186/s13063-021-05489-x/figures/2" data-track-dest="link:Figure2 Full size image" aria-label="Full size image figure 2" rel="nofollow"><span>Full size image</span><svg width="16" height="16" focusable="false" role="img" aria-hidden="true" class="u-icon"><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#icon-eds-i-chevron-right-small"></use></svg></a></div></figure></div><p>Key ML terms and principles may be found in Table <a data-track="click" data-track-label="link" data-track-action="table anchor" href="/articles/10.1186/s13063-021-05489-x#Tab1">1</a>. Many of the ML applications discussed in this article rely on deep neural networks, a subtype of ML in which interactions between multiple (sometimes many) hidden layers of the mathematical model enable complex, high-dimensional tasks, such as natural language processing, optical character recognition, and unsupervised learning. In January 2020, a diverse group of stakeholders, including leading biomedical and ML researchers, along with representatives from the US Food and Drug Administration (FDA), artificial intelligence technology and data analytics companies, non-profit organizations, patient advocacy groups, and pharmaceutical companies convened in Washington, DC, to discuss the role of ML in clinical research. In the setting of relatively scarce published data about ML application to clinical research, the attendees at this meeting offered significant personal, institutional, corporate, and regulatory experience pertaining to ML for clinical research. Attendees gave presentations in their areas of expertise, and effort was made to invite talks covering the entire spectrum of clinical research with presenters from multiple stakeholder groups for each topic. Subjects about which presentations were elicited in advance were intentionally broad and included current and planned applications of ML to clinical research, guidelines for the successful integration of ML into clinical research, and approaches to overcoming the barriers to implementation. Regular discussion periods generated additional areas of interest and concern and were moderated jointly by experts in ML, clinical research, and patient care. During the discussion periods, attendees focused on current issues in ML, including data biases, logistics of prospective validation, and the ethical issues associated with machines making decisions in a research context. This article provides a summary of the conference proceedings, outlining ways in which ML is currently being used for various clinical research applications in addition to possible future opportunities. It was generated through a collaborative writing process in which drafts were iterated through continued debate about unresolved issues from the conference itself. For many of the topics covered, no consensus about best practices was reached, and a diversity of opinions is conveyed in those instances. This article also serves as a call for collaboration between clinical researchers, ML experts, and other stakeholders from academia and industry in order to overcome the significant remaining barriers to its use, helping ML in clinical research to best serve <i>all</i> stakeholders. </p><div class="c-article-table" data-test="inline-table" data-container-section="table" id="table-1"><figure><figcaption class="c-article-table__figcaption"><b id="Tab1" data-test="table-caption">Table 1 Key terms related to machine learning in clinical research</b></figcaption><div class="u-text-right u-hide-print"><a class="c-article__pill-button" data-test="table-link" data-track="click" data-track-action="view table" data-track-label="button" rel="nofollow" href="/articles/10.1186/s13063-021-05489-x/tables/1" aria-label="Full size table 1"><span>Full size table</span><svg width="16" height="16" focusable="false" role="img" aria-hidden="true" class="u-icon"><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#icon-eds-i-chevron-right-small"></use></svg></a></div></figure></div></div></div></section><section data-title="The role of ML in preclinical drug discovery and development research"><div class="c-article-section" id="Sec2-section"><h2 class="c-article-section__title js-section-title js-c-reading-companion-sections-item" id="Sec2">The role of ML in preclinical drug discovery and development research</h2><div class="c-article-section__content" id="Sec2-content"><p>Successful clinical trials require significant preclinical investigation and planning, during which promising candidate molecules and targets are identified and the investigational strategy to achieve regulatory approval is defined. Missteps in this phase can delay the identification of promising drugs or doom clinical trials to eventual failure. ML can help researchers leverage previous and ongoing research to decrease the inefficiencies of the preclinical process.</p><h3 class="c-article__sub-heading" id="Sec3">Drug target identification, candidate molecule generation, and mechanism elucidation</h3><p>ML can streamline the process and increase the success of drug target identification and candidate molecule generation through synthesis of massive amounts of existing research, elucidation of drug mechanisms, and predictive modeling of protein structures and future drug target interactions [<a data-track="click" data-track-action="reference anchor" data-track-label="link" data-test="citation-ref" aria-label="Reference 1" title="Senior AW, Evans R, Jumper J, Kirkpatrick J, Sifre L, Green T, et al. Improved protein structure prediction using potentials from deep learning. Nature. 2020;577(7792):706–10. 
 https://doi.org/10.1038/s41586-019-1923-7
 
 ." href="/articles/10.1186/s13063-021-05489-x#ref-CR1" id="ref-link-section-d45421490e1303">1</a>]. Fauqueur et al. demonstrated the ability to identify specific types of gene-disease relationships from large databases even when relevant data-points were sparse [<a data-track="click" data-track-action="reference anchor" data-track-label="link" data-test="citation-ref" aria-label="Reference 2" title="Fauqueur JTA, Togia T. Constructing large scale biomedical knowledge bases from scratch with rapid annotation of interpretable patterns. In: Proceedings of the 18th BioNLP Workshop and Shared Task; 2019. 
 https://doi.org/10.18653/v1/w19-5016
 
 ." href="/articles/10.1186/s13063-021-05489-x#ref-CR2" id="ref-link-section-d45421490e1306">2</a>], while Jia et al. were able to extract drug-gene-mutation interactions from the text of scientific manuscripts [<a data-track="click" data-track-action="reference anchor" data-track-label="link" data-test="citation-ref" aria-label="Reference 3" title="Jia R, Wong C, Poon H. Document-level N-ary relation extraction with multiscale representation learning. Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers); 2019; Minneapolis: Association for Computational Linguistics. 
 https://ui.adsabs.harvard.edu/abs/2019arXiv190402347J
 
 ." href="/articles/10.1186/s13063-021-05489-x#ref-CR3" id="ref-link-section-d45421490e1309">3</a>]. This work, along with other efforts to render extremely large amounts of biomedical data interpretable by humans [<a data-track="click" data-track-action="reference anchor" data-track-label="link" data-test="citation-ref" aria-label="Reference 4" title="Dezso Z, Ceccarelli M. Machine learning prediction of oncology drug targets based on protein and network properties. BMC Bioinformatics. 2020;21(1):104. 
 https://doi.org/10.1186/s12859-020-3442-9
 
 ." href="/articles/10.1186/s13063-021-05489-x#ref-CR4" id="ref-link-section-d45421490e1312">4</a>, <a data-track="click" data-track-action="reference anchor" data-track-label="link" data-test="citation-ref" aria-label="Reference 5" title="Bagherian M, Sabeti E, Wang K, Sartor MA, Nikolovska-Coleska Z, Najarian K. Machine learning approaches and databases for prediction of drug-target interaction: a survey paper. Brief Bioinform. 2021;22(1):247–69. 
 https://doi.org/10.1093/bib/bbz157
 
 ." href="/articles/10.1186/s13063-021-05489-x#ref-CR5" id="ref-link-section-d45421490e1315">5</a>], helps researchers leverage and avoid duplicating prior work in order to target more promising avenues for further investigation. Once promising areas of investigation have been identified, ML also has a role to play in the generation of possible candidate molecules, for instance through use of a gated graph neural network to optimize molecules within the constraints of a target biological system [<a data-track="click" data-track-action="reference anchor" data-track-label="link" data-test="citation-ref" aria-label="Reference 6" title="Liu QAM, Brockschmidt M, Gaunt AL. Constrained graph variational autoencoders for molecule design. NeurIPS 2018. 2018;arXiv:1805.09076:7806–15." href="/articles/10.1186/s13063-021-05489-x#ref-CR6" id="ref-link-section-d45421490e1319">6</a>]. In situations in which a drug candidate performs differently in vivo than expected, ML can synthesize and analyze enormous amounts of data to better elucidate the drug’s mechanism, as Madhukar et al. showed by applying a Bayesian ML approach to an anti-cancer compound [<a data-track="click" data-track-action="reference anchor" data-track-label="link" data-test="citation-ref" aria-label="Reference 7" title="Madhukar NS, Khade PK, Huang L, Gayvert K, Galletti G, Stogniew M, et al. A Bayesian machine learning approach for drug target identification using diverse data types. Nat Commun. 2019;10(1):5221. 
 https://doi.org/10.1038/s41467-019-12928-6
 
 ." href="/articles/10.1186/s13063-021-05489-x#ref-CR7" id="ref-link-section-d45421490e1322">7</a>]. This type of work helps increase the chance that drugs are tested in populations most likely to benefit from them. In the case of the drug evaluated by Madhukar et al., a better understanding of its mechanism facilitated new clinical trials in a cancer type (pheochromocytoma) more likely to respond to the drug (rather than prostate and endometrial cancers, among others).</p><p>Interpretation of large amounts of highly dimensional data generated during in vitro translational research (including benchtop biological, chemical, and biochemical investigation) informs the choice of certain next steps over others, but this process of interpretation and integration is complex and prone to bias and error. Aspuru-Guzik has led several successful efforts to use experimental output as input for autonomous ML-powered laboratories, integrating ML into the planning, interpretation, and synthesis phases of drug development [<a data-track="click" data-track-action="reference anchor" data-track-label="link" data-test="citation-ref" aria-label="Reference 8" title="Langner S, Hase F, Perea JD, Stubhan T, Hauch J, Roch LM, et al. Beyond ternary OPV: high-throughput experimentation and self-driving laboratories optimize multicomponent systems. Adv Mater. 2020;32(14):e1907801. 
 https://doi.org/10.1002/adma.201907801
 
 ." href="/articles/10.1186/s13063-021-05489-x#ref-CR8" id="ref-link-section-d45421490e1328">8</a>, <a data-track="click" data-track-action="reference anchor" data-track-label="link" data-test="citation-ref" aria-label="Reference 9" title="Granda JM, Donina L, Dragone V, Long DL, Cronin L. Controlling an organic synthesis robot with machine learning to search for new reactivity. Nature. 2018;559(7714):377–81. 
 https://doi.org/10.1038/s41586-018-0307-8
 
 ." href="/articles/10.1186/s13063-021-05489-x#ref-CR9" id="ref-link-section-d45421490e1331">9</a>]. More recently, products of ML-enabled drug development have approached human testing. For example, an obsessive-compulsive personality disorder drug purportedly developed using AI-based methods is scheduled to begin phase I trials this year. The lay press reports that the drug was selected from among only 250 candidates and developed in only 12 months compared with the 2000+ candidates and nearly five years of development more typically required [<a data-track="click" data-track-action="reference anchor" data-track-label="link" data-test="citation-ref" aria-label="Reference 10" title="Koh D. Sumitomo Dainippon Pharma and Exscientia achieve breakthrough in AI drug discovery: Healthcare IT News - Portland, ME: Healthcare IT News; 2020." href="/articles/10.1186/s13063-021-05489-x#ref-CR10" id="ref-link-section-d45421490e1334">10</a>]. However, due to the lack of peer-reviewed publications about the development of this drug, the details of its development cannot be confirmed or leveraged for future work.</p><h3 class="c-article__sub-heading" id="Sec4">Clinical study protocol optimization</h3><p>As therapeutic compounds approach human trials, ML has a role to play in maximizing the success and efficiency of trials during the planning phase through application of simulation techniques to large amounts of data from prior trials in order to facilitate trial protocol development. For instance, study simulation may optimize the choice of treatment regimens for testing, as shown in a reinforcement learning approaches to Alzheimer’s disease and to non-small cell lung cancer [<a data-track="click" data-track-action="reference anchor" data-track-label="link" data-test="citation-ref" aria-label="Reference 11" title="Romero K, Ito K, Rogers JA, Polhamus D, Qiu R, Stephenson D, et al. The future is now: model-based clinical trial design for Alzheimer's disease. Clin Pharmacol Ther. 2015;97(3):210–4. 
 https://doi.org/10.1002/cpt.16
 
 ." href="/articles/10.1186/s13063-021-05489-x#ref-CR11" id="ref-link-section-d45421490e1345">11</a>, <a data-track="click" data-track-action="reference anchor" data-track-label="link" data-test="citation-ref" aria-label="Reference 12" title="Zhao Y, Zeng D, Socinski MA, Kosorok MR. Reinforcement learning strategies for clinical trials in nonsmall cell lung cancer. Biometrics. 2011;67(4):1422–33. 
 https://doi.org/10.1111/j.1541-0420.2011.01572.x
 
 ." href="/articles/10.1186/s13063-021-05489-x#ref-CR12" id="ref-link-section-d45421490e1348">12</a>]. A start-up company called Trials.AI allows investigators to upload protocols and uses natural language processing to identify potential pitfalls and barriers to successful trial completion (such as inclusion/exclusion criteria or outcome measures) [<a data-track="click" data-track-action="reference anchor" data-track-label="link" data-test="citation-ref" aria-label="Reference 13" title="trials.ai 2019 [cited 2021 February 2]. Available from: 
 trials.ai
 
 ." href="/articles/10.1186/s13063-021-05489-x#ref-CR13" id="ref-link-section-d45421490e1351">13</a>]. Unfortunately, performance of these example models has not been evaluated in a peer-reviewed manner, and they therefore offer only conceptual promise that ML in research planning can help ensure that a given trial design is optimally suited to the stakeholders’ needs.</p><p>In summary, there are clear opportunities to use ML to improve the efficiency and yield of preclinical investigation and clinical trial planning. However, most peer-reviewed reports of ML use in this capacity focus on preclinical research and development rather than clinical trial planning. This may be due to the greater availability of suitable large, highly dimensional datasets in translational settings in addition to greater potential costs, risks, and regulatory hurdles associated with ML use in clinical trial settings. Peer-reviewed evidence of ML application to clinical trial planning is needed in order to overcome these hurdles.</p></div></div></section><section data-title="The role of ML in clinical trial participant management"><div class="c-article-section" id="Sec5-section"><h2 class="c-article-section__title js-section-title js-c-reading-companion-sections-item" id="Sec5">The role of ML in clinical trial participant management</h2><div class="c-article-section__content" id="Sec5-content"><p>Clinical trial participant management includes the selection of target patient populations, patient recruiting, and participant retention. Unfortunately, despite significant resources generally being devoted to participant management, including time, planning, and trial coordinator effort, patient drop-out and non-adherence often cause studies to exceed allowable time or cost or fail to produce useable data. In fact, it has been estimated that between 33.6 and 52.4% of phase 1–3 clinical trials that support drug development fail to proceed to the next trial phase, leading to a 13.8% overall chance that a drug tested in phase I reaches approval [<a data-track="click" data-track-action="reference anchor" data-track-label="link" data-test="citation-ref" aria-label="Reference 14" title="Wong CH, Siah KW, Lo AW. Estimation of clinical trial success rates and related parameters. Biostatistics. 2019;20(2):273–86. 
 https://doi.org/10.1093/biostatistics/kxx069
 
 ." href="/articles/10.1186/s13063-021-05489-x#ref-CR14" id="ref-link-section-d45421490e1366">14</a>]. ML approaches can facilitate more efficient and fair participant identification, recruitment, and retention.</p><h3 class="c-article__sub-heading" id="Sec6">Selection of patient populations for investigation</h3><p>Improved selection of specific patient populations for trials may decrease the sample size required to observe a significant effect. Put another way, improvements to patient population selection may decrease the number of patients exposed to interventions from which they are unlikely to derive benefit. This area remains challenging as prior work has discovered that for every 1 intended response, there are 3 to 24 non-responders for the top medications, resulting in a large number of patients who receive harmful side effects over the intended effect [<a data-track="click" data-track-action="reference anchor" data-track-label="link" data-test="citation-ref" aria-label="Reference 15" title="Schork NJ. Personalized medicine: time for one-person trials. Nature. 2015;520(7549):609–11. 
 https://doi.org/10.1038/520609a
 
 ." href="/articles/10.1186/s13063-021-05489-x#ref-CR15" id="ref-link-section-d45421490e1376">15</a>]. In addition to facilitating patient population selection through the rapid analysis of large databases of prior research (as discussed above), unsupervised ML of patient populations can identify patterns in patient features that can be used to select patient phenotypes that are most likely to benefit from the proposed drug or intervention [<a data-track="click" data-track-action="reference anchor" data-track-label="link" data-test="citation-ref" aria-label="Reference 16" title="Glicksberg BS, Miotto R, Johnson KW, Shameer K, Li L, Chen R, et al. Automated disease cohort selection using word embeddings from electronic health records. Pac Symp Biocomput. 2018;23:145–56." href="/articles/10.1186/s13063-021-05489-x#ref-CR16" id="ref-link-section-d45421490e1379">16</a>]. Unstructured data is critical to phenotyping and identifying representative cohorts, indicating that considering additional data for patients is a crucial step toward identifying robust, representative cohorts [<a data-track="click" data-track-action="reference anchor" data-track-label="link" data-test="citation-ref" aria-label="Reference 17" title="Liao KP, Cai T, Savova GK, Murphy SN, Karlson EW, Ananthakrishnan AN, et al. Development of phenotype algorithms using electronic medical records and incorporating natural language processing. BMJ. 2015;350(apr24 11):h1885. 
 https://doi.org/10.1136/bmj.h1885
 
 ." href="/articles/10.1186/s13063-021-05489-x#ref-CR17" id="ref-link-section-d45421490e1382">17</a>]. For example, unsupervised learning of electronic health record (EHR) and genetic data from 11,210 patients elucidated three different subtypes of diabetes mellitus type II with distinct phenotypic expressions, each of which may have a different need for and response to a candidate therapy [<a data-track="click" data-track-action="reference anchor" data-track-label="link" data-test="citation-ref" aria-label="Reference 18" title="Li L, Cheng WY, Glicksberg BS, Gottesman O, Tamler R, Chen R, et al. Identification of type 2 diabetes subgroups through topological analysis of patient similarity. Sci Transl Med. 2015;7(311):311ra174." href="/articles/10.1186/s13063-021-05489-x#ref-CR18" id="ref-link-section-d45421490e1385">18</a>]. Bullfrog AI is a start-up that has sought to capitalize on the promise of targeted patient population selection, analyzing clinical trial data sets “to predict which patients will respond to a particular therapy in development, thereby improving inclusion/exclusion criteria and ensuring primary study outcomes are achieved” [<a data-track="click" data-track-action="reference anchor" data-track-label="link" data-test="citation-ref" aria-label="Reference 19" title="Our Solution 2021 [cited 2021 February 2]. Available from: 
 https://www.bullfrogai.com/our-solution/
 
 ." href="/articles/10.1186/s13063-021-05489-x#ref-CR19" id="ref-link-section-d45421490e1388">19</a>]. Though appealing in principle, this unsupported claim conflates outcome prediction (which is unlikely to succeed and runs counter to the intent of clinical research) with cohort selection (which would ideally identify patients on the basis of therapeutically relevant subtypes). Successfully identifying more selective patient populations does carry potential pitfalls: first, trials may be less likely to generate important negative data about subgroups that <i>would not</i> benefit from the intervention; and second, trials may miss subgroups who <i>would</i> have benefitted from the intervention, but whom the ML model missed. These potential pitfalls may be more likely to affect rural, remote, or underserved patient subgroups with more limited healthcare interactions. These two pitfalls carry possible implications for drug/device development regulatory approval and commercialization, as pivotal trials in more highly selected, and less representative, patient subgroups may require balancing the benefits of greater trial success with the drawbacks of more limited indications for drug/device use.</p><h3 class="c-article__sub-heading" id="Sec7">Participant identification and recruitment</h3><p>Once the specific cohort has been selected, natural language processing (NLP) has shown promise in identification of patients matching the desired phenotype, which is otherwise a labor-intensive process. For instance, a cross-modal inference learning model algorithm jointly encodes enrollment criteria (text) and patient records (tabular data) into a shared latent space, matching patients to trials using EHR data in a significantly more efficient manner than other machine learning approaches [<a data-track="click" data-track-action="reference anchor" data-track-label="link" data-test="citation-ref" aria-label="Reference 20" title="Zhang X, Xiao C, Glass LM, Sun J. DeepEnroll: patient-trial matching with deep embedding and entailment prediction. In: Proceedings of the Web Conference 2020. Taipei: Association for Computing Machinery; 2020. p. 1029–37." href="/articles/10.1186/s13063-021-05489-x#ref-CR20" id="ref-link-section-d45421490e1406">20</a>]. Some commercial entities offer similar services, including Mendel.AI and Deep6AI, though peer-reviewed evidence of their development and performance metrics is unavailable, raising questions about how these approaches perform [<a data-track="click" data-track-action="reference anchor" data-track-label="link" data-test="citation-ref" aria-label="Reference 21" title="Calaprice-Whitty D, Galil K, Salloum W, Zariv A, Jimenez B. Improving clinical trial participant prescreening with artificial intelligence (AI): a comparison of the results of AI-assisted vs standard methods in 3 oncology trials. Ther Innov Regul Sci. 2020;54(1):69–74. 
 https://doi.org/10.1007/s43441-019-00030-4
 
 ." href="/articles/10.1186/s13063-021-05489-x#ref-CR21" id="ref-link-section-d45421490e1409">21</a>, <a data-track="click" data-track-action="reference anchor" data-track-label="link" data-test="citation-ref" aria-label="Reference 22" title="How it works 2019 [cited 2021 February 2]. Available from: 
 https://deep6.ai/how-it-works/
 
 ." href="/articles/10.1186/s13063-021-05489-x#ref-CR22" id="ref-link-section-d45421490e1412">22</a>]. A potential opportunity of this approach is that it allows trialists to avoid relying on the completeness of structured data fields for participant identification, which has been shown to significantly bias trial cohorts [<a data-track="click" data-track-action="reference anchor" data-track-label="link" data-test="citation-ref" aria-label="Reference 23" title="Vassy JL, Ho YL, Honerlaw J, Cho K, Gaziano JM, Wilson PWF, et al. Yield and bias in defining a cohort study baseline from electronic health record data. J Biomed Inform. 2018;78:54–9. 
 https://doi.org/10.1016/j.jbi.2017.12.017
 
 ." href="/articles/10.1186/s13063-021-05489-x#ref-CR23" id="ref-link-section-d45421490e1415">23</a>, <a data-track="click" data-track-action="reference anchor" data-track-label="link" data-test="citation-ref" aria-label="Reference 24" title="Weber GM, Adams WG, Bernstam EV, Bickel JP, Fox KP, Marsolo K, et al. Biases introduced by filtering electronic health records for patients with “complete data”. J Am Med Inform Assoc. 2017;24(6):1134–41. 
 https://doi.org/10.1093/jamia/ocx071
 
 ." href="/articles/10.1186/s13063-021-05489-x#ref-CR24" id="ref-link-section-d45421490e1418">24</a>]. Unfortunately, to the extent that novel ML approaches to patient identification rely on EHRs, biases in the EHR data may affect the algorithms’ performances, leading to replacement of one source of bias (underlying the completeness of structured data) with another (underlying the generation of EHR documentation).</p><h3 class="c-article__sub-heading" id="Sec8">Participant retention, monitoring, and protocol adherence</h3><p>Two broad approaches are available to improve participant retention and protocol adherence using ML-assisted methods. The first is to use ML to collect and analyze large amounts of data to identify and intervene upon participants at high risk of study non-compliance. The second approach is to use ML to decrease participant study burden and thereby improve participants’ experiences.</p><p>AiCure is a commercial entity focused on protocol adherence using facial recognition technology to ensure patients take the assigned medication. AiCure was demonstrated to be more effective than a modified directly observed therapy strategy at detecting and improving patient adherence in both a schizophrenia trial and an anticoagulation trial among patients with a history of recent stroke [<a data-track="click" data-track-action="reference anchor" data-track-label="link" data-test="citation-ref" aria-label="Reference 25" title="Bain EE, Shafner L, Walling DP, Othman AA, Chuang-Stein C, Hinkle J, et al. Use of a novel artificial intelligence platform on mobile devices to assess dosing compliance in a phase 2 clinical trial in subjects with schizophrenia. JMIR Mhealth Uhealth. 2017;5(2):e18. 
 https://doi.org/10.2196/mhealth.7030
 
 ." href="/articles/10.1186/s13063-021-05489-x#ref-CR25" id="ref-link-section-d45421490e1432">25</a>, <a data-track="click" data-track-action="reference anchor" data-track-label="link" data-test="citation-ref" aria-label="Reference 26" title="Labovitz DL, Shafner L, Reyes Gil M, Virmani D, Hanina A. Using artificial intelligence to reduce the risk of nonadherence in patients on anticoagulation therapy. Stroke. 2017;48(5):1416–9. 
 https://doi.org/10.1161/STROKEAHA.116.016281
 
 ." href="/articles/10.1186/s13063-021-05489-x#ref-CR26" id="ref-link-section-d45421490e1435">26</a>]. Unfortunately, AiCure’s model development and validation process has not been published, heightening concerns that it may perform differently in different patient subgroups, as has been demonstrated in other areas of computer vision [<a data-track="click" data-track-action="reference anchor" data-track-label="link" data-test="citation-ref" aria-label="Reference 27" title="Adamson AS, Smith A. Machine learning and health care disparities in dermatology. JAMA Dermatol. 2018;154(11):1247–8. 
 https://doi.org/10.1001/jamadermatol.2018.2348
 
 ." href="/articles/10.1186/s13063-021-05489-x#ref-CR27" id="ref-link-section-d45421490e1438">27</a>]. Furthermore, these approaches, though promising, may encounter a potential barrier to implementation because their perceived invasiveness of privacy may not be acceptable to all research participants and because selecting patients with access to and comfort with the necessary devices and technology may introduce bias.</p><p>The other approach to improving participant retention uses ML to reduce the trial burden for participants using passive data collection techniques (methods will be discussed further in the “Data collection and management” section) and by extracting more information from available data generated during clinical practice and/or by study activities. Information created during routine clinical care can be processed using ML methods to yield data for investigational purposes. For instance, generative adversarial network modeling of slides stained with hematoxylin and eosin in the standard clinical fashion can detect which patients require more intensive and expensive multiplexed imaging, rather than subjecting all participants to that added burden [<a data-track="click" data-track-action="reference anchor" data-track-label="link" data-test="citation-ref" aria-label="Reference 28" title="Burlingame EA, Margolin AA, Gray JW, Chang YH. SHIFT: speedy histopathological-to-immunofluorescent translation of whole slide images using conditional generative adversarial networks. Proc SPIE Int Soc Opt Eng. 2018;10581. 
 https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6166432/
 
 ." href="/articles/10.1186/s13063-021-05489-x#ref-CR28" id="ref-link-section-d45421490e1444">28</a>]. NLP can also facilitate repurposing of clinical documentation for study use, such as auto-populating study case report forms, often through reliance on the Unified Medical Language System [<a data-track="click" data-track-action="reference anchor" data-track-label="link" data-test="citation-ref" aria-label="Reference 29" title="Han J, Chen K, Fang L, Zhang S, Wang F, Ma H, et al. Improving the efficacy of the data entry process for clinical research with a natural language processing-driven medical information extraction system: quantitative field research. JMIR Med Inform. 2019;7(3):e13331. 
 https://doi.org/10.2196/13331
 
 ." href="/articles/10.1186/s13063-021-05489-x#ref-CR29" id="ref-link-section-d45421490e1447">29</a>, <a data-track="click" data-track-action="reference anchor" data-track-label="link" data-test="citation-ref" aria-label="Reference 30" title="Fonferko-Shadrach B, Lacey AS, Roberts A, Akbari A, Thompson S, Ford DV, et al. Using natural language processing to extract structured epilepsy data from unstructured clinic letters: development and validation of the ExECT (extraction of epilepsy clinical text) system. BMJ Open. 2019;9(4):e023232. 
 https://doi.org/10.1136/bmjopen-2018-023232
 
 ." href="/articles/10.1186/s13063-021-05489-x#ref-CR30" id="ref-link-section-d45421490e1450">30</a>]. Patients also create valuable content outside of the clinical trial context that ML can process into study data to reduce the burden of data collection for trial participants, such as natural language processing of social media posts to identify serious drug reactions with high fidelity [<a data-track="click" data-track-action="reference anchor" data-track-label="link" data-test="citation-ref" aria-label="Reference 31" title="Gavrielov-Yusim N, Kurzinger ML, Nishikawa C, Pan C, Pouget J, Epstein LB, et al. Comparison of text processing methods in social media-based signal detection. Pharmacoepidemiol Drug Saf. 2019;28(10):1309–17. 
 https://doi.org/10.1002/pds.4857
 
 ." href="/articles/10.1186/s13063-021-05489-x#ref-CR31" id="ref-link-section-d45421490e1453">31</a>]. Patient data from wearable devices have proven to be able to correlate participant activity with the International Parkinson and Movement Disorders Society Unified Parkinson’s Disease Rating Scale, distinguish between neuropsychiatric symptomatology patterns, and identify patient falls [<a data-track="click" data-track-action="reference anchor" data-track-label="link" data-test="citation-ref" title="Barnett I, Torous J, Staples P, Sandoval L, Keshavan M, Onnela JP. Relapse prediction in schizophrenia through digital phenotyping: a pilot study. Neuropsychopharmacology. 2018;43(8):1660–6. 
 https://doi.org/10.1038/s41386-018-0030-z
 
 ." href="#ref-CR32" id="ref-link-section-d45421490e1456">32</a>,<a data-track="click" data-track-action="reference anchor" data-track-label="link" data-test="citation-ref" title="Chaudhuri S, Oudejans D, Thompson HJ, Demiris G. Real-world accuracy and use of a wearable fall detection device by older adults. J Am Geriatr Soc. 2015;63(11):2415–6. 
 https://doi.org/10.1111/jgs.13804
 
 ." href="#ref-CR33" id="ref-link-section-d45421490e1456_1">33</a>,<a data-track="click" data-track-action="reference anchor" data-track-label="link" data-test="citation-ref" aria-label="Reference 34" title="Chen R, Jankovic F, Marinsek N, Foschini L, Kourtis L, Signorini A, et al. Developing measures of cognitive impairment in the real world from consumer-grade multimodal sensor streams. In: Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. Anchorage: Association for Computing Machinery; 2019. p. 2145–55." href="/articles/10.1186/s13063-021-05489-x#ref-CR34" id="ref-link-section-d45421490e1460">34</a>].</p><p>In summary, although ML and NLP have shown promise across a broad range of activities related to improving the management of participants in clinical trials, the implications of these applications of ML/NLP in regard to clinical trial quality and participant experience are unclear. Studies comparing different approaches to participant management are a necessary next step toward identifying best practices.</p></div></div></section><section data-title="Data collection and management"><div class="c-article-section" id="Sec9-section"><h2 class="c-article-section__title js-section-title js-c-reading-companion-sections-item" id="Sec9">Data collection and management</h2><div class="c-article-section__content" id="Sec9-content"><p>The use of ML in clinical trials can change the data collection, management, and analysis techniques required. However, ML methods can help address some of the difficulties associated with missing data and collecting real-world data.</p><h3 class="c-article__sub-heading" id="Sec10">Collection, processing, and management of data from wearable and other smart devices</h3><p>Patient-generated health data from wearable and other mobile/electronic devices can supplement or even replace study visits and their associated traditional data collection in certain situations. Wearables and other devices may enable the validation and use of new, patient-centered biomarkers. Developing new “digital biomarkers” from the data collected by a mobile device’s various sensors (such as cameras, audio recorders, accelerometers, and photoplethysmograms) often requires ML processing to derive actionable insights because the data yielded from these devices can be sparse as well as variable in quality, availability, and synchronicity. Using the relatively large and complex data yielded by wearables and other devices for research purposes therefore requires specialized data collection, storage, validation, and analysis techniques [<a data-track="click" data-track-action="reference anchor" data-track-label="link" data-test="citation-ref" title="Chen R, Jankovic F, Marinsek N, Foschini L, Kourtis L, Signorini A, et al. Developing measures of cognitive impairment in the real world from consumer-grade multimodal sensor streams. In: Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. Anchorage: Association for Computing Machinery; 2019. p. 2145–55." href="#ref-CR34" id="ref-link-section-d45421490e1482">34</a>,<a data-track="click" data-track-action="reference anchor" data-track-label="link" data-test="citation-ref" title="Yurtman A, Barshan B, Fidan B. Activity recognition invariant to wearable sensor unit orientation using differential rotational transformations represented by quaternions. Sensors (Basel). 2018;18(8):2725. 
 https://pubmed.ncbi.nlm.nih.gov/30126235/
 
 ." href="#ref-CR35" id="ref-link-section-d45421490e1482_1">35</a>,<a data-track="click" data-track-action="reference anchor" data-track-label="link" data-test="citation-ref" title="Lu K, Yang L, Seoane F, Abtahi F, Forsman M, Lindecrantz K. Fusion of heart rate, respiration and motion measurements from a wearable sensor system to enhance energy expenditure estimation. Sensors (Basel). 2018;18(9):3092. 
 https://pubmed.ncbi.nlm.nih.gov/30223429/
 
 ." href="#ref-CR36" id="ref-link-section-d45421490e1482_2">36</a>,<a data-track="click" data-track-action="reference anchor" data-track-label="link" data-test="citation-ref" aria-label="Reference 37" title="Cheung YK, Hsueh PS, Ensari I, Willey JZ, Diaz KM. Quantile coarsening analysis of high-volume wearable activity data in a longitudinal observational study. Sensors (Basel). 2018;18(9):3056. 
 https://pubmed.ncbi.nlm.nih.gov/30213093/
 
 ." href="/articles/10.1186/s13063-021-05489-x#ref-CR37" id="ref-link-section-d45421490e1485">37</a>]. For instance, a deep neural network was used to process input from a mobile single-lead electrocardiogram platform [<a data-track="click" data-track-action="reference anchor" data-track-label="link" data-test="citation-ref" aria-label="Reference 38" title="Hannun AY, Rajpurkar P, Haghpanahi M, Tison GH, Bourn C, Turakhia MP, et al. Cardiologist-level arrhythmia detection and classification in ambulatory electrocardiograms using a deep neural network. Nat Med. 2019;25(1):65–9. 
 https://doi.org/10.1038/s41591-018-0268-3
 
 ." href="/articles/10.1186/s13063-021-05489-x#ref-CR38" id="ref-link-section-d45421490e1488">38</a>], a random forest model was used to process audio output from patients with Parkinson’s disease [<a data-track="click" data-track-action="reference anchor" data-track-label="link" data-test="citation-ref" aria-label="Reference 39" title="Ozkanca Y, Ozturk MG, Ekmekci MN, Atkins DC, Demiroglu C, Ghomi RH. Depression screening from voice samples of patients affected by Parkinson's disease. Digit Biomark. 2019;3(2):72–82. 
 https://doi.org/10.1159/000500354
 
 ." href="/articles/10.1186/s13063-021-05489-x#ref-CR39" id="ref-link-section-d45421490e1491">39</a>], and a recurrent neural network was used to process accelerometer data from patients with atopic dermatitis [<a data-track="click" data-track-action="reference anchor" data-track-label="link" data-test="citation-ref" aria-label="Reference 40" title="Moreau A, Anderer P, Ross M, Cerny A, Almazan TH, Peterson B, et al. Detection of nocturnal scratching movements in patients with atopic dermatitis using accelerometers and recurrent neural networks. IEEE J Biomed Health Inform. 2018;22(4):1011–8. 
 https://doi.org/10.1109/JBHI.2017.2710798
 
 ." href="/articles/10.1186/s13063-021-05489-x#ref-CR40" id="ref-link-section-d45421490e1494">40</a>]. These novel digital biomarkers may facilitate the efficient conduct and patient-centeredness of clinical trials, but this approach carries potential pitfalls. As has been shown to occur with an electrocardiogram classification model, ML processing of wearable sensor output to derive research endpoints introduces the possibility of corrupt results if the ML model is subverted by intentionally or unintentionally modified sensor data (though this risk exists with any data regardless of processing technique) [<a data-track="click" data-track-action="reference anchor" data-track-label="link" data-test="citation-ref" aria-label="Reference 41" title="Han X, Hu Y, Foschini L, Chinitz L, Jankelson L, Ranganath R. Deep learning models for electrocardiograms are susceptible to adversarial attack. Nat Med. 2020;26(3):360–3. 
 https://doi.org/10.1038/s41591-020-0791-x
 
 Epub 2020/03/11. PubMed PMID: 32152582." href="/articles/10.1186/s13063-021-05489-x#ref-CR41" id="ref-link-section-d45421490e1498">41</a>]. Because of the complexity involved, software intended to diagnose, monitor, or treat medical conditions is regulated by the FDA, and the FDA has processes and guidance related to biomarker validation and qualification for use in regulatory trials.</p><p>Beyond the development of novel digital biomarkers, other device-related opportunities in patient centricity include the ability to export data and analytics back to participants to facilitate education and insight. Barriers to implementation of ML processing of device data include better defining how previously validated clinical endpoints and patient-centric digital biomarkers overlap as well as understanding participant opinions about privacy in relation to the sharing and use of device data. FDA approval of novel biomarkers will also be required. Researchers interested in leveraging the power of these devices must explain to patients their risks and benefits both for ethical and privacy-related reasons and because implementation without addressing participant concerns has the potential to <i>worsen</i> participant recruitment and retention [<a data-track="click" data-track-action="reference anchor" data-track-label="link" data-test="citation-ref" aria-label="Reference 42" title="Doerr M, Maguire Truong A, Bot BM, Wilbanks J, Suver C, Mangravite LM. Formative evaluation of participant experience with mobile econsent in the app-mediated Parkinson mPower study: a mixed methods study. JMIR Mhealth Uhealth. 2017;5(2):e14. 
 https://doi.org/10.2196/mhealth.6521
 
 ." href="/articles/10.1186/s13063-021-05489-x#ref-CR42" id="ref-link-section-d45421490e1507">42</a>].</p><h3 class="c-article__sub-heading" id="Sec11">Study data collection, verification, and surveillance</h3><p>An appealing application of ML, specifically NLP, to study data management is to automate data collection into case report forms, decreasing the time, expense, and potential for error associated with human data extraction, whether in prospective trials or retrospective reviews. Though this use requires overcoming variable data structures and provenances, it has shown early promise in cancer [<a data-track="click" data-track-action="reference anchor" data-track-label="link" data-test="citation-ref" aria-label="Reference 43" title="Savova GK, Danciu I, Alamudun F, Miller T, Lin C, Bitterman DS, et al. Use of natural language processing to extract clinical cancer phenotypes from electronic medical records. Cancer Res. 2019;79(21):5463–70. 
 https://doi.org/10.1158/0008-5472.CAN-19-0579
 
 ." href="/articles/10.1186/s13063-021-05489-x#ref-CR43" id="ref-link-section-d45421490e1518">43</a>, <a data-track="click" data-track-action="reference anchor" data-track-label="link" data-test="citation-ref" aria-label="Reference 44" title="Malke JC, Jin S, Camp SP, Lari B, Kell T, Simon JM, et al. Enhancing case capture, quality, and completeness of primary melanoma pathology records via natural language processing. JCO Clin Cancer Inform. 2019;3:1–11. 
 https://doi.org/10.1200/CCI.19.00006
 
 ." href="/articles/10.1186/s13063-021-05489-x#ref-CR44" id="ref-link-section-d45421490e1521">44</a>], epilepsy [<a data-track="click" data-track-action="reference anchor" data-track-label="link" data-test="citation-ref" aria-label="Reference 30" title="Fonferko-Shadrach B, Lacey AS, Roberts A, Akbari A, Thompson S, Ford DV, et al. Using natural language processing to extract structured epilepsy data from unstructured clinic letters: development and validation of the ExECT (extraction of epilepsy clinical text) system. BMJ Open. 2019;9(4):e023232. 
 https://doi.org/10.1136/bmjopen-2018-023232
 
 ." href="/articles/10.1186/s13063-021-05489-x#ref-CR30" id="ref-link-section-d45421490e1524">30</a>], and depression [<a data-track="click" data-track-action="reference anchor" data-track-label="link" data-test="citation-ref" aria-label="Reference 45" title="Vaci N, Liu Q, Kormilitzin A, De Crescenzo F, Kurtulmus A, Harvey J, et al. Natural language processing for structuring clinical text data on depression using UK-CRIS. Evid Based Ment Health. 2020;23(1):21–6. 
 https://doi.org/10.1136/ebmental-2019-300134
 
 ." href="/articles/10.1186/s13063-021-05489-x#ref-CR45" id="ref-link-section-d45421490e1527">45</a>], among other areas [<a data-track="click" data-track-action="reference anchor" data-track-label="link" data-test="citation-ref" aria-label="Reference 29" title="Han J, Chen K, Fang L, Zhang S, Wang F, Ma H, et al. Improving the efficacy of the data entry process for clinical research with a natural language processing-driven medical information extraction system: quantitative field research. JMIR Med Inform. 2019;7(3):e13331. 
 https://doi.org/10.2196/13331
 
 ." href="/articles/10.1186/s13063-021-05489-x#ref-CR29" id="ref-link-section-d45421490e1530">29</a>]. Regardless of how data have been collected, ML can power risk-based monitoring approaches to clinical trial surveillance, enabling the prevention and/or early detection of site failure, fraud, and data inconsistencies or incompleteness that may delay database lock and subsequent analysis. For instance, even when humans collect data into case report forms (often transmitted in PDF form), the adequacy of the collected data for outcome ascertainment can be assessed by combining optical character recognition with NLP [<a data-track="click" data-track-action="reference anchor" data-track-label="link" data-test="citation-ref" aria-label="Reference 46" title="Tian Q, Liu M, Min L, An J, Lu X, Duan H. An automated data verification approach for improving data quality in a clinical registry. Comput Methods Programs Biomed. 2019;181:104840. 
 https://doi.org/10.1016/j.cmpb.2019.01.012
 
 ." href="/articles/10.1186/s13063-021-05489-x#ref-CR46" id="ref-link-section-d45421490e1534">46</a>]. Suspicious data patterns in clinical trials, or incorrect data in observational studies, can be identified by applying auto-encoders to distinguish plausible from implausible data [<a data-track="click" data-track-action="reference anchor" data-track-label="link" data-test="citation-ref" aria-label="Reference 47" title="Estiri H, Murphy SN. Semi-supervised encoding for outlier detection in clinical observation data. Comput Methods Programs Biomed. 2019;181:104830. 
 https://doi.org/10.1016/j.cmpb.2019.01.002
 
 ." href="/articles/10.1186/s13063-021-05489-x#ref-CR47" id="ref-link-section-d45421490e1537">47</a>].</p><h3 class="c-article__sub-heading" id="Sec12">Endpoint identification, adjudication, and detection of safety signals</h3><p>ML can also be applied to data processing. Semi-automated endpoint identification and adjudication offers the potential to reduce time, cost, and complexity compared with the current approach of manual adjudication of events by a committee of clinicians, because while endpoint adjudication has traditionally been a labor-intensive process, sorting and classifying events lies well within the capabilities of ML. For instance, IQVIA Inc. has described the ability to automatically process some adverse events related to drug therapies using a combination of optical character recognition and NLP, though this technique has not been described in peer-reviewed publications [<a data-track="click" data-track-action="reference anchor" data-track-label="link" data-test="citation-ref" aria-label="Reference 48" title="Glass, LMS G; Patil, R. AI in clinical development: improving safety and accelerating results. [White paper]. In press 2019." href="/articles/10.1186/s13063-021-05489-x#ref-CR48" id="ref-link-section-d45421490e1548">48</a>]. A potential barrier to implementation of semi-automated event adjudication is that endpoint definitions and the data required to support them often change from trial to trial, which theoretically requires re-training a classification model for each new trial (which is not a viable approach). More recently, efforts have been made to standardize outcomes in the field of cardiovascular research, though not all trials adhere to these outcomes. Trial data have not been pooled to facilitate model training for cardiovascular endpoints, and most fields have not yet undertaken similar efforts [<a data-track="click" data-track-action="reference anchor" data-track-label="link" data-test="citation-ref" aria-label="Reference 49" title="Hicks KA, Mahaffey KW, Mehran R, Nissen SE, Wiviott SD, Dunn B, et al. 2017 Cardiovascular and stroke endpoint definitions for clinical trials. Circulation. 2018;137(9):961–72. 
 https://doi.org/10.1161/CIRCULATIONAHA.117.033502
 
 ." href="/articles/10.1186/s13063-021-05489-x#ref-CR49" id="ref-link-section-d45421490e1551">49</a>]. Further efforts in this area will require true consensus about event definitions, use of consensus definitions, and a willingness of stakeholders to share adequate data for model training from across multiple trials.</p><h3 class="c-article__sub-heading" id="Sec13">Approaches to missing data</h3><p>ML can be used in several different ways to address the problem of missing data, across multiple causes for data missingness, data-related assumptions and goals, and data collection and intended analytic methods. Possible goals may be to impute specific estimates of the missing covariate values directly or to average over many possible values from some learned distribution to compute other quantities of interest. While the latest methods are evolving and more systematic comparisons are needed, some early evidence suggests more complex ML methods may not always be of benefit over simpler imputation methods, such as population mean imputation [<a data-track="click" data-track-action="reference anchor" data-track-label="link" data-test="citation-ref" aria-label="Reference 50" title="Liu Y, Gopalakrishnan V. An overview and evaluation of recent machine learning imputation methods using cardiac imaging data. Data (Basel). 2017;2(1):8. 
 https://pubmed.ncbi.nlm.nih.gov/28243594/
 
 ." href="/articles/10.1186/s13063-021-05489-x#ref-CR50" id="ref-link-section-d45421490e1563">50</a>]. Applications of missing value techniques include analysis of sparse datasets, such as registries, EHR data, ergonomic data, and data from wearable devices [<a data-track="click" data-track-action="reference anchor" data-track-label="link" data-test="citation-ref" title="Phung S, Kumar A, Kim J. A deep learning technique for imputing missing healthcare data. Conf Proc IEEE Eng Med Biol Soc. 2019;2019:6513–6. 
 https://doi.org/10.1109/EMBC.2019.8856760
 
 Epub 2020/01/18PubMed PMID: 31947333." href="#ref-CR51" id="ref-link-section-d45421490e1566">51</a>,<a data-track="click" data-track-action="reference anchor" data-track-label="link" data-test="citation-ref" title="Qiu YL, Zheng H, Gevaert OJ. A deep learning framework for imputing missing values in genomic data; 2018." href="#ref-CR52" id="ref-link-section-d45421490e1566_1">52</a>,<a data-track="click" data-track-action="reference anchor" data-track-label="link" data-test="citation-ref" title="Feng T, Narayanan S. Imputing missing data in large-scale multivariate biomedical wearable recordings using bidirectional recurrent neural networks with temporal activation regularization. In: 2019 41st Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC); 2019." href="#ref-CR53" id="ref-link-section-d45421490e1566_2">53</a>,<a data-track="click" data-track-action="reference anchor" data-track-label="link" data-test="citation-ref" aria-label="Reference 54" title="Luo Y, Szolovits P, Dighe AS, Baron JM. 3D-MICE: integration of cross-sectional and longitudinal imputation for multi-analyte longitudinal clinical data. J Am Med Inform Assoc. 2018;25(6):645–53. 
 https://doi.org/10.1093/jamia/ocx133
 
 ." href="/articles/10.1186/s13063-021-05489-x#ref-CR54" id="ref-link-section-d45421490e1569">54</a>]. Although these techniques can help mitigate the negative effects of data missingness or scarcity, over-reliance on data augmentation methods may lead to the development of models with limited applicability to new, imperfect datasets. Therefore, a more meaningful approach would be to apply ML to improve data collection during the conduct of research itself.</p><h3 class="c-article__sub-heading" id="Sec14">Data analysis</h3><p>Data collected in clinical trials, registries, and clinical practices are fertile sources for hypothesis generation, risk modeling, and counterfactual simulation, and ML is well suited for these efforts. For instance, unsupervised learning can identify phenotypic clusters in real-world data that can be further explored in clinical trials [<a data-track="click" data-track-action="reference anchor" data-track-label="link" data-test="citation-ref" aria-label="Reference 55" title="Ngufor C, Warner MA, Murphree DH, Liu H, Carter R, Storlie CB, et al. Identification of Clinically meaningful plasma transfusion subgroups using unsupervised random forest clustering. AMIA Annu Symp Proc. 2017;2017:1332–41." href="/articles/10.1186/s13063-021-05489-x#ref-CR55" id="ref-link-section-d45421490e1580">55</a>, <a data-track="click" data-track-action="reference anchor" data-track-label="link" data-test="citation-ref" aria-label="Reference 56" title="Tomic A, Tomic I, Rosenberg-Hasson Y, Dekker CL, Maecker HT, Davis MM. SIMON, an automated machine learning system, reveals immune signatures of influenza vaccine responses. J Immunol. 2019;203(3):749–59. 
 https://doi.org/10.4049/jimmunol.1900033
 
 ." href="/articles/10.1186/s13063-021-05489-x#ref-CR56" id="ref-link-section-d45421490e1583">56</a>]. Furthermore, ML can potentially improve the ubiquitous practice of secondary trial analyses by more powerfully identifying treatment heterogeneity while still providing some protection (although incomplete) against false-positive discoveries, uncovering more promising avenues for future study [<a data-track="click" data-track-action="reference anchor" data-track-label="link" data-test="citation-ref" aria-label="Reference 57" title="Watson JA, Holmes CC. Machine learning analysis plans for randomised controlled trials: detecting treatment effect heterogeneity with strict control of type I error. Trials. 2020;21(1):156. 
 https://doi.org/10.1186/s13063-020-4076-y
 
 ." href="/articles/10.1186/s13063-021-05489-x#ref-CR57" id="ref-link-section-d45421490e1586">57</a>, <a data-track="click" data-track-action="reference anchor" data-track-label="link" data-test="citation-ref" aria-label="Reference 58" title="Rigdon J, Baiocchi M, Basu S. Preventing false discovery of heterogeneous treatment effect subgroups in randomized trials. Trials. 2018;19(1):382. 
 https://doi.org/10.1186/s13063-018-2774-5
 
 ." href="/articles/10.1186/s13063-021-05489-x#ref-CR58" id="ref-link-section-d45421490e1589">58</a>]. Additionally, ML is effectively used to generate risk predictions in retrospective datasets that can subsequently be prospectively validated. For instance, using a random forest model in COMPANION trial data, researchers were able to improve discrimination between patients who would do better or worse following cardiac resynchronization therapy compared with a multivariable logistic regression [<a data-track="click" data-track-action="reference anchor" data-track-label="link" data-test="citation-ref" aria-label="Reference 59" title="Kalscheur MM, Kipp RT, Tattersall MC, Mei C, Buhr KA, DeMets DL, et al. Machine learning algorithm predicts cardiac resynchronization therapy outcomes: lessons from the companion trial. Circ Arrhythm Electrophysiol. 2018;11(1):e005499. 
 https://doi.org/10.1161/CIRCEP.117.005499
 
 ." href="/articles/10.1186/s13063-021-05489-x#ref-CR59" id="ref-link-section-d45421490e1592">59</a>]. This demonstrates the ability of random forests to model interactions between features that are not captured by simpler models.</p><p>While predictive modeling is an important and necessary task, the derivation of real-world evidence from real-world data (i.e., making causal inferences) remains a highly sought-after (and very difficult) goal toward which ML offers some promise. Proposed techniques include optimal discriminant analysis, targeted maximum likelihood estimation, and ML-powered propensity score weighting [<a data-track="click" data-track-action="reference anchor" data-track-label="link" data-test="citation-ref" title="Linden A, Yarnold PR. Combining machine learning and propensity score weighting to estimate causal effects in multivalued treatments. J Eval Clin Pract. 2016;22(6):871–81. 
 https://doi.org/10.1111/jep.12610
 
 ." href="#ref-CR60" id="ref-link-section-d45421490e1598">60</a>,<a data-track="click" data-track-action="reference anchor" data-track-label="link" data-test="citation-ref" title="Schuler MS, Rose S. Targeted maximum likelihood estimation for causal inference in observational studies. Am J Epidemiol. 2017;185(1):65–73. 
 https://doi.org/10.1093/aje/kww165
 
 ." href="#ref-CR61" id="ref-link-section-d45421490e1598_1">61</a>,<a data-track="click" data-track-action="reference anchor" data-track-label="link" data-test="citation-ref" title="Wendling T, Jung K, Callahan A, Schuler A, Shah NH, Gallego B. Comparing methods for estimation of heterogeneous treatment effects using observational data from health care databases. Stat Med. 2018;37(23):3309–24. 
 https://doi.org/10.1002/sim.7820
 
 ." href="#ref-CR62" id="ref-link-section-d45421490e1598_2">62</a>,<a data-track="click" data-track-action="reference anchor" data-track-label="link" data-test="citation-ref" title="Schomaker M, Luque-Fernandez MA, Leroy V, Davies MA. Using longitudinal targeted maximum likelihood estimation in complex settings with dynamic interventions. Stat Med. 2019;38(24):4888–911. 
 https://doi.org/10.1002/sim.8340
 
 ." href="#ref-CR63" id="ref-link-section-d45421490e1598_3">63</a>,<a data-track="click" data-track-action="reference anchor" data-track-label="link" data-test="citation-ref" aria-label="Reference 64" title="Pirracchio R, Petersen ML, van der Laan M. Improving propensity score estimators’ robustness to model misspecification using super learner. Am J Epidemiol. 2015;181(2):108–19. 
 https://doi.org/10.1093/aje/kwu253
 
 ." href="/articles/10.1186/s13063-021-05489-x#ref-CR64" id="ref-link-section-d45421490e1601">64</a>]. A particularly intriguing technique involves use of ML to enable counterfactual policy estimation, in which existing data can be used to make predictions about outcomes under circumstances that do not yet, or could not, exist [<a data-track="click" data-track-action="reference anchor" data-track-label="link" data-test="citation-ref" aria-label="Reference 65" title="Gottesman O, Johansson F, Komorowski M, Faisal A, Sontag D, Doshi-Velez F, et al. Guidelines for reinforcement learning in healthcare. Nat Med. 2019;25(1):16–8. 
 https://doi.org/10.1038/s41591-018-0310-5
 
 ." href="/articles/10.1186/s13063-021-05489-x#ref-CR65" id="ref-link-section-d45421490e1604">65</a>]. For instance, trees of predictors can offer survival estimates for heart failure patients under the conditions of receiving or not receiving a heart transplant and reinforcement learning suggests improved treatment policies on the basis of prior sub-optimal treatments and outcomes [<a data-track="click" data-track-action="reference anchor" data-track-label="link" data-test="citation-ref" aria-label="Reference 66" title="Yoon J, Zame WR, Banerjee A, Cadeiras M, Alaa AM, van der Schaar M. Personalized survival predictions via trees of predictors: an application to cardiac transplantation. PLoS One. 2018;13(3):e0194985. 
 https://doi.org/10.1371/journal.pone.0194985
 
 ." href="/articles/10.1186/s13063-021-05489-x#ref-CR66" id="ref-link-section-d45421490e1607">66</a>, <a data-track="click" data-track-action="reference anchor" data-track-label="link" data-test="citation-ref" aria-label="Reference 67" title="Komorowski M, Celi LA, Badawi O, Gordon AC, Faisal AA. The artificial intelligence clinician learns optimal treatment strategies for sepsis in intensive care. Nat Med. 2018;24(11):1716–20. 
 https://doi.org/10.1038/s41591-018-0213-5
 
 ." href="/articles/10.1186/s13063-021-05489-x#ref-CR67" id="ref-link-section-d45421490e1610">67</a>]. Unfortunately, major barriers to implementation are a lack of interoperability between EHR data structures and fraught data sharing agreements that limit the amount of data available for model training [<a data-track="click" data-track-action="reference anchor" data-track-label="link" data-test="citation-ref" aria-label="Reference 68" title="Ghassemi M, Naumann T, Schulam P, Beam AL, Chen IY, Ranganath R. Practical guidance on artificial intelligence for health-care data. Lancet Digit Health. 2019;1(4):e157–9. 
 https://doi.org/10.1016/S2589-7500(19)30084-6
 
 ." href="/articles/10.1186/s13063-021-05489-x#ref-CR68" id="ref-link-section-d45421490e1614">68</a>].</p><p>In summary, there are many effective ML approaches to clinical trial data management, processing, and analysis but fewer techniques for improving the quality of data as they are generated and collected. As data availability and quality are the foundations of ML approaches, the conduct of high-quality trials remains of utmost importance to enable higher-level ML processing.</p></div></div></section><section data-title="Barriers to the integration of ML techniques in clinical research"><div class="c-article-section" id="Sec15-section"><h2 class="c-article-section__title js-section-title js-c-reading-companion-sections-item" id="Sec15">Barriers to the integration of ML techniques in clinical research</h2><div class="c-article-section__content" id="Sec15-content"><p>Both operational and philosophical barriers limit the harnessing of the full potential of ML for clinical research. ML in clinical research is a high-risk proposition due to the potential to propagate errors or biases through multiple research contexts and into the corpus of biomedical evidence due to the use of flawed models; however, as previously discussed, ML offers promising ways to improve the quality and efficiency of clinical research for patients and other stakeholders. Both the operational and philosophical barriers to ML integration require attention at each stage of model development and use to overcome hurdles while maximizing stakeholder confidence in the process and its results. Operational barriers to ML integration in clinical research can aggravate and reinforce philosophical concerns if not managed in a robust and transparent manner. For instance, inadequate training data and poor model calibration can lead to racial bias in model application, such as has been noted in ML for melanoma identification [<a data-track="click" data-track-action="reference anchor" data-track-label="link" data-test="citation-ref" aria-label="Reference 27" title="Adamson AS, Smith A. Machine learning and health care disparities in dermatology. JAMA Dermatol. 2018;154(11):1247–8. 
 https://doi.org/10.1001/jamadermatol.2018.2348
 
 ." href="/articles/10.1186/s13063-021-05489-x#ref-CR27" id="ref-link-section-d45421490e1629">27</a>]. Stakeholders, including regulatory agencies, funding sources, researchers, participants, and industry partners, must collaborate to fully integrate ML into clinical research. The wider ML community espouses “FAT (fairness, accountability, and transparency) ML” principles that also include responsibility, explainability, accuracy, auditability, and fairness and that should be applied to ML in clinical research, as discussed further.</p><h3 class="c-article__sub-heading" id="Sec16">Operational barriers to ML in clinical research</h3><p>The development of ML algorithms and their deployment for clinical research use is a multi-stage, multi-disciplinary process. The first step is to assemble a team with the clinical and ML domain expertise necessary for success. Failing to assemble such a team and to communicate openly within the team increases the risks of either developing a model that distorts clinical reality or using an ML technique that is inappropriate to the available data and research question at hand [<a data-track="click" data-track-action="reference anchor" data-track-label="link" data-test="citation-ref" aria-label="Reference 69" title="Wiens J, Saria S, Sendak M, Ghassemi M, Liu VX, Doshi-Velez F, et al. Do no harm: a roadmap for responsible machine learning for health care. Nat Med. 2019;25(9):1337–40. 
 https://doi.org/10.1038/s41591-019-0548-6
 
 Epub 2019/08/21. PubMed PMID: 31427808." href="/articles/10.1186/s13063-021-05489-x#ref-CR69" id="ref-link-section-d45421490e1639">69</a>]. For instance, a model to predict mortality created without any clinical team members may identify intubation as predictive of mortality, which is certainly true but likely clinically useless. Collaboration is necessary and valuable for both the data science and clinical science components of the team but may require additional up-front, cross-disciplinary training, transparency, and trust to fully operationalize.</p><p>The choice and availability of data for algorithm development and validation is both a stubborn and highly significant barrier to ML integration into clinical research, though its full discussion is outside the scope of this manuscript. Many recent ML models, especially deep neural networks, require large amounts of data to train and validate. To ensure generalizability beyond the training data set, developers should use multiple data sources during this process because a number of documented cases demonstrated that algorithms performed significantly differently in validation data sets compared with training data sets [<a data-track="click" data-track-action="reference anchor" data-track-label="link" data-test="citation-ref" aria-label="Reference 70" title="Nestor B, McDermott M, Chauhan G, et al. Rethinking clinical prediction: why machine learning must consider year of care and feature aggregation. arXiv preprint 2018;arXiv:181112583." href="/articles/10.1186/s13063-021-05489-x#ref-CR70" id="ref-link-section-d45421490e1645">70</a>]. Because data used in clinical research are often patient related and generated by institutions (in the case of EHR data) or companies (in the case of clinical trial data) at a significant cost, owners of data may be reluctant to share. Even when they are willing to share data, variation in data collection and storage techniques can hamper interoperability. Large datasets, such as MIMIC, eICU, and the UK Biobank, are good resources when other real-world data cannot be obtained [<a data-track="click" data-track-action="reference anchor" data-track-label="link" data-test="citation-ref" title="Johnson AE, Pollard TJ, Shen L, Lehman LW, Feng M, Ghassemi M, et al. MIMIC-III, a freely accessible critical care database. Sci Data. 2016;3(1):160035. 
 https://doi.org/10.1038/sdata.2016.35
 
 ." href="#ref-CR71" id="ref-link-section-d45421490e1648">71</a>,<a data-track="click" data-track-action="reference anchor" data-track-label="link" data-test="citation-ref" title="Pollard TJ, Johnson AEW, Raffa JD, Celi LA, Mark RG, Badawi O. The eICU Collaborative Research Database, a freely available multi-center database for critical care research. Sci Data. 2018;5(1):180178. 
 https://doi.org/10.1038/sdata.2018.178
 
 ." href="#ref-CR72" id="ref-link-section-d45421490e1648_1">72</a>,<a data-track="click" data-track-action="reference anchor" data-track-label="link" data-test="citation-ref" aria-label="Reference 73" title="UK Biobank. 
 www.ukbiobank.ac.uk
 
 . Accessed 22 Mar 2021." href="/articles/10.1186/s13063-021-05489-x#ref-CR73" id="ref-link-section-d45421490e1651">73</a>], but any single data source is inadequate to yield a model that is ready for use, especially because training on retrospective data (such as MIMIC and UK Biobank) does not always translate well to prospective applications. For example, Nestor et al. demonstrated the importance of considering year of care in MIMIC due to temporal drift, and Gong et al. demonstrated methods for feature aggregation across large temporal changes, such as EHR transitions [<a data-track="click" data-track-action="reference anchor" data-track-label="link" data-test="citation-ref" aria-label="Reference 70" title="Nestor B, McDermott M, Chauhan G, et al. Rethinking clinical prediction: why machine learning must consider year of care and feature aggregation. arXiv preprint 2018;arXiv:181112583." href="/articles/10.1186/s13063-021-05489-x#ref-CR70" id="ref-link-section-d45421490e1654">70</a>, <a data-track="click" data-track-action="reference anchor" data-track-label="link" data-test="citation-ref" aria-label="Reference 74" title="Gong JJ, Naumann T, Szolovits P, Guttag JV. Predicting clinical outcomes across changing electronic health record systems. In: Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. Halifax: Association for Computing Machinery; 2017. p. 1497–505." href="/articles/10.1186/s13063-021-05489-x#ref-CR74" id="ref-link-section-d45421490e1657">74</a>]. Furthermore, certain disease states and patient types are less likely to be well represented in data generated for the purpose of clinical care. For example, while MIMIC is widely used because of its public availability, models trained on its ICU population are unlikely to generalize to many applications outside critical care. These issues with data availability and quality are intimately associated with problems surrounding reproducibility and replicability [<a data-track="click" data-track-action="reference anchor" data-track-label="link" data-test="citation-ref" aria-label="Reference 75" title="Beam AL, Manrai AK, Ghassemi M. Challenges to the reproducibility of machine learning models in health care. JAMA. 2020;323(4):305–6. 
 https://doi.org/10.1001/jama.2019.20866
 
 ." href="/articles/10.1186/s13063-021-05489-x#ref-CR75" id="ref-link-section-d45421490e1661">75</a>], which are more difficult to achieve in ML-driven clinical research for a number of reasons in addition to data availability, including the role of randomness in many ML techniques and the computational expense of model replication. The ongoing difficulties with reproducibility and replicability of ML-driven clinical research threaten to undermine stakeholder confidence in ML integration into clinical research.</p><h3 class="c-article__sub-heading" id="Sec17">Philosophical barriers to ML in clinical research</h3><p><i>Explainability</i> refers to the concept that the processes underlying algorithmic output should be explainable to algorithm users in terms they understand. A large amount of research has been devoted to techniques to accomplish this, including attention scores and saliency maps, but concerns about the performance and suitability of these techniques persist [<a data-track="click" data-track-action="reference anchor" data-track-label="link" data-test="citation-ref" title="Adebayo J, Gilmer J, Muelly M, Goodfellow I, Hardt M, Kim B. Sanity checks for saliency maps. In: Proceedings of the 32nd International Conference on Neural Information Processing Systems. Montréal: Curran Associates Inc.; 2018. p. 9525–36." href="#ref-CR76" id="ref-link-section-d45421490e1674">76</a>,<a data-track="click" data-track-action="reference anchor" data-track-label="link" data-test="citation-ref" title="Wiegreffe S, Pinter Y. Attention is not not explanation. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). Hong Kong, China: Association for Computational Linguistics; 2019." href="#ref-CR77" id="ref-link-section-d45421490e1674_1">77</a>,<a data-track="click" data-track-action="reference anchor" data-track-label="link" data-test="citation-ref" title="Jain S, Wallace BC. Attention is not explanation: NAACL-HLT; 2019." href="#ref-CR78" id="ref-link-section-d45421490e1674_2">78</a>,<a data-track="click" data-track-action="reference anchor" data-track-label="link" data-test="citation-ref" aria-label="Reference 79" title="Serrano S, Smith NA. Is attention interpretable? In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 2931–2951, Stroudsburg, PA, USA, 2019. Association for Computational Linguistics." href="/articles/10.1186/s13063-021-05489-x#ref-CR79" id="ref-link-section-d45421490e1677">79</a>]. Though an appealing principle, a significant debate exists about whether the concept of explainability interferes unnecessarily with the ability of ML to positively contribute to clinical care and research. Explainability may lead researchers to incorrectly trust fundamentally flawed models. Proponents of this argument instead champion <i>trustworthiness</i>. Advocates of trustworthiness are of the opinion that many aspects of clinical medicine (and of clinical research)—such as laboratory assays, the complete mechanisms of certain medications, and statistical tests—that are not well or widely understood continue to be used because they have been shown to work reliably and well, even if <i>how</i> or <i>why</i> remains opaque to many end users [<a data-track="click" data-track-action="reference anchor" data-track-label="link" data-test="citation-ref" aria-label="Reference 80" title="Sendak M, Elish MC, Gao M, Futoma J, Ratliff W, Nichols M, et al. “The human body is a black box”: supporting clinical decision-making with deep learning. In: Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency. Barcelona: Association for Computing Machinery; 2020. p. 99–109." href="/articles/10.1186/s13063-021-05489-x#ref-CR80" id="ref-link-section-d45421490e1690">80</a>]. This philosophical barrier has more recently become an operational barrier as well with the passage of the European Union’s General Data Protection Regulation, which requires that automated decision-making algorithms provide “meaningful information about the logic involved.”</p><p>Part of the focus on explainability and trustworthiness is due to a desire to understand whether ML algorithms are introducing <i>bias</i> into model output, as was notably shown to be the case in a highly publicized series of ProPublica articles about recidivism prediction algorithms [<a data-track="click" data-track-action="reference anchor" data-track-label="link" data-test="citation-ref" aria-label="Reference 81" title="Angwin J LJ, Mattu S, Kirchner L. Machine bias. ProPublica. 2016 13 May 2020. Available from: 
 https://www.propublica.org/article/machine-bias-risk-assessments-in-criminal-sentencing
 
 ." href="/articles/10.1186/s13063-021-05489-x#ref-CR81" id="ref-link-section-d45421490e1699">81</a>]. Bias in clinical research–focused algorithms has the potential to be equally devastating, for instance, by theoretically suggesting non-representative study cohorts on the basis of a lower predicted participant drop-out.</p><h3 class="c-article__sub-heading" id="Sec18">Guidelines toward overcoming operational and philosophical barriers to ML in clinical research</h3><p>Because the operational problems previously detailed can potentiate the philosophical tangles of ML use in clinical research, many of the ways to overcome these hurdles overlap. The first and foremost approach to many of these issues includes data provenance, quality, and access. The open-access data sources previously discussed (MIMIC, UK Biobank) are good places to start, but inadequate on their own. Enhanced access to data and the technical expertise required to analyze it is needed. Attempts to render health data interoperable have been ongoing for decades, yielding data standard development initiatives and systems, such as the PCORnet Common Data Model [<a data-track="click" data-track-action="reference anchor" data-track-label="link" data-test="citation-ref" aria-label="Reference 82" title="Qualls LG, Phillips TA, Hammill BG, Topping J, Louzao DM, Brown JS, et al. Evaluating foundational data quality in the National Patient-Centered Clinical Research Network (PCORnet(R)). EGEMS (Wash DC). 2018;6(1):3." href="/articles/10.1186/s13063-021-05489-x#ref-CR82" id="ref-link-section-d45421490e1710">82</a>], FHIR [<a data-track="click" data-track-action="reference anchor" data-track-label="link" data-test="citation-ref" aria-label="Reference 83" title="Bosca D, Moner D, Maldonado JA, Robles M. Combining archetypes with fast health interoperability resources in future-proof health information systems. Stud Health Technol Inform. 2015;210:180–4." href="/articles/10.1186/s13063-021-05489-x#ref-CR83" id="ref-link-section-d45421490e1713">83</a>], i2b2 [<a data-track="click" data-track-action="reference anchor" data-track-label="link" data-test="citation-ref" aria-label="Reference 84" title="Klann JG, Abend A, Raghavan VA, Mandl KD, Murphy SN. Data interchange using i2b2. J Am Med Inform Assoc. 2016;23(5):909–15. 
 https://doi.org/10.1093/jamia/ocv188
 
 ." href="/articles/10.1186/s13063-021-05489-x#ref-CR84" id="ref-link-section-d45421490e1716">84</a>], and OMOP [<a data-track="click" data-track-action="reference anchor" data-track-label="link" data-test="citation-ref" aria-label="Reference 85" title="Overhage JM, Ryan PB, Reich CG, Hartzema AG, Stang PE. Validation of a common data model for active safety surveillance research. J Am Med Inform Assoc. 2012;19(1):54–60. 
 https://doi.org/10.1136/amiajnl-2011-000376
 
 ." href="/articles/10.1186/s13063-021-05489-x#ref-CR85" id="ref-link-section-d45421490e1719">85</a>]. Recently, regulation requiring health data interoperability through use of core data classes and elements has been enacted by the US Department of Health and Human Services and Centers for Medicare and Medicaid Services on the basis of the 21st Century Cures Act [<a data-track="click" data-track-action="reference anchor" data-track-label="link" data-test="citation-ref" aria-label="Reference 85" title="Overhage JM, Ryan PB, Reich CG, Hartzema AG, Stang PE. Validation of a common data model for active safety surveillance research. J Am Med Inform Assoc. 2012;19(1):54–60. 
 https://doi.org/10.1136/amiajnl-2011-000376
 
 ." href="/articles/10.1186/s13063-021-05489-x#ref-CR85" id="ref-link-section-d45421490e1722">85</a>, <a data-track="click" data-track-action="reference anchor" data-track-label="link" data-test="citation-ref" aria-label="Reference 86" title="21st Century Cures Act: Interoperability, information blocking, and the ONC Health IT Certification Program [updated 1 May 2020]. Available from: 
 https://www.federalregister.gov/documents/2020/05/01/2020-07419/21st-century-cures-act-interoperability-information-blocking-and-the-onc-health-it-certification
 
 . Accessed 16 May 2020." href="/articles/10.1186/s13063-021-05489-x#ref-CR86" id="ref-link-section-d45421490e1726">86</a>]. Where barriers to data sharing persist, other options to improve the amount of data available include federated data and cloud-based data access, in which developers can train and validate models on data that they do not own or directly interact with [<a data-track="click" data-track-action="reference anchor" data-track-label="link" data-test="citation-ref" title="Oh M, Park S, Kim S, Chae H. Machine learning-based analysis of multi-omics data on the cloud for investigating gene regulations. Brief Bioinform. 2020. Epub 2020/04/01. 
 https://doi.org/10.1093/bib/bbaa032
 
 ." href="#ref-CR87" id="ref-link-section-d45421490e1729">87</a>,<a data-track="click" data-track-action="reference anchor" data-track-label="link" data-test="citation-ref" title="Czeizler E, Wiessler W, Koester T, Hakala M, Basiri S, Jordan P, et al. Using federated data sources and Varian Learning Portal framework to train a neural network model for automatic organ segmentation. Phys Med. 2020;72:39–45. 
 https://doi.org/10.1016/j.ejmp.2020.03.011
 
 ." href="#ref-CR88" id="ref-link-section-d45421490e1729_1">88</a>,<a data-track="click" data-track-action="reference anchor" data-track-label="link" data-test="citation-ref" aria-label="Reference 89" title="Zerka F, Barakat S, Walsh S, Bogowicz M, Leijenaar RTH, Jochems A, et al. Systematic review of privacy-preserving distributed machine learning from federated databases in health care. JCO Clin Cancer Inform. 2020;4:184–200. 
 https://doi.org/10.1200/CCI.19.00047
 
 ." href="/articles/10.1186/s13063-021-05489-x#ref-CR89" id="ref-link-section-d45421490e1732">89</a>]. This has become increasingly common in certain fields, such as genomics and informatics, as evidenced by large consortia, such as eMERGE and OHDSI [<a data-track="click" data-track-action="reference anchor" data-track-label="link" data-test="citation-ref" aria-label="Reference 90" title="McCarty CA, Chisholm RL, Chute CG, Kullo IJ, Jarvik GP, Larson EB, et al. The eMERGE Network: a consortium of biorepositories linked to electronic medical records data for conducting genomic studies. BMC Med Genomics. 2011;4(1):13. 
 https://doi.org/10.1186/1755-8794-4-13
 
 ." href="/articles/10.1186/s13063-021-05489-x#ref-CR90" id="ref-link-section-d45421490e1735">90</a>, <a data-track="click" data-track-action="reference anchor" data-track-label="link" data-test="citation-ref" aria-label="Reference 91" title="Boyce RD, Ryan PB, Noren GN, Schuemie MJ, Reich C, Duke J, et al. Bridging islands of information to establish an integrated knowledge base of drugs and health outcomes of interest. Drug Saf. 2014;37(8):557–67. 
 https://doi.org/10.1007/s40264-014-0189-0
 
 ." href="/articles/10.1186/s13063-021-05489-x#ref-CR91" id="ref-link-section-d45421490e1738">91</a>].</p><p>Recently, a group of European universities and pharmaceutical companies have joined to create “MELODDY,” in which large amounts of drug development data will be shared while protecting companies’ proprietary information, though no academic publications have yet been produced [<a data-track="click" data-track-action="reference anchor" data-track-label="link" data-test="citation-ref" aria-label="Reference 91" title="Boyce RD, Ryan PB, Noren GN, Schuemie MJ, Reich C, Duke J, et al. Bridging islands of information to establish an integrated knowledge base of drugs and health outcomes of interest. Drug Saf. 2014;37(8):557–67. 
 https://doi.org/10.1007/s40264-014-0189-0
 
 ." href="/articles/10.1186/s13063-021-05489-x#ref-CR91" id="ref-link-section-d45421490e1744">91</a>]. “Challenges” in which teams compete to accomplish ML tasks often yield useful models, such as early sepsis prediction or more complete characterization of breast cancer cell lines, which can then be distributed to participating health institutions for validation in their local datasets [<a data-track="click" data-track-action="reference anchor" data-track-label="link" data-test="citation-ref" title="van Klaveren D, Steyerberg EW, Serruys PW, Kent DM. The proposed ‘concordance-statistic for benefit’ provided a useful metric when modeling heterogeneous treatment effects. J Clin Epidemiol. 2018;94:59–68. 
 https://doi.org/10.1016/j.jclinepi.2017.10.021
 
 ." href="#ref-CR92" id="ref-link-section-d45421490e1747">92</a>,<a data-track="click" data-track-action="reference anchor" data-track-label="link" data-test="citation-ref" title="Robbins RBE. An invisible hand: patients aren’t being told about the AI systems advising their care. STAT; 2020." href="#ref-CR93" id="ref-link-section-d45421490e1747_1">93</a>,<a data-track="click" data-track-action="reference anchor" data-track-label="link" data-test="citation-ref" title="Sterckx S, Rakic V, Cockbain J, Borry P. “You hoped we would sleep walk into accepting the collection of our data”: controversies surrounding the UK care.data scheme and their wider relevance for biomedical research. Med Health Care Philos. 2016;19(2):177–90. 
 https://doi.org/10.1007/s11019-015-9661-6
 
 ." href="#ref-CR94" id="ref-link-section-d45421490e1747_2">94</a>,<a data-track="click" data-track-action="reference anchor" data-track-label="link" data-test="citation-ref" aria-label="Reference 95" title="Committee on Understanding and Eliminating Racial and Ethnic Disparities in Health Care. Confronting racial and ethnic disparities in health care. Washington (DC): National Academies Press; 2003." href="/articles/10.1186/s13063-021-05489-x#ref-CR95" id="ref-link-section-d45421490e1750">95</a>].</p><p>Algorithm validation can both help ensure that ML models are appropriate for their intended clinical research use while also increasing stakeholder confidence in the use of ML in clinical research. Though the specifics continue to be debated, published best practices for specific use cases are emerging [<a data-track="click" data-track-action="reference anchor" data-track-label="link" data-test="citation-ref" aria-label="Reference 96" title="Criado PC. Invisible women. New York: Harry N. Abrams; 2019." href="/articles/10.1186/s13063-021-05489-x#ref-CR96" id="ref-link-section-d45421490e1756">96</a>]; recent suggestions to standardize such reporting in a one-page “model card” are notable [<a data-track="click" data-track-action="reference anchor" data-track-label="link" data-test="citation-ref" aria-label="Reference 97" title="Zhang H, Lu AX, Abdalla M, McDermott M, Ghassemi M. Hurtful words: quantifying biases in clinical contextual word embeddings. In: Proceedings of the ACM Conference on Health, Inference, and Learning. Toronto: Association for Computing Machinery; 2020. p. 110–20." href="/articles/10.1186/s13063-021-05489-x#ref-CR97" id="ref-link-section-d45421490e1759">97</a>]. For instance, possible model characteristics that could be reported include the intended use cohort, intended outcome of interest, required input data structure and necessary transformations, model type and structure, training cohort specifics, consequences of model application outside of intended use, and algorithm management of uncertainty. Performance metrics that are useful for algorithm evaluation in clinical contexts include receiver-operating characteristic and precision-recall curves, calibration, net benefit, and c-statistic for benefit [<a data-track="click" data-track-action="reference anchor" data-track-label="link" data-test="citation-ref" aria-label="Reference 92" title="van Klaveren D, Steyerberg EW, Serruys PW, Kent DM. The proposed ‘concordance-statistic for benefit’ provided a useful metric when modeling heterogeneous treatment effects. J Clin Epidemiol. 2018;94:59–68. 
 https://doi.org/10.1016/j.jclinepi.2017.10.021
 
 ." href="/articles/10.1186/s13063-021-05489-x#ref-CR92" id="ref-link-section-d45421490e1762">92</a>]. Depending on the intended use case, the most appropriate metrics to report or to optimize will differ. For instance, a model intended to identify patients at high risk for protocol non-adherence may have a higher tolerance for false-positives than one intended to simulate study drug dosages for trial planning. Consensus decisions about obligatory metrics for certain model structures and use cases are required to ensure that models with similar intended uses can be compared with one another. Developers will need to specify how often these metrics should be re-evaluated to assess for model drift. Ideally, evaluation of high-stakes clinical research models should be overseen by a neutral third party, such as a regulatory agency.</p><p>To foster trustworthiness even in the absence of explainability, it is essential that the model development and validation processes be <i>transparent</i>, including the reporting of model uncertainty. This may allow more advanced consumers to evaluate the model from a technical standpoint while at the very least helping less-advanced users to identify situations in which a model’s output should be approached with caution. For instance, understanding the source, structure, and drawbacks of the data used for model training and validation will provide insight into how the model’s output might be affected by the quality of the underlying data. However, trustworthiness may be built by running ML models in clinical research contexts in parallel with traditional research methods to show that the ML methods perform at least as well as traditional approaches. Though the importance of these principles may appear self-evident, the large number of ML models being used commercially for clinical research without reporting of the models’ development and performance characteristics suggests more work is needed to align stakeholders in this regard. Even while writing this manuscript, in which peer-reviewed publications were used whenever available, we encountered many cases in which the only “evidence” supporting a model’s performance was a commercial entity’s promotional material. In several other instances, the peer-reviewed articles available to support a commercial model’s performance offered no information at all about the model’s development or validation, which, as discussed earlier, is crucial to engendering trustworthiness. Another concerning aspect of commercial ML-enabled clinical research solutions is private companies’ and health care systems’ practice of training, validating, and applying models using patient data under the guise of quality improvement initiatives, thereby avoiding the need for ethical/institutional review board approval or patient consent [<a data-track="click" data-track-action="reference anchor" data-track-label="link" data-test="citation-ref" aria-label="Reference 93" title="Robbins RBE. An invisible hand: patients aren’t being told about the AI systems advising their care. STAT; 2020." href="/articles/10.1186/s13063-021-05489-x#ref-CR93" id="ref-link-section-d45421490e1771">93</a>]. This practice puts the entire field of ML development at risk of generating biased models and/or losing stakeholder buy-in (as occurred in dramatic fashion with the UK’s “Care.data” initiative) [<a data-track="click" data-track-action="reference anchor" data-track-label="link" data-test="citation-ref" aria-label="Reference 94" title="Sterckx S, Rakic V, Cockbain J, Borry P. “You hoped we would sleep walk into accepting the collection of our data”: controversies surrounding the UK care.data scheme and their wider relevance for biomedical research. Med Health Care Philos. 2016;19(2):177–90. 
 https://doi.org/10.1007/s11019-015-9661-6
 
 ." href="/articles/10.1186/s13063-021-05489-x#ref-CR94" id="ref-link-section-d45421490e1774">94</a>] and illustrates the need to build a more reasonable path toward ethical data sharing and more stringent processes surrounding model development and validation.</p><p>Although no FDA guidance is yet available specific to ML in clinical research, guidance on ML in clinical care and commentary from FDA representatives suggest several possible features of a regulatory approach to ML in clinical research. For instance, the FDA’s proposed ML-specific modifications to the “Software as a Medical Device” Regulations (SaMD) draw a distinction between fixed algorithms that were trained using ML techniques but frozen prior to deployment and those that continue to learn “in the wild.” These latter algorithms may more powerfully take advantage of the large amounts of data afforded by ongoing use but also pose additional risks of model drift with the potential need for iterative updates to the algorithm. In particular, model drift should often be <i>expected</i> because models that are incorporated into the decision-making process will inherently change the data they are exposed to in the future. The proposed ML-specific modifications to SaMD guidance outline an institution or organization-level approval pathway that would facilitate these ongoing algorithm updates within pre-approved boundaries (Fig. <a data-track="click" data-track-label="link" data-track-action="figure anchor" href="/articles/10.1186/s13063-021-05489-x#Fig3">3</a>). </p><div class="c-article-section__figure js-c-reading-companion-figures-item" data-test="figure" data-container-section="figure" id="figure-3" data-title="Fig. 3"><figure><figcaption><b id="Fig3" class="c-article-section__figure-caption" data-test="figure-caption-text">Fig. 3</b></figcaption><div class="c-article-section__figure-content"><div class="c-article-section__figure-item"><a class="c-article-section__figure-link" data-test="img-link" data-track="click" data-track-label="image" data-track-action="view figure" href="/articles/10.1186/s13063-021-05489-x/figures/3" rel="nofollow"><picture><source type="image/webp" srcset="//media.springernature.com/lw685/springer-static/image/art%3A10.1186%2Fs13063-021-05489-x/MediaObjects/13063_2021_5489_Fig3_HTML.png?as=webp"><img aria-describedby="Fig3" src="//media.springernature.com/lw685/springer-static/image/art%3A10.1186%2Fs13063-021-05489-x/MediaObjects/13063_2021_5489_Fig3_HTML.png" alt="figure 3" loading="lazy" width="685" height="456"></picture></a></div><div class="c-article-section__figure-description" data-test="bottom-caption" id="figure-3-desc"><p>FDA-proposed workflow to regulate machine learning algorithms under the Software as a Medical Device framework. From: Proposed regulatory framework for modifications to artificial intelligence/machine learning (AI/ML)-based software as a medical device: Discussion paper and request for feedback. <a href="https://www.fda.gov/media/122535/download">https://www.fda.gov/media/122535/download</a>. Accessed 17 May 2020</p></div></div><div class="u-text-right u-hide-print"><a class="c-article__pill-button" data-test="article-link" data-track="click" data-track-label="button" data-track-action="view figure" href="/articles/10.1186/s13063-021-05489-x/figures/3" data-track-dest="link:Figure3 Full size image" aria-label="Full size image figure 3" rel="nofollow"><span>Full size image</span><svg width="16" height="16" focusable="false" role="img" aria-hidden="true" class="u-icon"><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#icon-eds-i-chevron-right-small"></use></svg></a></div></figure></div><p>The optimal frequency of model re-evaluation by the FDA has yet to be determined (and may vary based off the model type, training set, and intended use), but clearly some form of recurrent review will be needed, prompted either by a certain time period, certain events (for instance, a global pandemic), or both. Discussion with representatives from the FDA indicates that ML in clinical research is viewed as a potentially high-risk use case due to the potential to propagate errors or biases through the algorithm into research studies; however, its potential opportunities were widely appreciated. Until formalized guidance about ML in clinical research is released, the FDA has clearly stated a willingness to work with sponsors and stakeholders on a case-by-case basis to determine the appropriate role of ML in research intended to support a regulatory application. However, this regulatory uncertainty could potentially stifle sponsors’ and stakeholders’ willingness to invest in ML for clinical research until guidance is drafted. This, in turn, may require additional work at a legislative level to provide a framework for further FDA guidance.</p><p>Concerns of bias are central to clinical research even when ML is not involved: clinical research and care have long histories of gender, racial, and socioeconomic bias [<a data-track="click" data-track-action="reference anchor" data-track-label="link" data-test="citation-ref" aria-label="Reference 95" title="Committee on Understanding and Eliminating Racial and Ethnic Disparities in Health Care. Confronting racial and ethnic disparities in health care. Washington (DC): National Academies Press; 2003." href="/articles/10.1186/s13063-021-05489-x#ref-CR95" id="ref-link-section-d45421490e1818">95</a>, <a data-track="click" data-track-action="reference anchor" data-track-label="link" data-test="citation-ref" aria-label="Reference 96" title="Criado PC. Invisible women. New York: Harry N. Abrams; 2019." href="/articles/10.1186/s13063-021-05489-x#ref-CR96" id="ref-link-section-d45421490e1821">96</a>]. The ability of ML to potentiate and perpetuate bias in clinical research, possibly without study teams’ awareness, must be actively managed. To the extent that bias can be identified, it can often be addressed and reduced; a worst-case scenario is application of a model with unknown bias in a new cohort with high-stakes results. As with much of ML in clinical research, data quality and quantity are critical in combating bias. No single perfect dataset exists, especially as models trained on real-world data will replicate the intentional or unintentional biases of the clinicians and researchers who generated those data [<a data-track="click" data-track-action="reference anchor" data-track-label="link" data-test="citation-ref" aria-label="Reference 97" title="Zhang H, Lu AX, Abdalla M, McDermott M, Ghassemi M. Hurtful words: quantifying biases in clinical contextual word embeddings. In: Proceedings of the ACM Conference on Health, Inference, and Learning. Toronto: Association for Computing Machinery; 2020. p. 110–20." href="/articles/10.1186/s13063-021-05489-x#ref-CR97" id="ref-link-section-d45421490e1824">97</a>]. Therefore, training models on more independent and diverse datasets decreases the likelihood of occult bias [<a data-track="click" data-track-action="reference anchor" data-track-label="link" data-test="citation-ref" aria-label="Reference 98" title="Chen IY, Joshi S, Ghassemi M. Treating health disparities with artificial intelligence. Nat Med. 2020;26(1):16–7. 
 https://doi.org/10.1038/s41591-019-0649-2
 
 ." href="/articles/10.1186/s13063-021-05489-x#ref-CR98" id="ref-link-section-d45421490e1827">98</a>]. Additionally, bias reduction can be approached through the model construction itself, such as by de-biasing word embeddings and using counterfactual fairness [<a data-track="click" data-track-action="reference anchor" data-track-label="link" data-test="citation-ref" title="Bolukbasi T, Chang K-W, Zou J, Saligrama V, Kalai A. Man is to computer programmer as woman is to homemaker? debiasing word embeddings. In: Proceedings of the 30th International Conference on Neural Information Processing Systems. Barcelona: Curran Associates Inc.; 2016. p. 4356–64." href="#ref-CR99" id="ref-link-section-d45421490e1830">99</a>,<a data-track="click" data-track-action="reference anchor" data-track-label="link" data-test="citation-ref" title="Kusner, Matt, Loftus, Joshua, Russell, Chris and Silva, Ricardo. Counterfactual fairness Conference. Proceedings of the 31st International Conference on Neural Information Processing Systems Conference. Long Beach, California, USA Publisher: Curran Associates Inc; 2017:4069–4079." href="#ref-CR100" id="ref-link-section-d45421490e1830_1">100</a>,<a data-track="click" data-track-action="reference anchor" data-track-label="link" data-test="citation-ref" title="Hardt M, Price E, Srebro N. Equality of opportunity in supervised learning. In: Proceedings of the 30th International Conference on Neural Information Processing Systems. Barcelona: Curran Associates Inc.; 2016. p. 3323–31." href="#ref-CR101" id="ref-link-section-d45421490e1830_2">101</a>,<a data-track="click" data-track-action="reference anchor" data-track-label="link" data-test="citation-ref" aria-label="Reference 102" title="Ustun B, Liu Y, Parkes D. Fairness without harm: decoupled classifiers with preference guarantees. In: Kamalika C, Ruslan S, editors. Proceedings of the 36th International Conference on Machine Learning; Proceedings of Machine Learning Research: PMLR %J Proceedings of Machine Learning Research; 2019. p. 6373–82." href="/articles/10.1186/s13063-021-05489-x#ref-CR102" id="ref-link-section-d45421490e1834">102</a>]. Clinical research teams may pre-specify certain subgroups of interest in which the algorithm must perform equally well [<a data-track="click" data-track-action="reference anchor" data-track-label="link" data-test="citation-ref" aria-label="Reference 103" title="Noseworthy PA, Attia ZI, Brewer LC, Hayes SN, Yao X, Kapa S, et al. Assessing and Mitigating bias in medical artificial intelligence: the effects of race and ethnicity on a deep learning model for ECG analysis. Circ Arrhythm Electrophysiol. 2020;13(3):e007988. 
 https://doi.org/10.1161/CIRCEP.119.007988
 
 ." href="/articles/10.1186/s13063-021-05489-x#ref-CR103" id="ref-link-section-d45421490e1837">103</a>]. Finally, while ML raises the specter of reinforcing and more efficiently operationalizing historical discrimination, ML may help us de-bias clinical research and care by monitoring and drawing attention to bias [<a data-track="click" data-track-action="reference anchor" data-track-label="link" data-test="citation-ref" aria-label="Reference 98" title="Chen IY, Joshi S, Ghassemi M. Treating health disparities with artificial intelligence. Nat Med. 2020;26(1):16–7. 
 https://doi.org/10.1038/s41591-019-0649-2
 
 ." href="/articles/10.1186/s13063-021-05489-x#ref-CR98" id="ref-link-section-d45421490e1840">98</a>]. Bias reduction is an area of ML in clinical research in which multi-disciplinary collaboration is especially vital and powerful: clinical scientists may be able to share perspective on long-standing biases in their domains of expertise, while more diverse teams may offer innovative insights into de-biasing ML models.</p></div></div></section><section data-title="Conclusion"><div class="c-article-section" id="Sec19-section"><h2 class="c-article-section__title js-section-title js-c-reading-companion-sections-item" id="Sec19">Conclusion</h2><div class="c-article-section__content" id="Sec19-content"><p>While traditional double-blinded, randomized, controlled clinical trials with their associated statistical methodologies remain the gold standard for biomedical evidence generation, augmentation with ML techniques offers the potential to improve the success and efficiency of clinical research, increasing its positive impact for all stakeholders. To the extent that ML-enabled clinical research can improve the efficiency and quality of biomedical evidence, it may save human lives and reduce human suffering, introducing an ethical imperative to explore this possibility. Realizing this potential will require overcoming issues with data structure and access, definitions of outcomes, transparency of development and validation processes, objectivity of certification, and the possibility of bias. The potential applications of ML to clinical research currently outstrip its actual use, both because few prospective studies are available about the relative effectiveness of ML versus traditional approaches and because change requires time, energy, and cooperation. Stakeholder willingness to integrate ML into clinical research relies in part on robust responses to issues of data provenance, bias, and validation as well as confidence in the regulatory structure surrounding ML in clinical research. The use of ML algorithms whose development has been opaque and without peer-reviewed publication must be addressed. The attendees of the January 2020 conference on ML in clinical research represent a broad swath of stakeholders with differing priorities and clinical research–related challenges, but all in attendance agreed that communication and collaboration are essential to implementation of this promising technology. Transparent discussion about the potential benefits and drawbacks of ML for clinical research and the sharing of best practices must continue not only in the academic community but in the lay press and government as well to ensure that ML in clinical research is applied in a fair, ethical, and open manner that is acceptable to all.</p></div></div></section> <section data-title="Availability of data and materials"><div class="c-article-section" id="availability-of-data-and-materials-section"><h2 class="c-article-section__title js-section-title js-c-reading-companion-sections-item" id="availability-of-data-and-materials">Availability of data and materials</h2><div class="c-article-section__content" id="availability-of-data-and-materials-content"> <p>Not applicable</p> </div></div></section><section data-title="Change history"><div class="c-article-section" id="change-history-section"><h2 class="c-article-section__title js-section-title js-c-reading-companion-sections-item" id="change-history">Change history</h2><div class="c-article-section__content" id="change-history-content"><ul class="c-article-change-list"><li class="c-article-change-list__item u-mb-24" id="chg1"><ins datetime="2021-09-06"><h3 class="c-article-change-list__heading u-h3 u-pr-8 u-display-inline">06 September 2021</h3><div class="c-article-change-list__details"><p>A Correction to this paper has been published: <a href="https://doi.org/10.1186/s13063-021-05571-4">https://doi.org/10.1186/s13063-021-05571-4</a></p></div></ins></li></ul></div></div></section><section data-title="Abbreviations"><div class="c-article-section" id="abbreviations-section"><h2 class="c-article-section__title js-section-title js-c-reading-companion-sections-item" id="abbreviations">Abbreviations</h2><div class="c-article-section__content" id="abbreviations-content"><dl class="c-abbreviation_list"><dt class="c-abbreviation_list__term u-text-bold u-float-left u-pr-16" style="min-width:50px;"><dfn>EHR:</dfn></dt><dd class="c-abbreviation_list__description u-mb-24"> <p>Electronic health record</p> </dd><dt class="c-abbreviation_list__term u-text-bold u-float-left u-pr-16" style="min-width:50px;"><dfn>FDA:</dfn></dt><dd class="c-abbreviation_list__description u-mb-24"> <p>US Food and Drug Administration</p> </dd><dt class="c-abbreviation_list__term u-text-bold u-float-left u-pr-16" style="min-width:50px;"><dfn>ML:</dfn></dt><dd class="c-abbreviation_list__description u-mb-24"> <p>Machine learning</p> </dd><dt class="c-abbreviation_list__term u-text-bold u-float-left u-pr-16" style="min-width:50px;"><dfn>NLP:</dfn></dt><dd class="c-abbreviation_list__description u-mb-24"> <p>Natural language processing</p> </dd><dt class="c-abbreviation_list__term u-text-bold u-float-left u-pr-16" style="min-width:50px;"><dfn>SaMD:</dfn></dt><dd class="c-abbreviation_list__description u-mb-24"> <p>Software as a Medical Device</p> </dd></dl></div></div></section><div id="MagazineFulltextArticleBodySuffix"><section aria-labelledby="Bib1" data-title="References"><div class="c-article-section" id="Bib1-section"><h2 class="c-article-section__title js-section-title js-c-reading-companion-sections-item" id="Bib1">References</h2><div class="c-article-section__content" id="Bib1-content"><div data-container-section="references"><ol class="c-article-references" data-track-component="outbound reference" data-track-context="references section"><li class="c-article-references__item js-c-reading-companion-references-item" data-counter="1."><p class="c-article-references__text" id="ref-CR1">Senior AW, Evans R, Jumper J, Kirkpatrick J, Sifre L, Green T, et al. Improved protein structure prediction using potentials from deep learning. Nature. 2020;577(7792):706–10. <a href="https://doi.org/10.1038/s41586-019-1923-7" data-track="click_references" data-track-action="external reference" data-track-value="external reference" data-track-label="10.1038/s41586-019-1923-7">https://doi.org/10.1038/s41586-019-1923-7</a>.</p><p class="c-article-references__links u-hide-print"><a data-track="click_references" rel="nofollow noopener" data-track-label="10.1038/s41586-019-1923-7" data-track-item_id="10.1038/s41586-019-1923-7" data-track-value="article reference" data-track-action="article reference" href="https://doi.org/10.1038%2Fs41586-019-1923-7" aria-label="Article reference 1" data-doi="10.1038/s41586-019-1923-7">Article</a> <a data-track="click_references" rel="nofollow noopener" data-track-label="link" data-track-item_id="link" data-track-value="pubmed reference" data-track-action="pubmed reference" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Abstract&list_uids=31942072" aria-label="PubMed reference 1">PubMed</a> <a data-track="click_references" rel="nofollow noopener" data-track-label="link" data-track-item_id="link" data-track-value="cas reference" data-track-action="cas reference" href="/articles/cas-redirect/1:CAS:528:DC%2BB3cXis1SisL0%3D" aria-label="CAS reference 1">CAS</a> <a data-track="click_references" data-track-action="google scholar reference" data-track-value="google scholar reference" data-track-label="link" data-track-item_id="link" rel="nofollow noopener" aria-label="Google Scholar reference 1" href="http://scholar.google.com/scholar_lookup?&title=Improved%20protein%20structure%20prediction%20using%20potentials%20from%20deep%20learning&journal=Nature.&doi=10.1038%2Fs41586-019-1923-7&volume=577&issue=7792&pages=706-710&publication_year=2020&author=Senior%2CAW&author=Evans%2CR&author=Jumper%2CJ&author=Kirkpatrick%2CJ&author=Sifre%2CL&author=Green%2CT&author=Qin%2CC&author=%C5%BD%C3%ADdek%2CA&author=Nelson%2CAWR&author=Bridgland%2CA&author=Penedones%2CH&author=Petersen%2CS&author=Simonyan%2CK&author=Crossan%2CS&author=Kohli%2CP&author=Jones%2CDT&author=Silver%2CD&author=Kavukcuoglu%2CK&author=Hassabis%2CD"> Google Scholar</a> </p></li><li class="c-article-references__item js-c-reading-companion-references-item" data-counter="2."><p class="c-article-references__text" id="ref-CR2">Fauqueur JTA, Togia T. Constructing large scale biomedical knowledge bases from scratch with rapid annotation of interpretable patterns. In: Proceedings of the 18th BioNLP Workshop and Shared Task; 2019. <a href="https://doi.org/10.18653/v1/w19-5016" data-track="click_references" data-track-action="external reference" data-track-value="external reference" data-track-label="10.18653/v1/w19-5016">https://doi.org/10.18653/v1/w19-5016</a>.</p><p class="c-article-references__links u-hide-print"><a data-track="click_references" rel="nofollow noopener" data-track-label="10.18653/v1/w19-5016" data-track-item_id="10.18653/v1/w19-5016" data-track-value="chapter reference" data-track-action="chapter reference" href="https://doi.org/10.18653%2Fv1%2Fw19-5016" aria-label="Chapter reference 2" data-doi="10.18653/v1/w19-5016">Chapter</a> <a data-track="click_references" data-track-action="google scholar reference" data-track-value="google scholar reference" data-track-label="link" data-track-item_id="link" rel="nofollow noopener" aria-label="Google Scholar reference 2" href="http://scholar.google.com/scholar_lookup?&title=Constructing%20large%20scale%20biomedical%20knowledge%20bases%20from%20scratch%20with%20rapid%20annotation%20of%20interpretable%20patterns&doi=10.18653%2Fv1%2Fw19-5016&publication_year=2019&author=Fauqueur%2CJTA&author=Togia%2CT"> Google Scholar</a> </p></li><li class="c-article-references__item js-c-reading-companion-references-item" data-counter="3."><p class="c-article-references__text" id="ref-CR3">Jia R, Wong C, Poon H. Document-level N-ary relation extraction with multiscale representation learning. Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers); 2019; Minneapolis: Association for Computational Linguistics. <a href="https://ui.adsabs.harvard.edu/abs/2019arXiv190402347J" data-track="click_references" data-track-action="external reference" data-track-value="external reference" data-track-label="https://ui.adsabs.harvard.edu/abs/2019arXiv190402347J">https://ui.adsabs.harvard.edu/abs/2019arXiv190402347J</a>.</p></li><li class="c-article-references__item js-c-reading-companion-references-item" data-counter="4."><p class="c-article-references__text" id="ref-CR4">Dezso Z, Ceccarelli M. Machine learning prediction of oncology drug targets based on protein and network properties. BMC Bioinformatics. 2020;21(1):104. <a href="https://doi.org/10.1186/s12859-020-3442-9" data-track="click_references" data-track-action="external reference" data-track-value="external reference" data-track-label="10.1186/s12859-020-3442-9">https://doi.org/10.1186/s12859-020-3442-9</a>.</p><p class="c-article-references__links u-hide-print"><a data-track="click_references" rel="noopener" data-track-label="10.1186/s12859-020-3442-9" data-track-item_id="10.1186/s12859-020-3442-9" data-track-value="article reference" data-track-action="article reference" href="https://link.springer.com/doi/10.1186/s12859-020-3442-9" aria-label="Article reference 4" data-doi="10.1186/s12859-020-3442-9">Article</a> <a data-track="click_references" rel="nofollow noopener" data-track-label="link" data-track-item_id="link" data-track-value="pubmed reference" data-track-action="pubmed reference" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Abstract&list_uids=32171238" aria-label="PubMed reference 4">PubMed</a> <a data-track="click_references" rel="nofollow noopener" data-track-label="link" data-track-item_id="link" data-track-value="pubmed central reference" data-track-action="pubmed central reference" href="http://www.ncbi.nlm.nih.gov/pmc/articles/PMC7071582" aria-label="PubMed Central reference 4">PubMed Central</a> <a data-track="click_references" rel="nofollow noopener" data-track-label="link" data-track-item_id="link" data-track-value="cas reference" data-track-action="cas reference" href="/articles/cas-redirect/1:CAS:528:DC%2BB3cXltFeqsr0%3D" aria-label="CAS reference 4">CAS</a> <a data-track="click_references" data-track-action="google scholar reference" data-track-value="google scholar reference" data-track-label="link" data-track-item_id="link" rel="nofollow noopener" aria-label="Google Scholar reference 4" href="http://scholar.google.com/scholar_lookup?&title=Machine%20learning%20prediction%20of%20oncology%20drug%20targets%20based%20on%20protein%20and%20network%20properties&journal=BMC%20Bioinformatics.&doi=10.1186%2Fs12859-020-3442-9&volume=21&issue=1&publication_year=2020&author=Dezso%2CZ&author=Ceccarelli%2CM"> Google Scholar</a> </p></li><li class="c-article-references__item js-c-reading-companion-references-item" data-counter="5."><p class="c-article-references__text" id="ref-CR5">Bagherian M, Sabeti E, Wang K, Sartor MA, Nikolovska-Coleska Z, Najarian K. Machine learning approaches and databases for prediction of drug-target interaction: a survey paper. Brief Bioinform. 2021;22(1):247–69. <a href="https://doi.org/10.1093/bib/bbz157" data-track="click_references" data-track-action="external reference" data-track-value="external reference" data-track-label="10.1093/bib/bbz157">https://doi.org/10.1093/bib/bbz157</a>.</p></li><li class="c-article-references__item js-c-reading-companion-references-item" data-counter="6."><p class="c-article-references__text" id="ref-CR6">Liu QAM, Brockschmidt M, Gaunt AL. Constrained graph variational autoencoders for molecule design. NeurIPS 2018. 2018;arXiv:1805.09076:7806–15.</p></li><li class="c-article-references__item js-c-reading-companion-references-item" data-counter="7."><p class="c-article-references__text" id="ref-CR7">Madhukar NS, Khade PK, Huang L, Gayvert K, Galletti G, Stogniew M, et al. A Bayesian machine learning approach for drug target identification using diverse data types. Nat Commun. 2019;10(1):5221. <a href="https://doi.org/10.1038/s41467-019-12928-6" data-track="click_references" data-track-action="external reference" data-track-value="external reference" data-track-label="10.1038/s41467-019-12928-6">https://doi.org/10.1038/s41467-019-12928-6</a>.</p><p class="c-article-references__links u-hide-print"><a data-track="click_references" rel="nofollow noopener" data-track-label="10.1038/s41467-019-12928-6" data-track-item_id="10.1038/s41467-019-12928-6" data-track-value="article reference" data-track-action="article reference" href="https://doi.org/10.1038%2Fs41467-019-12928-6" aria-label="Article reference 7" data-doi="10.1038/s41467-019-12928-6">Article</a> <a data-track="click_references" rel="nofollow noopener" data-track-label="link" data-track-item_id="link" data-track-value="pubmed reference" data-track-action="pubmed reference" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Abstract&list_uids=31745082" aria-label="PubMed reference 7">PubMed</a> <a data-track="click_references" rel="nofollow noopener" data-track-label="link" data-track-item_id="link" data-track-value="pubmed central reference" data-track-action="pubmed central reference" href="http://www.ncbi.nlm.nih.gov/pmc/articles/PMC6863850" aria-label="PubMed Central reference 7">PubMed Central</a> <a data-track="click_references" rel="nofollow noopener" data-track-label="link" data-track-item_id="link" data-track-value="cas reference" data-track-action="cas reference" href="/articles/cas-redirect/1:CAS:528:DC%2BC1MXit1ams7vP" aria-label="CAS reference 7">CAS</a> <a data-track="click_references" data-track-action="google scholar reference" data-track-value="google scholar reference" data-track-label="link" data-track-item_id="link" rel="nofollow noopener" aria-label="Google Scholar reference 7" href="http://scholar.google.com/scholar_lookup?&title=A%20Bayesian%20machine%20learning%20approach%20for%20drug%20target%20identification%20using%20diverse%20data%20types&journal=Nat%20Commun.&doi=10.1038%2Fs41467-019-12928-6&volume=10&issue=1&publication_year=2019&author=Madhukar%2CNS&author=Khade%2CPK&author=Huang%2CL&author=Gayvert%2CK&author=Galletti%2CG&author=Stogniew%2CM&author=Allen%2CJE&author=Giannakakou%2CP&author=Elemento%2CO"> Google Scholar</a> </p></li><li class="c-article-references__item js-c-reading-companion-references-item" data-counter="8."><p class="c-article-references__text" id="ref-CR8">Langner S, Hase F, Perea JD, Stubhan T, Hauch J, Roch LM, et al. Beyond ternary OPV: high-throughput experimentation and self-driving laboratories optimize multicomponent systems. Adv Mater. 2020;32(14):e1907801. <a href="https://doi.org/10.1002/adma.201907801" data-track="click_references" data-track-action="external reference" data-track-value="external reference" data-track-label="10.1002/adma.201907801">https://doi.org/10.1002/adma.201907801</a>.</p><p class="c-article-references__links u-hide-print"><a data-track="click_references" rel="nofollow noopener" data-track-label="10.1002/adma.201907801" data-track-item_id="10.1002/adma.201907801" data-track-value="article reference" data-track-action="article reference" href="https://doi.org/10.1002%2Fadma.201907801" aria-label="Article reference 8" data-doi="10.1002/adma.201907801">Article</a> <a data-track="click_references" rel="nofollow noopener" data-track-label="link" data-track-item_id="link" data-track-value="pubmed reference" data-track-action="pubmed reference" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Abstract&list_uids=32049386" aria-label="PubMed reference 8">PubMed</a> <a data-track="click_references" rel="nofollow noopener" data-track-label="link" data-track-item_id="link" data-track-value="cas reference" data-track-action="cas reference" href="/articles/cas-redirect/1:CAS:528:DC%2BB3cXivFygsro%3D" aria-label="CAS reference 8">CAS</a> <a data-track="click_references" data-track-action="google scholar reference" data-track-value="google scholar reference" data-track-label="link" data-track-item_id="link" rel="nofollow noopener" aria-label="Google Scholar reference 8" href="http://scholar.google.com/scholar_lookup?&title=Beyond%20ternary%20OPV%3A%20high-throughput%20experimentation%20and%20self-driving%20laboratories%20optimize%20multicomponent%20systems&journal=Adv%20Mater.&doi=10.1002%2Fadma.201907801&volume=32&issue=14&publication_year=2020&author=Langner%2CS&author=Hase%2CF&author=Perea%2CJD&author=Stubhan%2CT&author=Hauch%2CJ&author=Roch%2CLM"> Google Scholar</a> </p></li><li class="c-article-references__item js-c-reading-companion-references-item" data-counter="9."><p class="c-article-references__text" id="ref-CR9">Granda JM, Donina L, Dragone V, Long DL, Cronin L. Controlling an organic synthesis robot with machine learning to search for new reactivity. Nature. 2018;559(7714):377–81. <a href="https://doi.org/10.1038/s41586-018-0307-8" data-track="click_references" data-track-action="external reference" data-track-value="external reference" data-track-label="10.1038/s41586-018-0307-8">https://doi.org/10.1038/s41586-018-0307-8</a>.</p><p class="c-article-references__links u-hide-print"><a data-track="click_references" rel="nofollow noopener" data-track-label="10.1038/s41586-018-0307-8" data-track-item_id="10.1038/s41586-018-0307-8" data-track-value="article reference" data-track-action="article reference" href="https://doi.org/10.1038%2Fs41586-018-0307-8" aria-label="Article reference 9" data-doi="10.1038/s41586-018-0307-8">Article</a> <a data-track="click_references" rel="nofollow noopener" data-track-label="link" data-track-item_id="link" data-track-value="pubmed reference" data-track-action="pubmed reference" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Abstract&list_uids=30022133" aria-label="PubMed reference 9">PubMed</a> <a data-track="click_references" rel="nofollow noopener" data-track-label="link" data-track-item_id="link" data-track-value="pubmed central reference" data-track-action="pubmed central reference" href="http://www.ncbi.nlm.nih.gov/pmc/articles/PMC6223543" aria-label="PubMed Central reference 9">PubMed Central</a> <a data-track="click_references" rel="nofollow noopener" data-track-label="link" data-track-item_id="link" data-track-value="cas reference" data-track-action="cas reference" href="/articles/cas-redirect/1:CAS:528:DC%2BC1cXhtlClsb%2FJ" aria-label="CAS reference 9">CAS</a> <a data-track="click_references" data-track-action="google scholar reference" data-track-value="google scholar reference" data-track-label="link" data-track-item_id="link" rel="nofollow noopener" aria-label="Google Scholar reference 9" href="http://scholar.google.com/scholar_lookup?&title=Controlling%20an%20organic%20synthesis%20robot%20with%20machine%20learning%20to%20search%20for%20new%20reactivity&journal=Nature.&doi=10.1038%2Fs41586-018-0307-8&volume=559&issue=7714&pages=377-381&publication_year=2018&author=Granda%2CJM&author=Donina%2CL&author=Dragone%2CV&author=Long%2CDL&author=Cronin%2CL"> Google Scholar</a> </p></li><li class="c-article-references__item js-c-reading-companion-references-item" data-counter="10."><p class="c-article-references__text" id="ref-CR10">Koh D. Sumitomo Dainippon Pharma and Exscientia achieve breakthrough in AI drug discovery: Healthcare IT News - Portland, ME: Healthcare IT News; 2020.</p></li><li class="c-article-references__item js-c-reading-companion-references-item" data-counter="11."><p class="c-article-references__text" id="ref-CR11">Romero K, Ito K, Rogers JA, Polhamus D, Qiu R, Stephenson D, et al. The future is now: model-based clinical trial design for Alzheimer's disease. Clin Pharmacol Ther. 2015;97(3):210–4. <a href="https://doi.org/10.1002/cpt.16" data-track="click_references" data-track-action="external reference" data-track-value="external reference" data-track-label="10.1002/cpt.16">https://doi.org/10.1002/cpt.16</a>.</p><p class="c-article-references__links u-hide-print"><a data-track="click_references" rel="nofollow noopener" data-track-label="10.1002/cpt.16" data-track-item_id="10.1002/cpt.16" data-track-value="article reference" data-track-action="article reference" href="https://doi.org/10.1002%2Fcpt.16" aria-label="Article reference 11" data-doi="10.1002/cpt.16">Article</a> <a data-track="click_references" rel="nofollow noopener" data-track-label="link" data-track-item_id="link" data-track-value="pubmed reference" data-track-action="pubmed reference" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Abstract&list_uids=25669145" aria-label="PubMed reference 11">PubMed</a> <a data-track="click_references" rel="nofollow noopener" data-track-label="link" data-track-item_id="link" data-track-value="cas reference" data-track-action="cas reference" href="/articles/cas-redirect/1:CAS:528:DC%2BC2MXkvFCksr8%3D" aria-label="CAS reference 11">CAS</a> <a data-track="click_references" data-track-action="google scholar reference" data-track-value="google scholar reference" data-track-label="link" data-track-item_id="link" rel="nofollow noopener" aria-label="Google Scholar reference 11" href="http://scholar.google.com/scholar_lookup?&title=The%20future%20is%20now%3A%20model-based%20clinical%20trial%20design%20for%20Alzheimer%27s%20disease&journal=Clin%20Pharmacol%20Ther.&doi=10.1002%2Fcpt.16&volume=97&issue=3&pages=210-214&publication_year=2015&author=Romero%2CK&author=Ito%2CK&author=Rogers%2CJA&author=Polhamus%2CD&author=Qiu%2CR&author=Stephenson%2CD&author=Mohs%2CR&author=Lalonde%2CR&author=Sinha%2CV&author=Wang%2CY&author=Brown%2CD&author=Isaac%2CM&author=Vamvakas%2CS&author=Hemmings%2CR&author=Pani%2CL&author=Bain%2CLJ&author=Corrigan%2CB"> Google Scholar</a> </p></li><li class="c-article-references__item js-c-reading-companion-references-item" data-counter="12."><p class="c-article-references__text" id="ref-CR12">Zhao Y, Zeng D, Socinski MA, Kosorok MR. Reinforcement learning strategies for clinical trials in nonsmall cell lung cancer. Biometrics. 2011;67(4):1422–33. <a href="https://doi.org/10.1111/j.1541-0420.2011.01572.x" data-track="click_references" data-track-action="external reference" data-track-value="external reference" data-track-label="10.1111/j.1541-0420.2011.01572.x">https://doi.org/10.1111/j.1541-0420.2011.01572.x</a>.</p><p class="c-article-references__links u-hide-print"><a data-track="click_references" rel="nofollow noopener" data-track-label="10.1111/j.1541-0420.2011.01572.x" data-track-item_id="10.1111/j.1541-0420.2011.01572.x" data-track-value="article reference" data-track-action="article reference" href="https://doi.org/10.1111%2Fj.1541-0420.2011.01572.x" aria-label="Article reference 12" data-doi="10.1111/j.1541-0420.2011.01572.x">Article</a> <a data-track="click_references" rel="nofollow noopener" data-track-label="link" data-track-item_id="link" data-track-value="pubmed reference" data-track-action="pubmed reference" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Abstract&list_uids=21385164" aria-label="PubMed reference 12">PubMed</a> <a data-track="click_references" rel="nofollow noopener" data-track-label="link" data-track-item_id="link" data-track-value="pubmed central reference" data-track-action="pubmed central reference" href="http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3138840" aria-label="PubMed Central reference 12">PubMed Central</a> <a data-track="click_references" data-track-action="google scholar reference" data-track-value="google scholar reference" data-track-label="link" data-track-item_id="link" rel="nofollow noopener" aria-label="Google Scholar reference 12" href="http://scholar.google.com/scholar_lookup?&title=Reinforcement%20learning%20strategies%20for%20clinical%20trials%20in%20nonsmall%20cell%20lung%20cancer&journal=Biometrics.&doi=10.1111%2Fj.1541-0420.2011.01572.x&volume=67&issue=4&pages=1422-1433&publication_year=2011&author=Zhao%2CY&author=Zeng%2CD&author=Socinski%2CMA&author=Kosorok%2CMR"> Google Scholar</a> </p></li><li class="c-article-references__item js-c-reading-companion-references-item" data-counter="13."><p class="c-article-references__text" id="ref-CR13">trials.ai 2019 [cited 2021 February 2]. Available from: <a href="http://trials.ai" data-track="click_references" data-track-action="external reference" data-track-value="external reference" data-track-label="http://trials.ai">trials.ai</a>.</p></li><li class="c-article-references__item js-c-reading-companion-references-item" data-counter="14."><p class="c-article-references__text" id="ref-CR14">Wong CH, Siah KW, Lo AW. Estimation of clinical trial success rates and related parameters. Biostatistics. 2019;20(2):273–86. <a href="https://doi.org/10.1093/biostatistics/kxx069" data-track="click_references" data-track-action="external reference" data-track-value="external reference" data-track-label="10.1093/biostatistics/kxx069">https://doi.org/10.1093/biostatistics/kxx069</a>.</p><p class="c-article-references__links u-hide-print"><a data-track="click_references" rel="nofollow noopener" data-track-label="10.1093/biostatistics/kxx069" data-track-item_id="10.1093/biostatistics/kxx069" data-track-value="article reference" data-track-action="article reference" href="https://doi.org/10.1093%2Fbiostatistics%2Fkxx069" aria-label="Article reference 14" data-doi="10.1093/biostatistics/kxx069">Article</a> <a data-track="click_references" rel="nofollow noopener" data-track-label="link" data-track-item_id="link" data-track-value="pubmed reference" data-track-action="pubmed reference" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Abstract&list_uids=29394327" aria-label="PubMed reference 14">PubMed</a> <a data-track="click_references" data-track-action="google scholar reference" data-track-value="google scholar reference" data-track-label="link" data-track-item_id="link" rel="nofollow noopener" aria-label="Google Scholar reference 14" href="http://scholar.google.com/scholar_lookup?&title=Estimation%20of%20clinical%20trial%20success%20rates%20and%20related%20parameters&journal=Biostatistics.&doi=10.1093%2Fbiostatistics%2Fkxx069&volume=20&issue=2&pages=273-286&publication_year=2019&author=Wong%2CCH&author=Siah%2CKW&author=Lo%2CAW"> Google Scholar</a> </p></li><li class="c-article-references__item js-c-reading-companion-references-item" data-counter="15."><p class="c-article-references__text" id="ref-CR15">Schork NJ. Personalized medicine: time for one-person trials. Nature. 2015;520(7549):609–11. <a href="https://doi.org/10.1038/520609a" data-track="click_references" data-track-action="external reference" data-track-value="external reference" data-track-label="10.1038/520609a">https://doi.org/10.1038/520609a</a>.</p><p class="c-article-references__links u-hide-print"><a data-track="click_references" rel="nofollow noopener" data-track-label="10.1038/520609a" data-track-item_id="10.1038/520609a" data-track-value="article reference" data-track-action="article reference" href="https://doi.org/10.1038%2F520609a" aria-label="Article reference 15" data-doi="10.1038/520609a">Article</a> <a data-track="click_references" rel="nofollow noopener" data-track-label="link" data-track-item_id="link" data-track-value="pubmed reference" data-track-action="pubmed reference" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Abstract&list_uids=25925459" aria-label="PubMed reference 15">PubMed</a> <a data-track="click_references" rel="nofollow noopener" data-track-label="link" data-track-item_id="link" data-track-value="cas reference" data-track-action="cas reference" href="/articles/cas-redirect/1:CAS:528:DC%2BC2MXnsVWqsLk%3D" aria-label="CAS reference 15">CAS</a> <a data-track="click_references" data-track-action="google scholar reference" data-track-value="google scholar reference" data-track-label="link" data-track-item_id="link" rel="nofollow noopener" aria-label="Google Scholar reference 15" href="http://scholar.google.com/scholar_lookup?&title=Personalized%20medicine%3A%20time%20for%20one-person%20trials&journal=Nature.&doi=10.1038%2F520609a&volume=520&issue=7549&pages=609-611&publication_year=2015&author=Schork%2CNJ"> Google Scholar</a> </p></li><li class="c-article-references__item js-c-reading-companion-references-item" data-counter="16."><p class="c-article-references__text" id="ref-CR16">Glicksberg BS, Miotto R, Johnson KW, Shameer K, Li L, Chen R, et al. Automated disease cohort selection using word embeddings from electronic health records. Pac Symp Biocomput. 2018;23:145–56.</p><p class="c-article-references__links u-hide-print"><a data-track="click_references" rel="nofollow noopener" data-track-label="link" data-track-item_id="link" data-track-value="pubmed reference" data-track-action="pubmed reference" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Abstract&list_uids=29218877" aria-label="PubMed reference 16">PubMed</a> <a data-track="click_references" rel="nofollow noopener" data-track-label="link" data-track-item_id="link" data-track-value="pubmed central reference" data-track-action="pubmed central reference" href="http://www.ncbi.nlm.nih.gov/pmc/articles/PMC5788312" aria-label="PubMed Central reference 16">PubMed Central</a> <a data-track="click_references" data-track-action="google scholar reference" data-track-value="google scholar reference" data-track-label="link" data-track-item_id="link" rel="nofollow noopener" aria-label="Google Scholar reference 16" href="http://scholar.google.com/scholar_lookup?&title=Automated%20disease%20cohort%20selection%20using%20word%20embeddings%20from%20electronic%20health%20records&journal=Pac%20Symp%20Biocomput.&volume=23&pages=145-156&publication_year=2018&author=Glicksberg%2CBS&author=Miotto%2CR&author=Johnson%2CKW&author=Shameer%2CK&author=Li%2CL&author=Chen%2CR&author=Dudley%2CJT"> Google Scholar</a> </p></li><li class="c-article-references__item js-c-reading-companion-references-item" data-counter="17."><p class="c-article-references__text" id="ref-CR17">Liao KP, Cai T, Savova GK, Murphy SN, Karlson EW, Ananthakrishnan AN, et al. Development of phenotype algorithms using electronic medical records and incorporating natural language processing. BMJ. 2015;350(apr24 11):h1885. <a href="https://doi.org/10.1136/bmj.h1885" data-track="click_references" data-track-action="external reference" data-track-value="external reference" data-track-label="10.1136/bmj.h1885">https://doi.org/10.1136/bmj.h1885</a>.</p><p class="c-article-references__links u-hide-print"><a data-track="click_references" rel="nofollow noopener" data-track-label="10.1136/bmj.h1885" data-track-item_id="10.1136/bmj.h1885" data-track-value="article reference" data-track-action="article reference" href="https://doi.org/10.1136%2Fbmj.h1885" aria-label="Article reference 17" data-doi="10.1136/bmj.h1885">Article</a> <a data-track="click_references" rel="nofollow noopener" data-track-label="link" data-track-item_id="link" data-track-value="pubmed reference" data-track-action="pubmed reference" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Abstract&list_uids=25911572" aria-label="PubMed reference 17">PubMed</a> <a data-track="click_references" rel="nofollow noopener" data-track-label="link" data-track-item_id="link" data-track-value="pubmed central reference" data-track-action="pubmed central reference" href="http://www.ncbi.nlm.nih.gov/pmc/articles/PMC4707569" aria-label="PubMed Central reference 17">PubMed Central</a> <a data-track="click_references" data-track-action="google scholar reference" data-track-value="google scholar reference" data-track-label="link" data-track-item_id="link" rel="nofollow noopener" aria-label="Google Scholar reference 17" href="http://scholar.google.com/scholar_lookup?&title=Development%20of%20phenotype%20algorithms%20using%20electronic%20medical%20records%20and%20incorporating%20natural%20language%20processing&journal=BMJ.&doi=10.1136%2Fbmj.h1885&volume=350&issue=apr24%2011&publication_year=2015&author=Liao%2CKP&author=Cai%2CT&author=Savova%2CGK&author=Murphy%2CSN&author=Karlson%2CEW&author=Ananthakrishnan%2CAN&author=Gainer%2CVS&author=Shaw%2CSY&author=Xia%2CZ&author=Szolovits%2CP&author=Churchill%2CS&author=Kohane%2CI"> Google Scholar</a> </p></li><li class="c-article-references__item js-c-reading-companion-references-item" data-counter="18."><p class="c-article-references__text" id="ref-CR18">Li L, Cheng WY, Glicksberg BS, Gottesman O, Tamler R, Chen R, et al. Identification of type 2 diabetes subgroups through topological analysis of patient similarity. Sci Transl Med. 2015;7(311):311ra174.</p><p class="c-article-references__links u-hide-print"><a data-track="click_references" rel="nofollow noopener" data-track-label="10.1126/scitranslmed.aaa9364" data-track-item_id="10.1126/scitranslmed.aaa9364" data-track-value="article reference" data-track-action="article reference" href="https://doi.org/10.1126%2Fscitranslmed.aaa9364" aria-label="Article reference 18" data-doi="10.1126/scitranslmed.aaa9364">Article</a> <a data-track="click_references" rel="nofollow noopener" data-track-label="link" data-track-item_id="link" data-track-value="cas reference" data-track-action="cas reference" href="/articles/cas-redirect/1:CAS:528:DC%2BC28XmvV2htrs%3D" aria-label="CAS reference 18">CAS</a> <a data-track="click_references" data-track-action="google scholar reference" data-track-value="google scholar reference" data-track-label="link" data-track-item_id="link" rel="nofollow noopener" aria-label="Google Scholar reference 18" href="http://scholar.google.com/scholar_lookup?&title=Identification%20of%20type%202%20diabetes%20subgroups%20through%20topological%20analysis%20of%20patient%20similarity&journal=Sci%20Transl%20Med&doi=10.1126%2Fscitranslmed.aaa9364&volume=7&issue=311&publication_year=2015&author=Li%2CL&author=Cheng%2CWY&author=Glicksberg%2CBS&author=Gottesman%2CO&author=Tamler%2CR&author=Chen%2CR"> Google Scholar</a> </p></li><li class="c-article-references__item js-c-reading-companion-references-item" data-counter="19."><p class="c-article-references__text" id="ref-CR19">Our Solution 2021 [cited 2021 February 2]. Available from: <a href="https://www.bullfrogai.com/our-solution/" data-track="click_references" data-track-action="external reference" data-track-value="external reference" data-track-label="https://www.bullfrogai.com/our-solution/">https://www.bullfrogai.com/our-solution/</a>.</p></li><li class="c-article-references__item js-c-reading-companion-references-item" data-counter="20."><p class="c-article-references__text" id="ref-CR20">Zhang X, Xiao C, Glass LM, Sun J. DeepEnroll: patient-trial matching with deep embedding and entailment prediction. In: Proceedings of the Web Conference 2020. Taipei: Association for Computing Machinery; 2020. p. 1029–37.</p><p class="c-article-references__links u-hide-print"><a data-track="click_references" rel="nofollow noopener" data-track-label="10.1145/3366423.3380181" data-track-item_id="10.1145/3366423.3380181" data-track-value="chapter reference" data-track-action="chapter reference" href="https://doi.org/10.1145%2F3366423.3380181" aria-label="Chapter reference 20" data-doi="10.1145/3366423.3380181">Chapter</a> <a data-track="click_references" data-track-action="google scholar reference" data-track-value="google scholar reference" data-track-label="link" data-track-item_id="link" rel="nofollow noopener" aria-label="Google Scholar reference 20" href="http://scholar.google.com/scholar_lookup?&title=DeepEnroll%3A%20patient-trial%20matching%20with%20deep%20embedding%20and%20entailment%20prediction&doi=10.1145%2F3366423.3380181&pages=1029-1037&publication_year=2020&author=Zhang%2CX&author=Xiao%2CC&author=Glass%2CLM&author=Sun%2CJ"> Google Scholar</a> </p></li><li class="c-article-references__item js-c-reading-companion-references-item" data-counter="21."><p class="c-article-references__text" id="ref-CR21">Calaprice-Whitty D, Galil K, Salloum W, Zariv A, Jimenez B. Improving clinical trial participant prescreening with artificial intelligence (AI): a comparison of the results of AI-assisted vs standard methods in 3 oncology trials. Ther Innov Regul Sci. 2020;54(1):69–74. <a href="https://doi.org/10.1007/s43441-019-00030-4" data-track="click_references" data-track-action="external reference" data-track-value="external reference" data-track-label="10.1007/s43441-019-00030-4">https://doi.org/10.1007/s43441-019-00030-4</a>.</p><p class="c-article-references__links u-hide-print"><a data-track="click_references" rel="noopener" data-track-label="10.1007/s43441-019-00030-4" data-track-item_id="10.1007/s43441-019-00030-4" data-track-value="article reference" data-track-action="article reference" href="https://link.springer.com/doi/10.1007/s43441-019-00030-4" aria-label="Article reference 21" data-doi="10.1007/s43441-019-00030-4">Article</a> <a data-track="click_references" rel="nofollow noopener" data-track-label="link" data-track-item_id="link" data-track-value="pubmed reference" data-track-action="pubmed reference" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Abstract&list_uids=32008227" aria-label="PubMed reference 21">PubMed</a> <a data-track="click_references" data-track-action="google scholar reference" data-track-value="google scholar reference" data-track-label="link" data-track-item_id="link" rel="nofollow noopener" aria-label="Google Scholar reference 21" href="http://scholar.google.com/scholar_lookup?&title=Improving%20clinical%20trial%20participant%20prescreening%20with%20artificial%20intelligence%20%28AI%29%3A%20a%20comparison%20of%20the%20results%20of%20AI-assisted%20vs%20standard%20methods%20in%203%20oncology%20trials&journal=Ther%20Innov%20Regul%20Sci.&doi=10.1007%2Fs43441-019-00030-4&volume=54&issue=1&pages=69-74&publication_year=2020&author=Calaprice-Whitty%2CD&author=Galil%2CK&author=Salloum%2CW&author=Zariv%2CA&author=Jimenez%2CB"> Google Scholar</a> </p></li><li class="c-article-references__item js-c-reading-companion-references-item" data-counter="22."><p class="c-article-references__text" id="ref-CR22">How it works 2019 [cited 2021 February 2]. Available from: <a href="https://deep6.ai/how-it-works/" data-track="click_references" data-track-action="external reference" data-track-value="external reference" data-track-label="https://deep6.ai/how-it-works/">https://deep6.ai/how-it-works/</a>.</p></li><li class="c-article-references__item js-c-reading-companion-references-item" data-counter="23."><p class="c-article-references__text" id="ref-CR23">Vassy JL, Ho YL, Honerlaw J, Cho K, Gaziano JM, Wilson PWF, et al. Yield and bias in defining a cohort study baseline from electronic health record data. J Biomed Inform. 2018;78:54–9. <a href="https://doi.org/10.1016/j.jbi.2017.12.017" data-track="click_references" data-track-action="external reference" data-track-value="external reference" data-track-label="10.1016/j.jbi.2017.12.017">https://doi.org/10.1016/j.jbi.2017.12.017</a>.</p><p class="c-article-references__links u-hide-print"><a data-track="click_references" rel="nofollow noopener" data-track-label="10.1016/j.jbi.2017.12.017" data-track-item_id="10.1016/j.jbi.2017.12.017" data-track-value="article reference" data-track-action="article reference" href="https://doi.org/10.1016%2Fj.jbi.2017.12.017" aria-label="Article reference 23" data-doi="10.1016/j.jbi.2017.12.017">Article</a> <a data-track="click_references" rel="nofollow noopener" data-track-label="link" data-track-item_id="link" data-track-value="pubmed reference" data-track-action="pubmed reference" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Abstract&list_uids=29305952" aria-label="PubMed reference 23">PubMed</a> <a data-track="click_references" rel="nofollow noopener" data-track-label="link" data-track-item_id="link" data-track-value="pubmed central reference" data-track-action="pubmed central reference" href="http://www.ncbi.nlm.nih.gov/pmc/articles/PMC5846098" aria-label="PubMed Central reference 23">PubMed Central</a> <a data-track="click_references" data-track-action="google scholar reference" data-track-value="google scholar reference" data-track-label="link" data-track-item_id="link" rel="nofollow noopener" aria-label="Google Scholar reference 23" href="http://scholar.google.com/scholar_lookup?&title=Yield%20and%20bias%20in%20defining%20a%20cohort%20study%20baseline%20from%20electronic%20health%20record%20data&journal=J%20Biomed%20Inform.&doi=10.1016%2Fj.jbi.2017.12.017&volume=78&pages=54-59&publication_year=2018&author=Vassy%2CJL&author=Ho%2CYL&author=Honerlaw%2CJ&author=Cho%2CK&author=Gaziano%2CJM&author=Wilson%2CPWF&author=Gagnon%2CDR"> Google Scholar</a> </p></li><li class="c-article-references__item js-c-reading-companion-references-item" data-counter="24."><p class="c-article-references__text" id="ref-CR24">Weber GM, Adams WG, Bernstam EV, Bickel JP, Fox KP, Marsolo K, et al. Biases introduced by filtering electronic health records for patients with “complete data”. J Am Med Inform Assoc. 2017;24(6):1134–41. <a href="https://doi.org/10.1093/jamia/ocx071" data-track="click_references" data-track-action="external reference" data-track-value="external reference" data-track-label="10.1093/jamia/ocx071">https://doi.org/10.1093/jamia/ocx071</a>.</p><p class="c-article-references__links u-hide-print"><a data-track="click_references" rel="nofollow noopener" data-track-label="10.1093/jamia/ocx071" data-track-item_id="10.1093/jamia/ocx071" data-track-value="article reference" data-track-action="article reference" href="https://doi.org/10.1093%2Fjamia%2Focx071" aria-label="Article reference 24" data-doi="10.1093/jamia/ocx071">Article</a> <a data-track="click_references" rel="nofollow noopener" data-track-label="link" data-track-item_id="link" data-track-value="pubmed reference" data-track-action="pubmed reference" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Abstract&list_uids=29016972" aria-label="PubMed reference 24">PubMed</a> <a data-track="click_references" rel="nofollow noopener" data-track-label="link" data-track-item_id="link" data-track-value="pubmed central reference" data-track-action="pubmed central reference" href="http://www.ncbi.nlm.nih.gov/pmc/articles/PMC6080680" aria-label="PubMed Central reference 24">PubMed Central</a> <a data-track="click_references" data-track-action="google scholar reference" data-track-value="google scholar reference" data-track-label="link" data-track-item_id="link" rel="nofollow noopener" aria-label="Google Scholar reference 24" href="http://scholar.google.com/scholar_lookup?&title=Biases%20introduced%20by%20filtering%20electronic%20health%20records%20for%20patients%20with%20%E2%80%9Ccomplete%20data%E2%80%9D&journal=J%20Am%20Med%20Inform%20Assoc.&doi=10.1093%2Fjamia%2Focx071&volume=24&issue=6&pages=1134-1141&publication_year=2017&author=Weber%2CGM&author=Adams%2CWG&author=Bernstam%2CEV&author=Bickel%2CJP&author=Fox%2CKP&author=Marsolo%2CK&author=Raghavan%2CVA&author=Turchin%2CA&author=Zhou%2CX&author=Murphy%2CSN&author=Mandl%2CKD"> Google Scholar</a> </p></li><li class="c-article-references__item js-c-reading-companion-references-item" data-counter="25."><p class="c-article-references__text" id="ref-CR25">Bain EE, Shafner L, Walling DP, Othman AA, Chuang-Stein C, Hinkle J, et al. Use of a novel artificial intelligence platform on mobile devices to assess dosing compliance in a phase 2 clinical trial in subjects with schizophrenia. JMIR Mhealth Uhealth. 2017;5(2):e18. <a href="https://doi.org/10.2196/mhealth.7030" data-track="click_references" data-track-action="external reference" data-track-value="external reference" data-track-label="10.2196/mhealth.7030">https://doi.org/10.2196/mhealth.7030</a>.</p><p class="c-article-references__links u-hide-print"><a data-track="click_references" rel="nofollow noopener" data-track-label="10.2196/mhealth.7030" data-track-item_id="10.2196/mhealth.7030" data-track-value="article reference" data-track-action="article reference" href="https://doi.org/10.2196%2Fmhealth.7030" aria-label="Article reference 25" data-doi="10.2196/mhealth.7030">Article</a> <a data-track="click_references" rel="nofollow noopener" data-track-label="link" data-track-item_id="link" data-track-value="pubmed reference" data-track-action="pubmed reference" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Abstract&list_uids=28223265" aria-label="PubMed reference 25">PubMed</a> <a data-track="click_references" rel="nofollow noopener" data-track-label="link" data-track-item_id="link" data-track-value="pubmed central reference" data-track-action="pubmed central reference" href="http://www.ncbi.nlm.nih.gov/pmc/articles/PMC5340925" aria-label="PubMed Central reference 25">PubMed Central</a> <a data-track="click_references" data-track-action="google scholar reference" data-track-value="google scholar reference" data-track-label="link" data-track-item_id="link" rel="nofollow noopener" aria-label="Google Scholar reference 25" href="http://scholar.google.com/scholar_lookup?&title=Use%20of%20a%20novel%20artificial%20intelligence%20platform%20on%20mobile%20devices%20to%20assess%20dosing%20compliance%20in%20a%20phase%202%20clinical%20trial%20in%20subjects%20with%20schizophrenia&journal=JMIR%20Mhealth%20Uhealth.&doi=10.2196%2Fmhealth.7030&volume=5&issue=2&publication_year=2017&author=Bain%2CEE&author=Shafner%2CL&author=Walling%2CDP&author=Othman%2CAA&author=Chuang-Stein%2CC&author=Hinkle%2CJ&author=Hanina%2CA"> Google Scholar</a> </p></li><li class="c-article-references__item js-c-reading-companion-references-item" data-counter="26."><p class="c-article-references__text" id="ref-CR26">Labovitz DL, Shafner L, Reyes Gil M, Virmani D, Hanina A. Using artificial intelligence to reduce the risk of nonadherence in patients on anticoagulation therapy. Stroke. 2017;48(5):1416–9. <a href="https://doi.org/10.1161/STROKEAHA.116.016281" data-track="click_references" data-track-action="external reference" data-track-value="external reference" data-track-label="10.1161/STROKEAHA.116.016281">https://doi.org/10.1161/STROKEAHA.116.016281</a>.</p><p class="c-article-references__links u-hide-print"><a data-track="click_references" rel="nofollow noopener" data-track-label="10.1161/STROKEAHA.116.016281" data-track-item_id="10.1161/STROKEAHA.116.016281" data-track-value="article reference" data-track-action="article reference" href="https://doi.org/10.1161%2FSTROKEAHA.116.016281" aria-label="Article reference 26" data-doi="10.1161/STROKEAHA.116.016281">Article</a> <a data-track="click_references" rel="nofollow noopener" data-track-label="link" data-track-item_id="link" data-track-value="pubmed reference" data-track-action="pubmed reference" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Abstract&list_uids=28386037" aria-label="PubMed reference 26">PubMed</a> <a data-track="click_references" rel="nofollow noopener" data-track-label="link" data-track-item_id="link" data-track-value="pubmed central reference" data-track-action="pubmed central reference" href="http://www.ncbi.nlm.nih.gov/pmc/articles/PMC5432369" aria-label="PubMed Central reference 26">PubMed Central</a> <a data-track="click_references" data-track-action="google scholar reference" data-track-value="google scholar reference" data-track-label="link" data-track-item_id="link" rel="nofollow noopener" aria-label="Google Scholar reference 26" href="http://scholar.google.com/scholar_lookup?&title=Using%20artificial%20intelligence%20to%20reduce%20the%20risk%20of%20nonadherence%20in%20patients%20on%20anticoagulation%20therapy&journal=Stroke.&doi=10.1161%2FSTROKEAHA.116.016281&volume=48&issue=5&pages=1416-1419&publication_year=2017&author=Labovitz%2CDL&author=Shafner%2CL&author=Reyes%20Gil%2CM&author=Virmani%2CD&author=Hanina%2CA"> Google Scholar</a> </p></li><li class="c-article-references__item js-c-reading-companion-references-item" data-counter="27."><p class="c-article-references__text" id="ref-CR27">Adamson AS, Smith A. Machine learning and health care disparities in dermatology. JAMA Dermatol. 2018;154(11):1247–8. <a href="https://doi.org/10.1001/jamadermatol.2018.2348" data-track="click_references" data-track-action="external reference" data-track-value="external reference" data-track-label="10.1001/jamadermatol.2018.2348">https://doi.org/10.1001/jamadermatol.2018.2348</a>.</p><p class="c-article-references__links u-hide-print"><a data-track="click_references" rel="nofollow noopener" data-track-label="10.1001/jamadermatol.2018.2348" data-track-item_id="10.1001/jamadermatol.2018.2348" data-track-value="article reference" data-track-action="article reference" href="https://doi.org/10.1001%2Fjamadermatol.2018.2348" aria-label="Article reference 27" data-doi="10.1001/jamadermatol.2018.2348">Article</a> <a data-track="click_references" rel="nofollow noopener" data-track-label="link" data-track-item_id="link" data-track-value="pubmed reference" data-track-action="pubmed reference" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Abstract&list_uids=30073260" aria-label="PubMed reference 27">PubMed</a> <a data-track="click_references" data-track-action="google scholar reference" data-track-value="google scholar reference" data-track-label="link" data-track-item_id="link" rel="nofollow noopener" aria-label="Google Scholar reference 27" href="http://scholar.google.com/scholar_lookup?&title=Machine%20learning%20and%20health%20care%20disparities%20in%20dermatology&journal=JAMA%20Dermatol.&doi=10.1001%2Fjamadermatol.2018.2348&volume=154&issue=11&pages=1247-1248&publication_year=2018&author=Adamson%2CAS&author=Smith%2CA"> Google Scholar</a> </p></li><li class="c-article-references__item js-c-reading-companion-references-item" data-counter="28."><p class="c-article-references__text" id="ref-CR28">Burlingame EA, Margolin AA, Gray JW, Chang YH. SHIFT: speedy histopathological-to-immunofluorescent translation of whole slide images using conditional generative adversarial networks. Proc SPIE Int Soc Opt Eng. 2018;10581. <a href="https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6166432/" data-track="click_references" data-track-action="external reference" data-track-value="external reference" data-track-label="https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6166432/">https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6166432/</a>.</p></li><li class="c-article-references__item js-c-reading-companion-references-item" data-counter="29."><p class="c-article-references__text" id="ref-CR29">Han J, Chen K, Fang L, Zhang S, Wang F, Ma H, et al. Improving the efficacy of the data entry process for clinical research with a natural language processing-driven medical information extraction system: quantitative field research. JMIR Med Inform. 2019;7(3):e13331. <a href="https://doi.org/10.2196/13331" data-track="click_references" data-track-action="external reference" data-track-value="external reference" data-track-label="10.2196/13331">https://doi.org/10.2196/13331</a>.</p><p class="c-article-references__links u-hide-print"><a data-track="click_references" rel="nofollow noopener" data-track-label="10.2196/13331" data-track-item_id="10.2196/13331" data-track-value="article reference" data-track-action="article reference" href="https://doi.org/10.2196%2F13331" aria-label="Article reference 29" data-doi="10.2196/13331">Article</a> <a data-track="click_references" rel="nofollow noopener" data-track-label="link" data-track-item_id="link" data-track-value="pubmed reference" data-track-action="pubmed reference" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Abstract&list_uids=31313661" aria-label="PubMed reference 29">PubMed</a> <a data-track="click_references" rel="nofollow noopener" data-track-label="link" data-track-item_id="link" data-track-value="pubmed central reference" data-track-action="pubmed central reference" href="http://www.ncbi.nlm.nih.gov/pmc/articles/PMC6672807" aria-label="PubMed Central reference 29">PubMed Central</a> <a data-track="click_references" data-track-action="google scholar reference" data-track-value="google scholar reference" data-track-label="link" data-track-item_id="link" rel="nofollow noopener" aria-label="Google Scholar reference 29" href="http://scholar.google.com/scholar_lookup?&title=Improving%20the%20efficacy%20of%20the%20data%20entry%20process%20for%20clinical%20research%20with%20a%20natural%20language%20processing-driven%20medical%20information%20extraction%20system%3A%20quantitative%20field%20research&journal=JMIR%20Med%20Inform.&doi=10.2196%2F13331&volume=7&issue=3&publication_year=2019&author=Han%2CJ&author=Chen%2CK&author=Fang%2CL&author=Zhang%2CS&author=Wang%2CF&author=Ma%2CH&author=Zhao%2CL&author=Liu%2CS"> Google Scholar</a> </p></li><li class="c-article-references__item js-c-reading-companion-references-item" data-counter="30."><p class="c-article-references__text" id="ref-CR30">Fonferko-Shadrach B, Lacey AS, Roberts A, Akbari A, Thompson S, Ford DV, et al. Using natural language processing to extract structured epilepsy data from unstructured clinic letters: development and validation of the ExECT (extraction of epilepsy clinical text) system. BMJ Open. 2019;9(4):e023232. <a href="https://doi.org/10.1136/bmjopen-2018-023232" data-track="click_references" data-track-action="external reference" data-track-value="external reference" data-track-label="10.1136/bmjopen-2018-023232">https://doi.org/10.1136/bmjopen-2018-023232</a>.</p><p class="c-article-references__links u-hide-print"><a data-track="click_references" rel="nofollow noopener" data-track-label="10.1136/bmjopen-2018-023232" data-track-item_id="10.1136/bmjopen-2018-023232" data-track-value="article reference" data-track-action="article reference" href="https://doi.org/10.1136%2Fbmjopen-2018-023232" aria-label="Article reference 30" data-doi="10.1136/bmjopen-2018-023232">Article</a> <a data-track="click_references" rel="nofollow noopener" data-track-label="link" data-track-item_id="link" data-track-value="pubmed reference" data-track-action="pubmed reference" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Abstract&list_uids=30940752" aria-label="PubMed reference 30">PubMed</a> <a data-track="click_references" rel="nofollow noopener" data-track-label="link" data-track-item_id="link" data-track-value="pubmed central reference" data-track-action="pubmed central reference" href="http://www.ncbi.nlm.nih.gov/pmc/articles/PMC6500195" aria-label="PubMed Central reference 30">PubMed Central</a> <a data-track="click_references" data-track-action="google scholar reference" data-track-value="google scholar reference" data-track-label="link" data-track-item_id="link" rel="nofollow noopener" aria-label="Google Scholar reference 30" href="http://scholar.google.com/scholar_lookup?&title=Using%20natural%20language%20processing%20to%20extract%20structured%20epilepsy%20data%20from%20unstructured%20clinic%20letters%3A%20development%20and%20validation%20of%20the%20ExECT%20%28extraction%20of%20epilepsy%20clinical%20text%29%20system&journal=BMJ%20Open.&doi=10.1136%2Fbmjopen-2018-023232&volume=9&issue=4&publication_year=2019&author=Fonferko-Shadrach%2CB&author=Lacey%2CAS&author=Roberts%2CA&author=Akbari%2CA&author=Thompson%2CS&author=Ford%2CDV&author=Lyons%2CRA&author=Rees%2CMI&author=Pickrell%2CWO"> Google Scholar</a> </p></li><li class="c-article-references__item js-c-reading-companion-references-item" data-counter="31."><p class="c-article-references__text" id="ref-CR31">Gavrielov-Yusim N, Kurzinger ML, Nishikawa C, Pan C, Pouget J, Epstein LB, et al. Comparison of text processing methods in social media-based signal detection. Pharmacoepidemiol Drug Saf. 2019;28(10):1309–17. <a href="https://doi.org/10.1002/pds.4857" data-track="click_references" data-track-action="external reference" data-track-value="external reference" data-track-label="10.1002/pds.4857">https://doi.org/10.1002/pds.4857</a>.</p><p class="c-article-references__links u-hide-print"><a data-track="click_references" rel="nofollow noopener" data-track-label="10.1002/pds.4857" data-track-item_id="10.1002/pds.4857" data-track-value="article reference" data-track-action="article reference" href="https://doi.org/10.1002%2Fpds.4857" aria-label="Article reference 31" data-doi="10.1002/pds.4857">Article</a> <a data-track="click_references" rel="nofollow noopener" data-track-label="link" data-track-item_id="link" data-track-value="pubmed reference" data-track-action="pubmed reference" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Abstract&list_uids=31392844" aria-label="PubMed reference 31">PubMed</a> <a data-track="click_references" data-track-action="google scholar reference" data-track-value="google scholar reference" data-track-label="link" data-track-item_id="link" rel="nofollow noopener" aria-label="Google Scholar reference 31" href="http://scholar.google.com/scholar_lookup?&title=Comparison%20of%20text%20processing%20methods%20in%20social%20media-based%20signal%20detection&journal=Pharmacoepidemiol%20Drug%20Saf.&doi=10.1002%2Fpds.4857&volume=28&issue=10&pages=1309-1317&publication_year=2019&author=Gavrielov-Yusim%2CN&author=Kurzinger%2CML&author=Nishikawa%2CC&author=Pan%2CC&author=Pouget%2CJ&author=Epstein%2CLB"> Google Scholar</a> </p></li><li class="c-article-references__item js-c-reading-companion-references-item" data-counter="32."><p class="c-article-references__text" id="ref-CR32">Barnett I, Torous J, Staples P, Sandoval L, Keshavan M, Onnela JP. Relapse prediction in schizophrenia through digital phenotyping: a pilot study. Neuropsychopharmacology. 2018;43(8):1660–6. <a href="https://doi.org/10.1038/s41386-018-0030-z" data-track="click_references" data-track-action="external reference" data-track-value="external reference" data-track-label="10.1038/s41386-018-0030-z">https://doi.org/10.1038/s41386-018-0030-z</a>.</p><p class="c-article-references__links u-hide-print"><a data-track="click_references" rel="nofollow noopener" data-track-label="10.1038/s41386-018-0030-z" data-track-item_id="10.1038/s41386-018-0030-z" data-track-value="article reference" data-track-action="article reference" href="https://doi.org/10.1038%2Fs41386-018-0030-z" aria-label="Article reference 32" data-doi="10.1038/s41386-018-0030-z">Article</a> <a data-track="click_references" rel="nofollow noopener" data-track-label="link" data-track-item_id="link" data-track-value="pubmed reference" data-track-action="pubmed reference" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Abstract&list_uids=29511333" aria-label="PubMed reference 32">PubMed</a> <a data-track="click_references" rel="nofollow noopener" data-track-label="link" data-track-item_id="link" data-track-value="pubmed central reference" data-track-action="pubmed central reference" href="http://www.ncbi.nlm.nih.gov/pmc/articles/PMC6006347" aria-label="PubMed Central reference 32">PubMed Central</a> <a data-track="click_references" data-track-action="google scholar reference" data-track-value="google scholar reference" data-track-label="link" data-track-item_id="link" rel="nofollow noopener" aria-label="Google Scholar reference 32" href="http://scholar.google.com/scholar_lookup?&title=Relapse%20prediction%20in%20schizophrenia%20through%20digital%20phenotyping%3A%20a%20pilot%20study&journal=Neuropsychopharmacology.&doi=10.1038%2Fs41386-018-0030-z&volume=43&issue=8&pages=1660-1666&publication_year=2018&author=Barnett%2CI&author=Torous%2CJ&author=Staples%2CP&author=Sandoval%2CL&author=Keshavan%2CM&author=Onnela%2CJP"> Google Scholar</a> </p></li><li class="c-article-references__item js-c-reading-companion-references-item" data-counter="33."><p class="c-article-references__text" id="ref-CR33">Chaudhuri S, Oudejans D, Thompson HJ, Demiris G. Real-world accuracy and use of a wearable fall detection device by older adults. J Am Geriatr Soc. 2015;63(11):2415–6. <a href="https://doi.org/10.1111/jgs.13804" data-track="click_references" data-track-action="external reference" data-track-value="external reference" data-track-label="10.1111/jgs.13804">https://doi.org/10.1111/jgs.13804</a>.</p><p class="c-article-references__links u-hide-print"><a data-track="click_references" rel="nofollow noopener" data-track-label="10.1111/jgs.13804" data-track-item_id="10.1111/jgs.13804" data-track-value="article reference" data-track-action="article reference" href="https://doi.org/10.1111%2Fjgs.13804" aria-label="Article reference 33" data-doi="10.1111/jgs.13804">Article</a> <a data-track="click_references" rel="nofollow noopener" data-track-label="link" data-track-item_id="link" data-track-value="pubmed reference" data-track-action="pubmed reference" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Abstract&list_uids=26603067" aria-label="PubMed reference 33">PubMed</a> <a data-track="click_references" rel="nofollow noopener" data-track-label="link" data-track-item_id="link" data-track-value="pubmed central reference" data-track-action="pubmed central reference" href="http://www.ncbi.nlm.nih.gov/pmc/articles/PMC4662041" aria-label="PubMed Central reference 33">PubMed Central</a> <a data-track="click_references" data-track-action="google scholar reference" data-track-value="google scholar reference" data-track-label="link" data-track-item_id="link" rel="nofollow noopener" aria-label="Google Scholar reference 33" href="http://scholar.google.com/scholar_lookup?&title=Real-world%20accuracy%20and%20use%20of%20a%20wearable%20fall%20detection%20device%20by%20older%20adults&journal=J%20Am%20Geriatr%20Soc.&doi=10.1111%2Fjgs.13804&volume=63&issue=11&pages=2415-2416&publication_year=2015&author=Chaudhuri%2CS&author=Oudejans%2CD&author=Thompson%2CHJ&author=Demiris%2CG"> Google Scholar</a> </p></li><li class="c-article-references__item js-c-reading-companion-references-item" data-counter="34."><p class="c-article-references__text" id="ref-CR34">Chen R, Jankovic F, Marinsek N, Foschini L, Kourtis L, Signorini A, et al. Developing measures of cognitive impairment in the real world from consumer-grade multimodal sensor streams. In: Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. Anchorage: Association for Computing Machinery; 2019. p. 2145–55.</p><p class="c-article-references__links u-hide-print"><a data-track="click_references" rel="nofollow noopener" data-track-label="10.1145/3292500.3330690" data-track-item_id="10.1145/3292500.3330690" data-track-value="chapter reference" data-track-action="chapter reference" href="https://doi.org/10.1145%2F3292500.3330690" aria-label="Chapter reference 34" data-doi="10.1145/3292500.3330690">Chapter</a> <a data-track="click_references" data-track-action="google scholar reference" data-track-value="google scholar reference" data-track-label="link" data-track-item_id="link" rel="nofollow noopener" aria-label="Google Scholar reference 34" href="http://scholar.google.com/scholar_lookup?&title=Developing%20measures%20of%20cognitive%20impairment%20in%20the%20real%20world%20from%20consumer-grade%20multimodal%20sensor%20streams&doi=10.1145%2F3292500.3330690&pages=2145-2155&publication_year=2019&author=Chen%2CR&author=Jankovic%2CF&author=Marinsek%2CN&author=Foschini%2CL&author=Kourtis%2CL&author=Signorini%2CA"> Google Scholar</a> </p></li><li class="c-article-references__item js-c-reading-companion-references-item" data-counter="35."><p class="c-article-references__text" id="ref-CR35">Yurtman A, Barshan B, Fidan B. Activity recognition invariant to wearable sensor unit orientation using differential rotational transformations represented by quaternions. Sensors (Basel). 2018;18(8):2725. <a href="https://pubmed.ncbi.nlm.nih.gov/30126235/" data-track="click_references" data-track-action="external reference" data-track-value="external reference" data-track-label="https://pubmed.ncbi.nlm.nih.gov/30126235/">https://pubmed.ncbi.nlm.nih.gov/30126235/</a>.</p></li><li class="c-article-references__item js-c-reading-companion-references-item" data-counter="36."><p class="c-article-references__text" id="ref-CR36">Lu K, Yang L, Seoane F, Abtahi F, Forsman M, Lindecrantz K. Fusion of heart rate, respiration and motion measurements from a wearable sensor system to enhance energy expenditure estimation. Sensors (Basel). 2018;18(9):3092. <a href="https://pubmed.ncbi.nlm.nih.gov/30223429/" data-track="click_references" data-track-action="external reference" data-track-value="external reference" data-track-label="https://pubmed.ncbi.nlm.nih.gov/30223429/">https://pubmed.ncbi.nlm.nih.gov/30223429/</a>.</p></li><li class="c-article-references__item js-c-reading-companion-references-item" data-counter="37."><p class="c-article-references__text" id="ref-CR37">Cheung YK, Hsueh PS, Ensari I, Willey JZ, Diaz KM. Quantile coarsening analysis of high-volume wearable activity data in a longitudinal observational study. Sensors (Basel). 2018;18(9):3056. <a href="https://pubmed.ncbi.nlm.nih.gov/30213093/" data-track="click_references" data-track-action="external reference" data-track-value="external reference" data-track-label="https://pubmed.ncbi.nlm.nih.gov/30213093/">https://pubmed.ncbi.nlm.nih.gov/30213093/</a>.</p></li><li class="c-article-references__item js-c-reading-companion-references-item" data-counter="38."><p class="c-article-references__text" id="ref-CR38">Hannun AY, Rajpurkar P, Haghpanahi M, Tison GH, Bourn C, Turakhia MP, et al. Cardiologist-level arrhythmia detection and classification in ambulatory electrocardiograms using a deep neural network. Nat Med. 2019;25(1):65–9. <a href="https://doi.org/10.1038/s41591-018-0268-3" data-track="click_references" data-track-action="external reference" data-track-value="external reference" data-track-label="10.1038/s41591-018-0268-3">https://doi.org/10.1038/s41591-018-0268-3</a>.</p><p class="c-article-references__links u-hide-print"><a data-track="click_references" rel="nofollow noopener" data-track-label="10.1038/s41591-018-0268-3" data-track-item_id="10.1038/s41591-018-0268-3" data-track-value="article reference" data-track-action="article reference" href="https://doi.org/10.1038%2Fs41591-018-0268-3" aria-label="Article reference 38" data-doi="10.1038/s41591-018-0268-3">Article</a> <a data-track="click_references" rel="nofollow noopener" data-track-label="link" data-track-item_id="link" data-track-value="pubmed reference" data-track-action="pubmed reference" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Abstract&list_uids=30617320" aria-label="PubMed reference 38">PubMed</a> <a data-track="click_references" rel="nofollow noopener" data-track-label="link" data-track-item_id="link" data-track-value="pubmed central reference" data-track-action="pubmed central reference" href="http://www.ncbi.nlm.nih.gov/pmc/articles/PMC6784839" aria-label="PubMed Central reference 38">PubMed Central</a> <a data-track="click_references" rel="nofollow noopener" data-track-label="link" data-track-item_id="link" data-track-value="cas reference" data-track-action="cas reference" href="/articles/cas-redirect/1:CAS:528:DC%2BC1MXmvVOgsLs%3D" aria-label="CAS reference 38">CAS</a> <a data-track="click_references" data-track-action="google scholar reference" data-track-value="google scholar reference" data-track-label="link" data-track-item_id="link" rel="nofollow noopener" aria-label="Google Scholar reference 38" href="http://scholar.google.com/scholar_lookup?&title=Cardiologist-level%20arrhythmia%20detection%20and%20classification%20in%20ambulatory%20electrocardiograms%20using%20a%20deep%20neural%20network&journal=Nat%20Med.&doi=10.1038%2Fs41591-018-0268-3&volume=25&issue=1&pages=65-69&publication_year=2019&author=Hannun%2CAY&author=Rajpurkar%2CP&author=Haghpanahi%2CM&author=Tison%2CGH&author=Bourn%2CC&author=Turakhia%2CMP&author=Ng%2CAY"> Google Scholar</a> </p></li><li class="c-article-references__item js-c-reading-companion-references-item" data-counter="39."><p class="c-article-references__text" id="ref-CR39">Ozkanca Y, Ozturk MG, Ekmekci MN, Atkins DC, Demiroglu C, Ghomi RH. Depression screening from voice samples of patients affected by Parkinson's disease. Digit Biomark. 2019;3(2):72–82. <a href="https://doi.org/10.1159/000500354" data-track="click_references" data-track-action="external reference" data-track-value="external reference" data-track-label="10.1159/000500354">https://doi.org/10.1159/000500354</a>.</p><p class="c-article-references__links u-hide-print"><a data-track="click_references" rel="nofollow noopener" data-track-label="10.1159/000500354" data-track-item_id="10.1159/000500354" data-track-value="article reference" data-track-action="article reference" href="https://doi.org/10.1159%2F000500354" aria-label="Article reference 39" data-doi="10.1159/000500354">Article</a> <a data-track="click_references" rel="nofollow noopener" data-track-label="link" data-track-item_id="link" data-track-value="pubmed reference" data-track-action="pubmed reference" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Abstract&list_uids=31872172" aria-label="PubMed reference 39">PubMed</a> <a data-track="click_references" rel="nofollow noopener" data-track-label="link" data-track-item_id="link" data-track-value="pubmed central reference" data-track-action="pubmed central reference" href="http://www.ncbi.nlm.nih.gov/pmc/articles/PMC6927667" aria-label="PubMed Central reference 39">PubMed Central</a> <a data-track="click_references" data-track-action="google scholar reference" data-track-value="google scholar reference" data-track-label="link" data-track-item_id="link" rel="nofollow noopener" aria-label="Google Scholar reference 39" href="http://scholar.google.com/scholar_lookup?&title=Depression%20screening%20from%20voice%20samples%20of%20patients%20affected%20by%20Parkinson%27s%20disease&journal=Digit%20Biomark.&doi=10.1159%2F000500354&volume=3&issue=2&pages=72-82&publication_year=2019&author=Ozkanca%2CY&author=Ozturk%2CMG&author=Ekmekci%2CMN&author=Atkins%2CDC&author=Demiroglu%2CC&author=Ghomi%2CRH"> Google Scholar</a> </p></li><li class="c-article-references__item js-c-reading-companion-references-item" data-counter="40."><p class="c-article-references__text" id="ref-CR40">Moreau A, Anderer P, Ross M, Cerny A, Almazan TH, Peterson B, et al. Detection of nocturnal scratching movements in patients with atopic dermatitis using accelerometers and recurrent neural networks. IEEE J Biomed Health Inform. 2018;22(4):1011–8. <a href="https://doi.org/10.1109/JBHI.2017.2710798" data-track="click_references" data-track-action="external reference" data-track-value="external reference" data-track-label="10.1109/JBHI.2017.2710798">https://doi.org/10.1109/JBHI.2017.2710798</a>.</p><p class="c-article-references__links u-hide-print"><a data-track="click_references" rel="nofollow noopener" data-track-label="10.1109/JBHI.2017.2710798" data-track-item_id="10.1109/JBHI.2017.2710798" data-track-value="article reference" data-track-action="article reference" href="https://doi.org/10.1109%2FJBHI.2017.2710798" aria-label="Article reference 40" data-doi="10.1109/JBHI.2017.2710798">Article</a> <a data-track="click_references" rel="nofollow noopener" data-track-label="link" data-track-item_id="link" data-track-value="pubmed reference" data-track-action="pubmed reference" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Abstract&list_uids=28613187" aria-label="PubMed reference 40">PubMed</a> <a data-track="click_references" data-track-action="google scholar reference" data-track-value="google scholar reference" data-track-label="link" data-track-item_id="link" rel="nofollow noopener" aria-label="Google Scholar reference 40" href="http://scholar.google.com/scholar_lookup?&title=Detection%20of%20nocturnal%20scratching%20movements%20in%20patients%20with%20atopic%20dermatitis%20using%20accelerometers%20and%20recurrent%20neural%20networks&journal=IEEE%20J%20Biomed%20Health%20Inform.&doi=10.1109%2FJBHI.2017.2710798&volume=22&issue=4&pages=1011-1018&publication_year=2018&author=Moreau%2CA&author=Anderer%2CP&author=Ross%2CM&author=Cerny%2CA&author=Almazan%2CTH&author=Peterson%2CB&author=Moreau%2CA&author=Anderer%2CP&author=Ross%2CM&author=Cerny%2CA&author=Almazan%2CTH&author=Peterson%2CB"> Google Scholar</a> </p></li><li class="c-article-references__item js-c-reading-companion-references-item" data-counter="41."><p class="c-article-references__text" id="ref-CR41">Han X, Hu Y, Foschini L, Chinitz L, Jankelson L, Ranganath R. Deep learning models for electrocardiograms are susceptible to adversarial attack. Nat Med. 2020;26(3):360–3. <a href="https://doi.org/10.1038/s41591-020-0791-x" data-track="click_references" data-track-action="external reference" data-track-value="external reference" data-track-label="10.1038/s41591-020-0791-x">https://doi.org/10.1038/s41591-020-0791-x</a> Epub 2020/03/11. PubMed PMID: 32152582.</p><p class="c-article-references__links u-hide-print"><a data-track="click_references" rel="nofollow noopener" data-track-label="10.1038/s41591-020-0791-x" data-track-item_id="10.1038/s41591-020-0791-x" data-track-value="article reference" data-track-action="article reference" href="https://doi.org/10.1038%2Fs41591-020-0791-x" aria-label="Article reference 41" data-doi="10.1038/s41591-020-0791-x">Article</a> <a data-track="click_references" rel="nofollow noopener" data-track-label="link" data-track-item_id="link" data-track-value="pubmed reference" data-track-action="pubmed reference" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Abstract&list_uids=32152582" aria-label="PubMed reference 41">PubMed</a> <a data-track="click_references" rel="nofollow noopener" data-track-label="link" data-track-item_id="link" data-track-value="pubmed central reference" data-track-action="pubmed central reference" href="http://www.ncbi.nlm.nih.gov/pmc/articles/PMC8096552" aria-label="PubMed Central reference 41">PubMed Central</a> <a data-track="click_references" rel="nofollow noopener" data-track-label="link" data-track-item_id="link" data-track-value="cas reference" data-track-action="cas reference" href="/articles/cas-redirect/1:CAS:528:DC%2BB3cXksFKmurw%3D" aria-label="CAS reference 41">CAS</a> <a data-track="click_references" data-track-action="google scholar reference" data-track-value="google scholar reference" data-track-label="link" data-track-item_id="link" rel="nofollow noopener" aria-label="Google Scholar reference 41" href="http://scholar.google.com/scholar_lookup?&title=Deep%20learning%20models%20for%20electrocardiograms%20are%20susceptible%20to%20adversarial%20attack&journal=Nat%20Med.&doi=10.1038%2Fs41591-020-0791-x&volume=26&issue=3&pages=360-363&publication_year=2020&author=Han%2CX&author=Hu%2CY&author=Foschini%2CL&author=Chinitz%2CL&author=Jankelson%2CL&author=Ranganath%2CR"> Google Scholar</a> </p></li><li class="c-article-references__item js-c-reading-companion-references-item" data-counter="42."><p class="c-article-references__text" id="ref-CR42">Doerr M, Maguire Truong A, Bot BM, Wilbanks J, Suver C, Mangravite LM. Formative evaluation of participant experience with mobile econsent in the app-mediated Parkinson mPower study: a mixed methods study. JMIR Mhealth Uhealth. 2017;5(2):e14. <a href="https://doi.org/10.2196/mhealth.6521" data-track="click_references" data-track-action="external reference" data-track-value="external reference" data-track-label="10.2196/mhealth.6521">https://doi.org/10.2196/mhealth.6521</a>.</p><p class="c-article-references__links u-hide-print"><a data-track="click_references" rel="nofollow noopener" data-track-label="10.2196/mhealth.6521" data-track-item_id="10.2196/mhealth.6521" data-track-value="article reference" data-track-action="article reference" href="https://doi.org/10.2196%2Fmhealth.6521" aria-label="Article reference 42" data-doi="10.2196/mhealth.6521">Article</a> <a data-track="click_references" rel="nofollow noopener" data-track-label="link" data-track-item_id="link" data-track-value="pubmed reference" data-track-action="pubmed reference" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Abstract&list_uids=28209557" aria-label="PubMed reference 42">PubMed</a> <a data-track="click_references" rel="nofollow noopener" data-track-label="link" data-track-item_id="link" data-track-value="pubmed central reference" data-track-action="pubmed central reference" href="http://www.ncbi.nlm.nih.gov/pmc/articles/PMC5334514" aria-label="PubMed Central reference 42">PubMed Central</a> <a data-track="click_references" data-track-action="google scholar reference" data-track-value="google scholar reference" data-track-label="link" data-track-item_id="link" rel="nofollow noopener" aria-label="Google Scholar reference 42" href="http://scholar.google.com/scholar_lookup?&title=Formative%20evaluation%20of%20participant%20experience%20with%20mobile%20econsent%20in%20the%20app-mediated%20Parkinson%20mPower%20study%3A%20a%20mixed%20methods%20study&journal=JMIR%20Mhealth%20Uhealth.&doi=10.2196%2Fmhealth.6521&volume=5&issue=2&publication_year=2017&author=Doerr%2CM&author=Maguire%20Truong%2CA&author=Bot%2CBM&author=Wilbanks%2CJ&author=Suver%2CC&author=Mangravite%2CLM"> Google Scholar</a> </p></li><li class="c-article-references__item js-c-reading-companion-references-item" data-counter="43."><p class="c-article-references__text" id="ref-CR43">Savova GK, Danciu I, Alamudun F, Miller T, Lin C, Bitterman DS, et al. Use of natural language processing to extract clinical cancer phenotypes from electronic medical records. Cancer Res. 2019;79(21):5463–70. <a href="https://doi.org/10.1158/0008-5472.CAN-19-0579" data-track="click_references" data-track-action="external reference" data-track-value="external reference" data-track-label="10.1158/0008-5472.CAN-19-0579">https://doi.org/10.1158/0008-5472.CAN-19-0579</a>.</p><p class="c-article-references__links u-hide-print"><a data-track="click_references" rel="nofollow noopener" data-track-label="10.1158/0008-5472.CAN-19-0579" data-track-item_id="10.1158/0008-5472.CAN-19-0579" data-track-value="article reference" data-track-action="article reference" href="https://doi.org/10.1158%2F0008-5472.CAN-19-0579" aria-label="Article reference 43" data-doi="10.1158/0008-5472.CAN-19-0579">Article</a> <a data-track="click_references" rel="nofollow noopener" data-track-label="link" data-track-item_id="link" data-track-value="pubmed reference" data-track-action="pubmed reference" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Abstract&list_uids=31395609" aria-label="PubMed reference 43">PubMed</a> <a data-track="click_references" rel="nofollow noopener" data-track-label="link" data-track-item_id="link" data-track-value="pubmed central reference" data-track-action="pubmed central reference" href="http://www.ncbi.nlm.nih.gov/pmc/articles/PMC7227798" aria-label="PubMed Central reference 43">PubMed Central</a> <a data-track="click_references" rel="nofollow noopener" data-track-label="link" data-track-item_id="link" data-track-value="cas reference" data-track-action="cas reference" href="/articles/cas-redirect/1:CAS:528:DC%2BB3cXis1Cqsrc%3D" aria-label="CAS reference 43">CAS</a> <a data-track="click_references" data-track-action="google scholar reference" data-track-value="google scholar reference" data-track-label="link" data-track-item_id="link" rel="nofollow noopener" aria-label="Google Scholar reference 43" href="http://scholar.google.com/scholar_lookup?&title=Use%20of%20natural%20language%20processing%20to%20extract%20clinical%20cancer%20phenotypes%20from%20electronic%20medical%20records&journal=Cancer%20Res.&doi=10.1158%2F0008-5472.CAN-19-0579&volume=79&issue=21&pages=5463-5470&publication_year=2019&author=Savova%2CGK&author=Danciu%2CI&author=Alamudun%2CF&author=Miller%2CT&author=Lin%2CC&author=Bitterman%2CDS&author=Tourassi%2CG&author=Warner%2CJL"> Google Scholar</a> </p></li><li class="c-article-references__item js-c-reading-companion-references-item" data-counter="44."><p class="c-article-references__text" id="ref-CR44">Malke JC, Jin S, Camp SP, Lari B, Kell T, Simon JM, et al. Enhancing case capture, quality, and completeness of primary melanoma pathology records via natural language processing. JCO Clin Cancer Inform. 2019;3:1–11. <a href="https://doi.org/10.1200/CCI.19.00006" data-track="click_references" data-track-action="external reference" data-track-value="external reference" data-track-label="10.1200/CCI.19.00006">https://doi.org/10.1200/CCI.19.00006</a>.</p><p class="c-article-references__links u-hide-print"><a data-track="click_references" rel="nofollow noopener" data-track-label="10.1200/CCI.19.00006" data-track-item_id="10.1200/CCI.19.00006" data-track-value="article reference" data-track-action="article reference" href="https://doi.org/10.1200%2FCCI.19.00006" aria-label="Article reference 44" data-doi="10.1200/CCI.19.00006">Article</a> <a data-track="click_references" rel="nofollow noopener" data-track-label="link" data-track-item_id="link" data-track-value="pubmed reference" data-track-action="pubmed reference" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Abstract&list_uids=31442076" aria-label="PubMed reference 44">PubMed</a> <a data-track="click_references" data-track-action="google scholar reference" data-track-value="google scholar reference" data-track-label="link" data-track-item_id="link" rel="nofollow noopener" aria-label="Google Scholar reference 44" href="http://scholar.google.com/scholar_lookup?&title=Enhancing%20case%20capture%2C%20quality%2C%20and%20completeness%20of%20primary%20melanoma%20pathology%20records%20via%20natural%20language%20processing&journal=JCO%20Clin%20Cancer%20Inform.&doi=10.1200%2FCCI.19.00006&volume=3&pages=1-11&publication_year=2019&author=Malke%2CJC&author=Jin%2CS&author=Camp%2CSP&author=Lari%2CB&author=Kell%2CT&author=Simon%2CJM&author=Prieto%2CVG&author=Gershenwald%2CJE&author=Haydu%2CLE"> Google Scholar</a> </p></li><li class="c-article-references__item js-c-reading-companion-references-item" data-counter="45."><p class="c-article-references__text" id="ref-CR45">Vaci N, Liu Q, Kormilitzin A, De Crescenzo F, Kurtulmus A, Harvey J, et al. Natural language processing for structuring clinical text data on depression using UK-CRIS. Evid Based Ment Health. 2020;23(1):21–6. <a href="https://doi.org/10.1136/ebmental-2019-300134" data-track="click_references" data-track-action="external reference" data-track-value="external reference" data-track-label="10.1136/ebmental-2019-300134">https://doi.org/10.1136/ebmental-2019-300134</a>.</p><p class="c-article-references__links u-hide-print"><a data-track="click_references" rel="nofollow noopener" data-track-label="10.1136/ebmental-2019-300134" data-track-item_id="10.1136/ebmental-2019-300134" data-track-value="article reference" data-track-action="article reference" href="https://doi.org/10.1136%2Febmental-2019-300134" aria-label="Article reference 45" data-doi="10.1136/ebmental-2019-300134">Article</a> <a data-track="click_references" rel="nofollow noopener" data-track-label="link" data-track-item_id="link" data-track-value="pubmed reference" data-track-action="pubmed reference" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Abstract&list_uids=32046989" aria-label="PubMed reference 45">PubMed</a> <a data-track="click_references" data-track-action="google scholar reference" data-track-value="google scholar reference" data-track-label="link" data-track-item_id="link" rel="nofollow noopener" aria-label="Google Scholar reference 45" href="http://scholar.google.com/scholar_lookup?&title=Natural%20language%20processing%20for%20structuring%20clinical%20text%20data%20on%20depression%20using%20UK-CRIS&journal=Evid%20Based%20Ment%20Health.&doi=10.1136%2Febmental-2019-300134&volume=23&issue=1&pages=21-26&publication_year=2020&author=Vaci%2CN&author=Liu%2CQ&author=Kormilitzin%2CA&author=Crescenzo%2CF&author=Kurtulmus%2CA&author=Harvey%2CJ"> Google Scholar</a> </p></li><li class="c-article-references__item js-c-reading-companion-references-item" data-counter="46."><p class="c-article-references__text" id="ref-CR46">Tian Q, Liu M, Min L, An J, Lu X, Duan H. An automated data verification approach for improving data quality in a clinical registry. Comput Methods Programs Biomed. 2019;181:104840. <a href="https://doi.org/10.1016/j.cmpb.2019.01.012" data-track="click_references" data-track-action="external reference" data-track-value="external reference" data-track-label="10.1016/j.cmpb.2019.01.012">https://doi.org/10.1016/j.cmpb.2019.01.012</a>.</p><p class="c-article-references__links u-hide-print"><a data-track="click_references" rel="nofollow noopener" data-track-label="10.1016/j.cmpb.2019.01.012" data-track-item_id="10.1016/j.cmpb.2019.01.012" data-track-value="article reference" data-track-action="article reference" href="https://doi.org/10.1016%2Fj.cmpb.2019.01.012" aria-label="Article reference 46" data-doi="10.1016/j.cmpb.2019.01.012">Article</a> <a data-track="click_references" rel="nofollow noopener" data-track-label="link" data-track-item_id="link" data-track-value="pubmed reference" data-track-action="pubmed reference" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Abstract&list_uids=30777618" aria-label="PubMed reference 46">PubMed</a> <a data-track="click_references" data-track-action="google scholar reference" data-track-value="google scholar reference" data-track-label="link" data-track-item_id="link" rel="nofollow noopener" aria-label="Google Scholar reference 46" href="http://scholar.google.com/scholar_lookup?&title=An%20automated%20data%20verification%20approach%20for%20improving%20data%20quality%20in%20a%20clinical%20registry&journal=Comput%20Methods%20Programs%20Biomed.&doi=10.1016%2Fj.cmpb.2019.01.012&volume=181&publication_year=2019&author=Tian%2CQ&author=Liu%2CM&author=Min%2CL&author=An%2CJ&author=Lu%2CX&author=Duan%2CH"> Google Scholar</a> </p></li><li class="c-article-references__item js-c-reading-companion-references-item" data-counter="47."><p class="c-article-references__text" id="ref-CR47">Estiri H, Murphy SN. Semi-supervised encoding for outlier detection in clinical observation data. Comput Methods Programs Biomed. 2019;181:104830. <a href="https://doi.org/10.1016/j.cmpb.2019.01.002" data-track="click_references" data-track-action="external reference" data-track-value="external reference" data-track-label="10.1016/j.cmpb.2019.01.002">https://doi.org/10.1016/j.cmpb.2019.01.002</a>.</p><p class="c-article-references__links u-hide-print"><a data-track="click_references" rel="nofollow noopener" data-track-label="10.1016/j.cmpb.2019.01.002" data-track-item_id="10.1016/j.cmpb.2019.01.002" data-track-value="article reference" data-track-action="article reference" href="https://doi.org/10.1016%2Fj.cmpb.2019.01.002" aria-label="Article reference 47" data-doi="10.1016/j.cmpb.2019.01.002">Article</a> <a data-track="click_references" rel="nofollow noopener" data-track-label="link" data-track-item_id="link" data-track-value="pubmed reference" data-track-action="pubmed reference" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Abstract&list_uids=30658851" aria-label="PubMed reference 47">PubMed</a> <a data-track="click_references" data-track-action="google scholar reference" data-track-value="google scholar reference" data-track-label="link" data-track-item_id="link" rel="nofollow noopener" aria-label="Google Scholar reference 47" href="http://scholar.google.com/scholar_lookup?&title=Semi-supervised%20encoding%20for%20outlier%20detection%20in%20clinical%20observation%20data&journal=Comput%20Methods%20Programs%20Biomed.&doi=10.1016%2Fj.cmpb.2019.01.002&volume=181&publication_year=2019&author=Estiri%2CH&author=Murphy%2CSN"> Google Scholar</a> </p></li><li class="c-article-references__item js-c-reading-companion-references-item" data-counter="48."><p class="c-article-references__text" id="ref-CR48">Glass, LMS G; Patil, R. AI in clinical development: improving safety and accelerating results. [White paper]. In press 2019.</p><p class="c-article-references__links u-hide-print"><a data-track="click_references" data-track-action="google scholar reference" data-track-value="google scholar reference" data-track-label="link" data-track-item_id="link" rel="nofollow noopener" aria-label="Google Scholar reference 48" href="http://scholar.google.com/scholar_lookup?&title=AI%20in%20clinical%20development%3A%20improving%20safety%20and%20accelerating%20results.%20%5BWhite%20paper%5D&publication_year=2019&author=Glass%2CLMSG&author=Patil%2CR"> Google Scholar</a> </p></li><li class="c-article-references__item js-c-reading-companion-references-item" data-counter="49."><p class="c-article-references__text" id="ref-CR49">Hicks KA, Mahaffey KW, Mehran R, Nissen SE, Wiviott SD, Dunn B, et al. 2017 Cardiovascular and stroke endpoint definitions for clinical trials. Circulation. 2018;137(9):961–72. <a href="https://doi.org/10.1161/CIRCULATIONAHA.117.033502" data-track="click_references" data-track-action="external reference" data-track-value="external reference" data-track-label="10.1161/CIRCULATIONAHA.117.033502">https://doi.org/10.1161/CIRCULATIONAHA.117.033502</a>.</p><p class="c-article-references__links u-hide-print"><a data-track="click_references" rel="nofollow noopener" data-track-label="10.1161/CIRCULATIONAHA.117.033502" data-track-item_id="10.1161/CIRCULATIONAHA.117.033502" data-track-value="article reference" data-track-action="article reference" href="https://doi.org/10.1161%2FCIRCULATIONAHA.117.033502" aria-label="Article reference 49" data-doi="10.1161/CIRCULATIONAHA.117.033502">Article</a> <a data-track="click_references" rel="nofollow noopener" data-track-label="link" data-track-item_id="link" data-track-value="pubmed reference" data-track-action="pubmed reference" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Abstract&list_uids=29483172" aria-label="PubMed reference 49">PubMed</a> <a data-track="click_references" data-track-action="google scholar reference" data-track-value="google scholar reference" data-track-label="link" data-track-item_id="link" rel="nofollow noopener" aria-label="Google Scholar reference 49" href="http://scholar.google.com/scholar_lookup?&title=2017%20Cardiovascular%20and%20stroke%20endpoint%20definitions%20for%20clinical%20trials&journal=Circulation.&doi=10.1161%2FCIRCULATIONAHA.117.033502&volume=137&issue=9&pages=961-972&publication_year=2018&author=Hicks%2CKA&author=Mahaffey%2CKW&author=Mehran%2CR&author=Nissen%2CSE&author=Wiviott%2CSD&author=Dunn%2CB&author=Solomon%2CSD&author=Marler%2CJR&author=Teerlink%2CJR&author=Farb%2CA&author=Morrow%2CDA&author=Targum%2CSL&author=Sila%2CCA&author=Hai%2CMTT&author=Jaff%2CMR&author=Joffe%2CHV&author=Cutlip%2CDE&author=Desai%2CAS&author=Lewis%2CEF&author=Gibson%2CCM&author=Landray%2CMJ&author=Lincoff%2CAM&author=White%2CCJ&author=Brooks%2CSS&author=Rosenfield%2CK&author=Domanski%2CMJ&author=Lansky%2CAJ&author=McMurray%2CJ&author=Tcheng%2CJE&author=Steinhubl%2CSR&author=Burton%2CP&author=Mauri%2CL&author=O%27Connor%2CCM&author=Pfeffer%2CMA&author=Hung%2CHMJ&author=Stockbridge%2CNL&author=Chaitman%2CBR&author=Temple%2CRJ"> Google Scholar</a> </p></li><li class="c-article-references__item js-c-reading-companion-references-item" data-counter="50."><p class="c-article-references__text" id="ref-CR50">Liu Y, Gopalakrishnan V. An overview and evaluation of recent machine learning imputation methods using cardiac imaging data. Data (Basel). 2017;2(1):8. <a href="https://pubmed.ncbi.nlm.nih.gov/28243594/" data-track="click_references" data-track-action="external reference" data-track-value="external reference" data-track-label="https://pubmed.ncbi.nlm.nih.gov/28243594/">https://pubmed.ncbi.nlm.nih.gov/28243594/</a>.</p></li><li class="c-article-references__item js-c-reading-companion-references-item" data-counter="51."><p class="c-article-references__text" id="ref-CR51">Phung S, Kumar A, Kim J. A deep learning technique for imputing missing healthcare data. Conf Proc IEEE Eng Med Biol Soc. 2019;2019:6513–6. <a href="https://doi.org/10.1109/EMBC.2019.8856760" data-track="click_references" data-track-action="external reference" data-track-value="external reference" data-track-label="10.1109/EMBC.2019.8856760">https://doi.org/10.1109/EMBC.2019.8856760</a> Epub 2020/01/18PubMed PMID: 31947333.</p><p class="c-article-references__links u-hide-print"><a data-track="click_references" rel="nofollow noopener" data-track-label="10.1109/EMBC.2019.8856760" data-track-item_id="10.1109/EMBC.2019.8856760" data-track-value="article reference" data-track-action="article reference" href="https://doi.org/10.1109%2FEMBC.2019.8856760" aria-label="Article reference 51" data-doi="10.1109/EMBC.2019.8856760">Article</a> <a data-track="click_references" data-track-action="google scholar reference" data-track-value="google scholar reference" data-track-label="link" data-track-item_id="link" rel="nofollow noopener" aria-label="Google Scholar reference 51" href="http://scholar.google.com/scholar_lookup?&title=A%20deep%20learning%20technique%20for%20imputing%20missing%20healthcare%20data&journal=Conf%20Proc%20IEEE%20Eng%20Med%20Biol%20Soc.&doi=10.1109%2FEMBC.2019.8856760&volume=2019&pages=6513-6516&publication_year=2019&author=Phung%2CS&author=Kumar%2CA&author=Kim%2CJ"> Google Scholar</a> </p></li><li class="c-article-references__item js-c-reading-companion-references-item" data-counter="52."><p class="c-article-references__text" id="ref-CR52">Qiu YL, Zheng H, Gevaert OJ. A deep learning framework for imputing missing values in genomic data; 2018.</p><p class="c-article-references__links u-hide-print"><a data-track="click_references" rel="nofollow noopener" data-track-label="10.1101/406066" data-track-item_id="10.1101/406066" data-track-value="book reference" data-track-action="book reference" href="https://doi.org/10.1101%2F406066" aria-label="Book reference 52" data-doi="10.1101/406066">Book</a> <a data-track="click_references" data-track-action="google scholar reference" data-track-value="google scholar reference" data-track-label="link" data-track-item_id="link" rel="nofollow noopener" aria-label="Google Scholar reference 52" href="http://scholar.google.com/scholar_lookup?&title=A%20deep%20learning%20framework%20for%20imputing%20missing%20values%20in%20genomic%20data&doi=10.1101%2F406066&publication_year=2018&author=Qiu%2CYL&author=Zheng%2CH&author=Gevaert%2COJ"> Google Scholar</a> </p></li><li class="c-article-references__item js-c-reading-companion-references-item" data-counter="53."><p class="c-article-references__text" id="ref-CR53">Feng T, Narayanan S. Imputing missing data in large-scale multivariate biomedical wearable recordings using bidirectional recurrent neural networks with temporal activation regularization. In: 2019 41st Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC); 2019.</p><p class="c-article-references__links u-hide-print"><a data-track="click_references" data-track-action="google scholar reference" data-track-value="google scholar reference" data-track-label="link" data-track-item_id="link" rel="nofollow noopener" aria-label="Google Scholar reference 53" href="http://scholar.google.com/scholar_lookup?&title=Imputing%20missing%20data%20in%20large-scale%20multivariate%20biomedical%20wearable%20recordings%20using%20bidirectional%20recurrent%20neural%20networks%20with%20temporal%20activation%20regularization&publication_year=2019&author=Feng%2CT&author=Narayanan%2CS"> Google Scholar</a> </p></li><li class="c-article-references__item js-c-reading-companion-references-item" data-counter="54."><p class="c-article-references__text" id="ref-CR54">Luo Y, Szolovits P, Dighe AS, Baron JM. 3D-MICE: integration of cross-sectional and longitudinal imputation for multi-analyte longitudinal clinical data. J Am Med Inform Assoc. 2018;25(6):645–53. <a href="https://doi.org/10.1093/jamia/ocx133" data-track="click_references" data-track-action="external reference" data-track-value="external reference" data-track-label="10.1093/jamia/ocx133">https://doi.org/10.1093/jamia/ocx133</a>.</p><p class="c-article-references__links u-hide-print"><a data-track="click_references" rel="nofollow noopener" data-track-label="10.1093/jamia/ocx133" data-track-item_id="10.1093/jamia/ocx133" data-track-value="article reference" data-track-action="article reference" href="https://doi.org/10.1093%2Fjamia%2Focx133" aria-label="Article reference 54" data-doi="10.1093/jamia/ocx133">Article</a> <a data-track="click_references" rel="nofollow noopener" data-track-label="link" data-track-item_id="link" data-track-value="pubmed reference" data-track-action="pubmed reference" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Abstract&list_uids=29202205" aria-label="PubMed reference 54">PubMed</a> <a data-track="click_references" data-track-action="google scholar reference" data-track-value="google scholar reference" data-track-label="link" data-track-item_id="link" rel="nofollow noopener" aria-label="Google Scholar reference 54" href="http://scholar.google.com/scholar_lookup?&title=3D-MICE%3A%20integration%20of%20cross-sectional%20and%20longitudinal%20imputation%20for%20multi-analyte%20longitudinal%20clinical%20data&journal=J%20Am%20Med%20Inform%20Assoc.&doi=10.1093%2Fjamia%2Focx133&volume=25&issue=6&pages=645-653&publication_year=2018&author=Luo%2CY&author=Szolovits%2CP&author=Dighe%2CAS&author=Baron%2CJM"> Google Scholar</a> </p></li><li class="c-article-references__item js-c-reading-companion-references-item" data-counter="55."><p class="c-article-references__text" id="ref-CR55">Ngufor C, Warner MA, Murphree DH, Liu H, Carter R, Storlie CB, et al. Identification of Clinically meaningful plasma transfusion subgroups using unsupervised random forest clustering. AMIA Annu Symp Proc. 2017;2017:1332–41.</p><p class="c-article-references__links u-hide-print"><a data-track="click_references" rel="nofollow noopener" data-track-label="link" data-track-item_id="link" data-track-value="pubmed reference" data-track-action="pubmed reference" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Abstract&list_uids=29854202" aria-label="PubMed reference 55">PubMed</a> <a data-track="click_references" data-track-action="google scholar reference" data-track-value="google scholar reference" data-track-label="link" data-track-item_id="link" rel="nofollow noopener" aria-label="Google Scholar reference 55" href="http://scholar.google.com/scholar_lookup?&title=Identification%20of%20Clinically%20meaningful%20plasma%20transfusion%20subgroups%20using%20unsupervised%20random%20forest%20clustering&journal=AMIA%20Annu%20Symp%20Proc.&volume=2017&pages=1332-1341&publication_year=2017&author=Ngufor%2CC&author=Warner%2CMA&author=Murphree%2CDH&author=Liu%2CH&author=Carter%2CR&author=Storlie%2CCB"> Google Scholar</a> </p></li><li class="c-article-references__item js-c-reading-companion-references-item" data-counter="56."><p class="c-article-references__text" id="ref-CR56">Tomic A, Tomic I, Rosenberg-Hasson Y, Dekker CL, Maecker HT, Davis MM. SIMON, an automated machine learning system, reveals immune signatures of influenza vaccine responses. J Immunol. 2019;203(3):749–59. <a href="https://doi.org/10.4049/jimmunol.1900033" data-track="click_references" data-track-action="external reference" data-track-value="external reference" data-track-label="10.4049/jimmunol.1900033">https://doi.org/10.4049/jimmunol.1900033</a>.</p><p class="c-article-references__links u-hide-print"><a data-track="click_references" rel="nofollow noopener" data-track-label="10.4049/jimmunol.1900033" data-track-item_id="10.4049/jimmunol.1900033" data-track-value="article reference" data-track-action="article reference" href="https://doi.org/10.4049%2Fjimmunol.1900033" aria-label="Article reference 56" data-doi="10.4049/jimmunol.1900033">Article</a> <a data-track="click_references" rel="nofollow noopener" data-track-label="link" data-track-item_id="link" data-track-value="pubmed reference" data-track-action="pubmed reference" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Abstract&list_uids=31201239" aria-label="PubMed reference 56">PubMed</a> <a data-track="click_references" rel="nofollow noopener" data-track-label="link" data-track-item_id="link" data-track-value="pubmed central reference" data-track-action="pubmed central reference" href="http://www.ncbi.nlm.nih.gov/pmc/articles/PMC6643048" aria-label="PubMed Central reference 56">PubMed Central</a> <a data-track="click_references" rel="nofollow noopener" data-track-label="link" data-track-item_id="link" data-track-value="cas reference" data-track-action="cas reference" href="/articles/cas-redirect/1:CAS:528:DC%2BC1MXhvFWnsLvP" aria-label="CAS reference 56">CAS</a> <a data-track="click_references" data-track-action="google scholar reference" data-track-value="google scholar reference" data-track-label="link" data-track-item_id="link" rel="nofollow noopener" aria-label="Google Scholar reference 56" href="http://scholar.google.com/scholar_lookup?&title=SIMON%2C%20an%20automated%20machine%20learning%20system%2C%20reveals%20immune%20signatures%20of%20influenza%20vaccine%20responses&journal=J%20Immunol.&doi=10.4049%2Fjimmunol.1900033&volume=203&issue=3&pages=749-759&publication_year=2019&author=Tomic%2CA&author=Tomic%2CI&author=Rosenberg-Hasson%2CY&author=Dekker%2CCL&author=Maecker%2CHT&author=Davis%2CMM"> Google Scholar</a> </p></li><li class="c-article-references__item js-c-reading-companion-references-item" data-counter="57."><p class="c-article-references__text" id="ref-CR57">Watson JA, Holmes CC. Machine learning analysis plans for randomised controlled trials: detecting treatment effect heterogeneity with strict control of type I error. Trials. 2020;21(1):156. <a href="https://doi.org/10.1186/s13063-020-4076-y" data-track="click_references" data-track-action="external reference" data-track-value="external reference" data-track-label="10.1186/s13063-020-4076-y">https://doi.org/10.1186/s13063-020-4076-y</a>.</p><p class="c-article-references__links u-hide-print"><a data-track="click_references" rel="noopener" data-track-label="10.1186/s13063-020-4076-y" data-track-item_id="10.1186/s13063-020-4076-y" data-track-value="article reference" data-track-action="article reference" href="https://link.springer.com/doi/10.1186/s13063-020-4076-y" aria-label="Article reference 57" data-doi="10.1186/s13063-020-4076-y">Article</a> <a data-track="click_references" rel="nofollow noopener" data-track-label="link" data-track-item_id="link" data-track-value="pubmed reference" data-track-action="pubmed reference" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Abstract&list_uids=32041653" aria-label="PubMed reference 57">PubMed</a> <a data-track="click_references" rel="nofollow noopener" data-track-label="link" data-track-item_id="link" data-track-value="pubmed central reference" data-track-action="pubmed central reference" href="http://www.ncbi.nlm.nih.gov/pmc/articles/PMC7011561" aria-label="PubMed Central reference 57">PubMed Central</a> <a data-track="click_references" data-track-action="google scholar reference" data-track-value="google scholar reference" data-track-label="link" data-track-item_id="link" rel="nofollow noopener" aria-label="Google Scholar reference 57" href="http://scholar.google.com/scholar_lookup?&title=Machine%20learning%20analysis%20plans%20for%20randomised%20controlled%20trials%3A%20detecting%20treatment%20effect%20heterogeneity%20with%20strict%20control%20of%20type%20I%20error&journal=Trials.&doi=10.1186%2Fs13063-020-4076-y&volume=21&issue=1&publication_year=2020&author=Watson%2CJA&author=Holmes%2CCC"> Google Scholar</a> </p></li><li class="c-article-references__item js-c-reading-companion-references-item" data-counter="58."><p class="c-article-references__text" id="ref-CR58">Rigdon J, Baiocchi M, Basu S. Preventing false discovery of heterogeneous treatment effect subgroups in randomized trials. Trials. 2018;19(1):382. <a href="https://doi.org/10.1186/s13063-018-2774-5" data-track="click_references" data-track-action="external reference" data-track-value="external reference" data-track-label="10.1186/s13063-018-2774-5">https://doi.org/10.1186/s13063-018-2774-5</a>.</p><p class="c-article-references__links u-hide-print"><a data-track="click_references" rel="noopener" data-track-label="10.1186/s13063-018-2774-5" data-track-item_id="10.1186/s13063-018-2774-5" data-track-value="article reference" data-track-action="article reference" href="https://link.springer.com/doi/10.1186/s13063-018-2774-5" aria-label="Article reference 58" data-doi="10.1186/s13063-018-2774-5">Article</a> <a data-track="click_references" rel="nofollow noopener" data-track-label="link" data-track-item_id="link" data-track-value="pubmed reference" data-track-action="pubmed reference" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Abstract&list_uids=30012181" aria-label="PubMed reference 58">PubMed</a> <a data-track="click_references" rel="nofollow noopener" data-track-label="link" data-track-item_id="link" data-track-value="pubmed central reference" data-track-action="pubmed central reference" href="http://www.ncbi.nlm.nih.gov/pmc/articles/PMC6048878" aria-label="PubMed Central reference 58">PubMed Central</a> <a data-track="click_references" rel="nofollow noopener" data-track-label="link" data-track-item_id="link" data-track-value="cas reference" data-track-action="cas reference" href="/articles/cas-redirect/1:CAS:528:DC%2BC1MXjs1ert7w%3D" aria-label="CAS reference 58">CAS</a> <a data-track="click_references" data-track-action="google scholar reference" data-track-value="google scholar reference" data-track-label="link" data-track-item_id="link" rel="nofollow noopener" aria-label="Google Scholar reference 58" href="http://scholar.google.com/scholar_lookup?&title=Preventing%20false%20discovery%20of%20heterogeneous%20treatment%20effect%20subgroups%20in%20randomized%20trials&journal=Trials.&doi=10.1186%2Fs13063-018-2774-5&volume=19&issue=1&publication_year=2018&author=Rigdon%2CJ&author=Baiocchi%2CM&author=Basu%2CS"> Google Scholar</a> </p></li><li class="c-article-references__item js-c-reading-companion-references-item" data-counter="59."><p class="c-article-references__text" id="ref-CR59">Kalscheur MM, Kipp RT, Tattersall MC, Mei C, Buhr KA, DeMets DL, et al. Machine learning algorithm predicts cardiac resynchronization therapy outcomes: lessons from the companion trial. Circ Arrhythm Electrophysiol. 2018;11(1):e005499. <a href="https://doi.org/10.1161/CIRCEP.117.005499" data-track="click_references" data-track-action="external reference" data-track-value="external reference" data-track-label="10.1161/CIRCEP.117.005499">https://doi.org/10.1161/CIRCEP.117.005499</a>.</p><p class="c-article-references__links u-hide-print"><a data-track="click_references" rel="nofollow noopener" data-track-label="10.1161/CIRCEP.117.005499" data-track-item_id="10.1161/CIRCEP.117.005499" data-track-value="article reference" data-track-action="article reference" href="https://doi.org/10.1161%2FCIRCEP.117.005499" aria-label="Article reference 59" data-doi="10.1161/CIRCEP.117.005499">Article</a> <a data-track="click_references" rel="nofollow noopener" data-track-label="link" data-track-item_id="link" data-track-value="pubmed reference" data-track-action="pubmed reference" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Abstract&list_uids=29326129" aria-label="PubMed reference 59">PubMed</a> <a data-track="click_references" rel="nofollow noopener" data-track-label="link" data-track-item_id="link" data-track-value="pubmed central reference" data-track-action="pubmed central reference" href="http://www.ncbi.nlm.nih.gov/pmc/articles/PMC5769699" aria-label="PubMed Central reference 59">PubMed Central</a> <a data-track="click_references" data-track-action="google scholar reference" data-track-value="google scholar reference" data-track-label="link" data-track-item_id="link" rel="nofollow noopener" aria-label="Google Scholar reference 59" href="http://scholar.google.com/scholar_lookup?&title=Machine%20learning%20algorithm%20predicts%20cardiac%20resynchronization%20therapy%20outcomes%3A%20lessons%20from%20the%20companion%20trial&journal=Circ%20Arrhythm%20Electrophysiol.&doi=10.1161%2FCIRCEP.117.005499&volume=11&issue=1&publication_year=2018&author=Kalscheur%2CMM&author=Kipp%2CRT&author=Tattersall%2CMC&author=Mei%2CC&author=Buhr%2CKA&author=DeMets%2CDL&author=Field%2CME&author=Eckhardt%2CLL&author=Page%2CCD"> Google Scholar</a> </p></li><li class="c-article-references__item js-c-reading-companion-references-item" data-counter="60."><p class="c-article-references__text" id="ref-CR60">Linden A, Yarnold PR. Combining machine learning and propensity score weighting to estimate causal effects in multivalued treatments. J Eval Clin Pract. 2016;22(6):871–81. <a href="https://doi.org/10.1111/jep.12610" data-track="click_references" data-track-action="external reference" data-track-value="external reference" data-track-label="10.1111/jep.12610">https://doi.org/10.1111/jep.12610</a>.</p><p class="c-article-references__links u-hide-print"><a data-track="click_references" rel="nofollow noopener" data-track-label="10.1111/jep.12610" data-track-item_id="10.1111/jep.12610" data-track-value="article reference" data-track-action="article reference" href="https://doi.org/10.1111%2Fjep.12610" aria-label="Article reference 60" data-doi="10.1111/jep.12610">Article</a> <a data-track="click_references" rel="nofollow noopener" data-track-label="link" data-track-item_id="link" data-track-value="pubmed reference" data-track-action="pubmed reference" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Abstract&list_uids=27421786" aria-label="PubMed reference 60">PubMed</a> <a data-track="click_references" data-track-action="google scholar reference" data-track-value="google scholar reference" data-track-label="link" data-track-item_id="link" rel="nofollow noopener" aria-label="Google Scholar reference 60" href="http://scholar.google.com/scholar_lookup?&title=Combining%20machine%20learning%20and%20propensity%20score%20weighting%20to%20estimate%20causal%20effects%20in%20multivalued%20treatments&journal=J%20Eval%20Clin%20Pract.&doi=10.1111%2Fjep.12610&volume=22&issue=6&pages=871-881&publication_year=2016&author=Linden%2CA&author=Yarnold%2CPR"> Google Scholar</a> </p></li><li class="c-article-references__item js-c-reading-companion-references-item" data-counter="61."><p class="c-article-references__text" id="ref-CR61">Schuler MS, Rose S. Targeted maximum likelihood estimation for causal inference in observational studies. Am J Epidemiol. 2017;185(1):65–73. <a href="https://doi.org/10.1093/aje/kww165" data-track="click_references" data-track-action="external reference" data-track-value="external reference" data-track-label="10.1093/aje/kww165">https://doi.org/10.1093/aje/kww165</a>.</p><p class="c-article-references__links u-hide-print"><a data-track="click_references" rel="nofollow noopener" data-track-label="10.1093/aje/kww165" data-track-item_id="10.1093/aje/kww165" data-track-value="article reference" data-track-action="article reference" href="https://doi.org/10.1093%2Faje%2Fkww165" aria-label="Article reference 61" data-doi="10.1093/aje/kww165">Article</a> <a data-track="click_references" rel="nofollow noopener" data-track-label="link" data-track-item_id="link" data-track-value="pubmed reference" data-track-action="pubmed reference" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Abstract&list_uids=27941068" aria-label="PubMed reference 61">PubMed</a> <a data-track="click_references" data-track-action="google scholar reference" data-track-value="google scholar reference" data-track-label="link" data-track-item_id="link" rel="nofollow noopener" aria-label="Google Scholar reference 61" href="http://scholar.google.com/scholar_lookup?&title=Targeted%20maximum%20likelihood%20estimation%20for%20causal%20inference%20in%20observational%20studies&journal=Am%20J%20Epidemiol.&doi=10.1093%2Faje%2Fkww165&volume=185&issue=1&pages=65-73&publication_year=2017&author=Schuler%2CMS&author=Rose%2CS"> Google Scholar</a> </p></li><li class="c-article-references__item js-c-reading-companion-references-item" data-counter="62."><p class="c-article-references__text" id="ref-CR62">Wendling T, Jung K, Callahan A, Schuler A, Shah NH, Gallego B. Comparing methods for estimation of heterogeneous treatment effects using observational data from health care databases. Stat Med. 2018;37(23):3309–24. <a href="https://doi.org/10.1002/sim.7820" data-track="click_references" data-track-action="external reference" data-track-value="external reference" data-track-label="10.1002/sim.7820">https://doi.org/10.1002/sim.7820</a>.</p><p class="c-article-references__links u-hide-print"><a data-track="click_references" rel="nofollow noopener" data-track-label="10.1002/sim.7820" data-track-item_id="10.1002/sim.7820" data-track-value="article reference" data-track-action="article reference" href="https://doi.org/10.1002%2Fsim.7820" aria-label="Article reference 62" data-doi="10.1002/sim.7820">Article</a> <a data-track="click_references" rel="nofollow noopener" data-track-label="link" data-track-item_id="link" data-track-value="pubmed reference" data-track-action="pubmed reference" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Abstract&list_uids=29862536" aria-label="PubMed reference 62">PubMed</a> <a data-track="click_references" rel="nofollow noopener" data-track-label="link" data-track-item_id="link" data-track-value="cas reference" data-track-action="cas reference" href="/articles/cas-redirect/1:STN:280:DC%2BC1MbjtlCitQ%3D%3D" aria-label="CAS reference 62">CAS</a> <a data-track="click_references" data-track-action="google scholar reference" data-track-value="google scholar reference" data-track-label="link" data-track-item_id="link" rel="nofollow noopener" aria-label="Google Scholar reference 62" href="http://scholar.google.com/scholar_lookup?&title=Comparing%20methods%20for%20estimation%20of%20heterogeneous%20treatment%20effects%20using%20observational%20data%20from%20health%20care%20databases&journal=Stat%20Med.&doi=10.1002%2Fsim.7820&volume=37&issue=23&pages=3309-3324&publication_year=2018&author=Wendling%2CT&author=Jung%2CK&author=Callahan%2CA&author=Schuler%2CA&author=Shah%2CNH&author=Gallego%2CB"> Google Scholar</a> </p></li><li class="c-article-references__item js-c-reading-companion-references-item" data-counter="63."><p class="c-article-references__text" id="ref-CR63">Schomaker M, Luque-Fernandez MA, Leroy V, Davies MA. Using longitudinal targeted maximum likelihood estimation in complex settings with dynamic interventions. Stat Med. 2019;38(24):4888–911. <a href="https://doi.org/10.1002/sim.8340" data-track="click_references" data-track-action="external reference" data-track-value="external reference" data-track-label="10.1002/sim.8340">https://doi.org/10.1002/sim.8340</a>.</p><p class="c-article-references__links u-hide-print"><a data-track="click_references" rel="nofollow noopener" data-track-label="10.1002/sim.8340" data-track-item_id="10.1002/sim.8340" data-track-value="article reference" data-track-action="article reference" href="https://doi.org/10.1002%2Fsim.8340" aria-label="Article reference 63" data-doi="10.1002/sim.8340">Article</a> <a data-track="click_references" rel="nofollow noopener" data-track-label="link" data-track-item_id="link" data-track-value="pubmed reference" data-track-action="pubmed reference" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Abstract&list_uids=31436859" aria-label="PubMed reference 63">PubMed</a> <a data-track="click_references" rel="nofollow noopener" data-track-label="link" data-track-item_id="link" data-track-value="pubmed central reference" data-track-action="pubmed central reference" href="http://www.ncbi.nlm.nih.gov/pmc/articles/PMC6800798" aria-label="PubMed Central reference 63">PubMed Central</a> <a data-track="click_references" rel="nofollow noopener" data-track-label="link" data-track-item_id="link" data-track-value="cas reference" data-track-action="cas reference" href="/articles/cas-redirect/1:STN:280:DC%2BB3MrgtlOluw%3D%3D" aria-label="CAS reference 63">CAS</a> <a data-track="click_references" data-track-action="google scholar reference" data-track-value="google scholar reference" data-track-label="link" data-track-item_id="link" rel="nofollow noopener" aria-label="Google Scholar reference 63" href="http://scholar.google.com/scholar_lookup?&title=Using%20longitudinal%20targeted%20maximum%20likelihood%20estimation%20in%20complex%20settings%20with%20dynamic%20interventions&journal=Stat%20Med.&doi=10.1002%2Fsim.8340&volume=38&issue=24&pages=4888-4911&publication_year=2019&author=Schomaker%2CM&author=Luque-Fernandez%2CMA&author=Leroy%2CV&author=Davies%2CMA"> Google Scholar</a> </p></li><li class="c-article-references__item js-c-reading-companion-references-item" data-counter="64."><p class="c-article-references__text" id="ref-CR64">Pirracchio R, Petersen ML, van der Laan M. Improving propensity score estimators’ robustness to model misspecification using super learner. Am J Epidemiol. 2015;181(2):108–19. <a href="https://doi.org/10.1093/aje/kwu253" data-track="click_references" data-track-action="external reference" data-track-value="external reference" data-track-label="10.1093/aje/kwu253">https://doi.org/10.1093/aje/kwu253</a>.</p><p class="c-article-references__links u-hide-print"><a data-track="click_references" rel="nofollow noopener" data-track-label="10.1093/aje/kwu253" data-track-item_id="10.1093/aje/kwu253" data-track-value="article reference" data-track-action="article reference" href="https://doi.org/10.1093%2Faje%2Fkwu253" aria-label="Article reference 64" data-doi="10.1093/aje/kwu253">Article</a> <a data-track="click_references" rel="nofollow noopener" data-track-label="link" data-track-item_id="link" data-track-value="pubmed reference" data-track-action="pubmed reference" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Abstract&list_uids=25515168" aria-label="PubMed reference 64">PubMed</a> <a data-track="click_references" data-track-action="google scholar reference" data-track-value="google scholar reference" data-track-label="link" data-track-item_id="link" rel="nofollow noopener" aria-label="Google Scholar reference 64" href="http://scholar.google.com/scholar_lookup?&title=Improving%20propensity%20score%20estimators%E2%80%99%20robustness%20to%20model%20misspecification%20using%20super%20learner&journal=Am%20J%20Epidemiol.&doi=10.1093%2Faje%2Fkwu253&volume=181&issue=2&pages=108-119&publication_year=2015&author=Pirracchio%2CR&author=Petersen%2CML&author=Laan%2CM"> Google Scholar</a> </p></li><li class="c-article-references__item js-c-reading-companion-references-item" data-counter="65."><p class="c-article-references__text" id="ref-CR65">Gottesman O, Johansson F, Komorowski M, Faisal A, Sontag D, Doshi-Velez F, et al. Guidelines for reinforcement learning in healthcare. Nat Med. 2019;25(1):16–8. <a href="https://doi.org/10.1038/s41591-018-0310-5" data-track="click_references" data-track-action="external reference" data-track-value="external reference" data-track-label="10.1038/s41591-018-0310-5">https://doi.org/10.1038/s41591-018-0310-5</a>.</p><p class="c-article-references__links u-hide-print"><a data-track="click_references" rel="nofollow noopener" data-track-label="10.1038/s41591-018-0310-5" data-track-item_id="10.1038/s41591-018-0310-5" data-track-value="article reference" data-track-action="article reference" href="https://doi.org/10.1038%2Fs41591-018-0310-5" aria-label="Article reference 65" data-doi="10.1038/s41591-018-0310-5">Article</a> <a data-track="click_references" rel="nofollow noopener" data-track-label="link" data-track-item_id="link" data-track-value="pubmed reference" data-track-action="pubmed reference" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Abstract&list_uids=30617332" aria-label="PubMed reference 65">PubMed</a> <a data-track="click_references" rel="nofollow noopener" data-track-label="link" data-track-item_id="link" data-track-value="cas reference" data-track-action="cas reference" href="/articles/cas-redirect/1:CAS:528:DC%2BC1MXmvVOgsb4%3D" aria-label="CAS reference 65">CAS</a> <a data-track="click_references" data-track-action="google scholar reference" data-track-value="google scholar reference" data-track-label="link" data-track-item_id="link" rel="nofollow noopener" aria-label="Google Scholar reference 65" href="http://scholar.google.com/scholar_lookup?&title=Guidelines%20for%20reinforcement%20learning%20in%20healthcare&journal=Nat%20Med.&doi=10.1038%2Fs41591-018-0310-5&volume=25&issue=1&pages=16-18&publication_year=2019&author=Gottesman%2CO&author=Johansson%2CF&author=Komorowski%2CM&author=Faisal%2CA&author=Sontag%2CD&author=Doshi-Velez%2CF&author=Celi%2CLA"> Google Scholar</a> </p></li><li class="c-article-references__item js-c-reading-companion-references-item" data-counter="66."><p class="c-article-references__text" id="ref-CR66">Yoon J, Zame WR, Banerjee A, Cadeiras M, Alaa AM, van der Schaar M. Personalized survival predictions via trees of predictors: an application to cardiac transplantation. PLoS One. 2018;13(3):e0194985. <a href="https://doi.org/10.1371/journal.pone.0194985" data-track="click_references" data-track-action="external reference" data-track-value="external reference" data-track-label="10.1371/journal.pone.0194985">https://doi.org/10.1371/journal.pone.0194985</a>.</p><p class="c-article-references__links u-hide-print"><a data-track="click_references" rel="nofollow noopener" data-track-label="10.1371/journal.pone.0194985" data-track-item_id="10.1371/journal.pone.0194985" data-track-value="article reference" data-track-action="article reference" href="https://doi.org/10.1371%2Fjournal.pone.0194985" aria-label="Article reference 66" data-doi="10.1371/journal.pone.0194985">Article</a> <a data-track="click_references" rel="nofollow noopener" data-track-label="link" data-track-item_id="link" data-track-value="pubmed reference" data-track-action="pubmed reference" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Abstract&list_uids=29590219" aria-label="PubMed reference 66">PubMed</a> <a data-track="click_references" rel="nofollow noopener" data-track-label="link" data-track-item_id="link" data-track-value="pubmed central reference" data-track-action="pubmed central reference" href="http://www.ncbi.nlm.nih.gov/pmc/articles/PMC5874060" aria-label="PubMed Central reference 66">PubMed Central</a> <a data-track="click_references" rel="nofollow noopener" data-track-label="link" data-track-item_id="link" data-track-value="cas reference" data-track-action="cas reference" href="/articles/cas-redirect/1:CAS:528:DC%2BC1cXit1ansbjL" aria-label="CAS reference 66">CAS</a> <a data-track="click_references" data-track-action="google scholar reference" data-track-value="google scholar reference" data-track-label="link" data-track-item_id="link" rel="nofollow noopener" aria-label="Google Scholar reference 66" href="http://scholar.google.com/scholar_lookup?&title=Personalized%20survival%20predictions%20via%20trees%20of%20predictors%3A%20an%20application%20to%20cardiac%20transplantation&journal=PLoS%20One.&doi=10.1371%2Fjournal.pone.0194985&volume=13&issue=3&publication_year=2018&author=Yoon%2CJ&author=Zame%2CWR&author=Banerjee%2CA&author=Cadeiras%2CM&author=Alaa%2CAM&author=Schaar%2CM"> Google Scholar</a> </p></li><li class="c-article-references__item js-c-reading-companion-references-item" data-counter="67."><p class="c-article-references__text" id="ref-CR67">Komorowski M, Celi LA, Badawi O, Gordon AC, Faisal AA. The artificial intelligence clinician learns optimal treatment strategies for sepsis in intensive care. Nat Med. 2018;24(11):1716–20. <a href="https://doi.org/10.1038/s41591-018-0213-5" data-track="click_references" data-track-action="external reference" data-track-value="external reference" data-track-label="10.1038/s41591-018-0213-5">https://doi.org/10.1038/s41591-018-0213-5</a>.</p><p class="c-article-references__links u-hide-print"><a data-track="click_references" rel="nofollow noopener" data-track-label="10.1038/s41591-018-0213-5" data-track-item_id="10.1038/s41591-018-0213-5" data-track-value="article reference" data-track-action="article reference" href="https://doi.org/10.1038%2Fs41591-018-0213-5" aria-label="Article reference 67" data-doi="10.1038/s41591-018-0213-5">Article</a> <a data-track="click_references" rel="nofollow noopener" data-track-label="link" data-track-item_id="link" data-track-value="pubmed reference" data-track-action="pubmed reference" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Abstract&list_uids=30349085" aria-label="PubMed reference 67">PubMed</a> <a data-track="click_references" rel="nofollow noopener" data-track-label="link" data-track-item_id="link" data-track-value="cas reference" data-track-action="cas reference" href="/articles/cas-redirect/1:CAS:528:DC%2BC1cXhvF2lsbrF" aria-label="CAS reference 67">CAS</a> <a data-track="click_references" data-track-action="google scholar reference" data-track-value="google scholar reference" data-track-label="link" data-track-item_id="link" rel="nofollow noopener" aria-label="Google Scholar reference 67" href="http://scholar.google.com/scholar_lookup?&title=The%20artificial%20intelligence%20clinician%20learns%20optimal%20treatment%20strategies%20for%20sepsis%20in%20intensive%20care&journal=Nat%20Med.&doi=10.1038%2Fs41591-018-0213-5&volume=24&issue=11&pages=1716-1720&publication_year=2018&author=Komorowski%2CM&author=Celi%2CLA&author=Badawi%2CO&author=Gordon%2CAC&author=Faisal%2CAA"> Google Scholar</a> </p></li><li class="c-article-references__item js-c-reading-companion-references-item" data-counter="68."><p class="c-article-references__text" id="ref-CR68">Ghassemi M, Naumann T, Schulam P, Beam AL, Chen IY, Ranganath R. Practical guidance on artificial intelligence for health-care data. Lancet Digit Health. 2019;1(4):e157–9. <a href="https://doi.org/10.1016/S2589-7500(19)30084-6" data-track="click_references" data-track-action="external reference" data-track-value="external reference" data-track-label="10.1016/S2589-7500(19)30084-6">https://doi.org/10.1016/S2589-7500(19)30084-6</a>.</p><p class="c-article-references__links u-hide-print"><a data-track="click_references" rel="nofollow noopener" data-track-label="10.1016/S2589-7500(19)30084-6" data-track-item_id="10.1016/S2589-7500(19)30084-6" data-track-value="article reference" data-track-action="article reference" href="https://doi.org/10.1016%2FS2589-7500%2819%2930084-6" aria-label="Article reference 68" data-doi="10.1016/S2589-7500(19)30084-6">Article</a> <a data-track="click_references" rel="nofollow noopener" data-track-label="link" data-track-item_id="link" data-track-value="pubmed reference" data-track-action="pubmed reference" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Abstract&list_uids=33323184" aria-label="PubMed reference 68">PubMed</a> <a data-track="click_references" data-track-action="google scholar reference" data-track-value="google scholar reference" data-track-label="link" data-track-item_id="link" rel="nofollow noopener" aria-label="Google Scholar reference 68" href="http://scholar.google.com/scholar_lookup?&title=Practical%20guidance%20on%20artificial%20intelligence%20for%20health-care%20data&journal=Lancet%20Digit%20Health.&doi=10.1016%2FS2589-7500%2819%2930084-6&volume=1&issue=4&pages=e157-e159&publication_year=2019&author=Ghassemi%2CM&author=Naumann%2CT&author=Schulam%2CP&author=Beam%2CAL&author=Chen%2CIY&author=Ranganath%2CR"> Google Scholar</a> </p></li><li class="c-article-references__item js-c-reading-companion-references-item" data-counter="69."><p class="c-article-references__text" id="ref-CR69">Wiens J, Saria S, Sendak M, Ghassemi M, Liu VX, Doshi-Velez F, et al. Do no harm: a roadmap for responsible machine learning for health care. Nat Med. 2019;25(9):1337–40. <a href="https://doi.org/10.1038/s41591-019-0548-6" data-track="click_references" data-track-action="external reference" data-track-value="external reference" data-track-label="10.1038/s41591-019-0548-6">https://doi.org/10.1038/s41591-019-0548-6</a> Epub 2019/08/21. PubMed PMID: 31427808.</p><p class="c-article-references__links u-hide-print"><a data-track="click_references" rel="nofollow noopener" data-track-label="10.1038/s41591-019-0548-6" data-track-item_id="10.1038/s41591-019-0548-6" data-track-value="article reference" data-track-action="article reference" href="https://doi.org/10.1038%2Fs41591-019-0548-6" aria-label="Article reference 69" data-doi="10.1038/s41591-019-0548-6">Article</a> <a data-track="click_references" rel="nofollow noopener" data-track-label="link" data-track-item_id="link" data-track-value="pubmed reference" data-track-action="pubmed reference" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Abstract&list_uids=31427808" aria-label="PubMed reference 69">PubMed</a> <a data-track="click_references" rel="nofollow noopener" data-track-label="link" data-track-item_id="link" data-track-value="cas reference" data-track-action="cas reference" href="/articles/cas-redirect/1:CAS:528:DC%2BC1MXhs1WjurvM" aria-label="CAS reference 69">CAS</a> <a data-track="click_references" data-track-action="google scholar reference" data-track-value="google scholar reference" data-track-label="link" data-track-item_id="link" rel="nofollow noopener" aria-label="Google Scholar reference 69" href="http://scholar.google.com/scholar_lookup?&title=Do%20no%20harm%3A%20a%20roadmap%20for%20responsible%20machine%20learning%20for%20health%20care&journal=Nat%20Med.&doi=10.1038%2Fs41591-019-0548-6&volume=25&issue=9&pages=1337-1340&publication_year=2019&author=Wiens%2CJ&author=Saria%2CS&author=Sendak%2CM&author=Ghassemi%2CM&author=Liu%2CVX&author=Doshi-Velez%2CF&author=Jung%2CK&author=Heller%2CK&author=Kale%2CD&author=Saeed%2CM&author=Ossorio%2CPN&author=Thadaney-Israni%2CS&author=Goldenberg%2CA"> Google Scholar</a> </p></li><li class="c-article-references__item js-c-reading-companion-references-item" data-counter="70."><p class="c-article-references__text" id="ref-CR70">Nestor B, McDermott M, Chauhan G, et al. Rethinking clinical prediction: why machine learning must consider year of care and feature aggregation. arXiv preprint 2018;arXiv:181112583.</p></li><li class="c-article-references__item js-c-reading-companion-references-item" data-counter="71."><p class="c-article-references__text" id="ref-CR71">Johnson AE, Pollard TJ, Shen L, Lehman LW, Feng M, Ghassemi M, et al. MIMIC-III, a freely accessible critical care database. Sci Data. 2016;3(1):160035. <a href="https://doi.org/10.1038/sdata.2016.35" data-track="click_references" data-track-action="external reference" data-track-value="external reference" data-track-label="10.1038/sdata.2016.35">https://doi.org/10.1038/sdata.2016.35</a>.</p><p class="c-article-references__links u-hide-print"><a data-track="click_references" rel="nofollow noopener" data-track-label="10.1038/sdata.2016.35" data-track-item_id="10.1038/sdata.2016.35" data-track-value="article reference" data-track-action="article reference" href="https://doi.org/10.1038%2Fsdata.2016.35" aria-label="Article reference 71" data-doi="10.1038/sdata.2016.35">Article</a> <a data-track="click_references" rel="nofollow noopener" data-track-label="link" data-track-item_id="link" data-track-value="pubmed reference" data-track-action="pubmed reference" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Abstract&list_uids=27219127" aria-label="PubMed reference 71">PubMed</a> <a data-track="click_references" rel="nofollow noopener" data-track-label="link" data-track-item_id="link" data-track-value="pubmed central reference" data-track-action="pubmed central reference" href="http://www.ncbi.nlm.nih.gov/pmc/articles/PMC4878278" aria-label="PubMed Central reference 71">PubMed Central</a> <a data-track="click_references" rel="nofollow noopener" data-track-label="link" data-track-item_id="link" data-track-value="cas reference" data-track-action="cas reference" href="/articles/cas-redirect/1:CAS:528:DC%2BC28Xos1Wnu74%3D" aria-label="CAS reference 71">CAS</a> <a data-track="click_references" data-track-action="google scholar reference" data-track-value="google scholar reference" data-track-label="link" data-track-item_id="link" rel="nofollow noopener" aria-label="Google Scholar reference 71" href="http://scholar.google.com/scholar_lookup?&title=MIMIC-III%2C%20a%20freely%20accessible%20critical%20care%20database&journal=Sci%20Data.&doi=10.1038%2Fsdata.2016.35&volume=3&issue=1&publication_year=2016&author=Johnson%2CAE&author=Pollard%2CTJ&author=Shen%2CL&author=Lehman%2CLW&author=Feng%2CM&author=Ghassemi%2CM"> Google Scholar</a> </p></li><li class="c-article-references__item js-c-reading-companion-references-item" data-counter="72."><p class="c-article-references__text" id="ref-CR72">Pollard TJ, Johnson AEW, Raffa JD, Celi LA, Mark RG, Badawi O. The eICU Collaborative Research Database, a freely available multi-center database for critical care research. Sci Data. 2018;5(1):180178. <a href="https://doi.org/10.1038/sdata.2018.178" data-track="click_references" data-track-action="external reference" data-track-value="external reference" data-track-label="10.1038/sdata.2018.178">https://doi.org/10.1038/sdata.2018.178</a>.</p><p class="c-article-references__links u-hide-print"><a data-track="click_references" rel="nofollow noopener" data-track-label="10.1038/sdata.2018.178" data-track-item_id="10.1038/sdata.2018.178" data-track-value="article reference" data-track-action="article reference" href="https://doi.org/10.1038%2Fsdata.2018.178" aria-label="Article reference 72" data-doi="10.1038/sdata.2018.178">Article</a> <a data-track="click_references" rel="nofollow noopener" data-track-label="link" data-track-item_id="link" data-track-value="pubmed reference" data-track-action="pubmed reference" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Abstract&list_uids=30204154" aria-label="PubMed reference 72">PubMed</a> <a data-track="click_references" rel="nofollow noopener" data-track-label="link" data-track-item_id="link" data-track-value="pubmed central reference" data-track-action="pubmed central reference" href="http://www.ncbi.nlm.nih.gov/pmc/articles/PMC6132188" aria-label="PubMed Central reference 72">PubMed Central</a> <a data-track="click_references" data-track-action="google scholar reference" data-track-value="google scholar reference" data-track-label="link" data-track-item_id="link" rel="nofollow noopener" aria-label="Google Scholar reference 72" href="http://scholar.google.com/scholar_lookup?&title=The%20eICU%20Collaborative%20Research%20Database%2C%20a%20freely%20available%20multi-center%20database%20for%20critical%20care%20research&journal=Sci%20Data.&doi=10.1038%2Fsdata.2018.178&volume=5&issue=1&publication_year=2018&author=Pollard%2CTJ&author=Johnson%2CAEW&author=Raffa%2CJD&author=Celi%2CLA&author=Mark%2CRG&author=Badawi%2CO"> Google Scholar</a> </p></li><li class="c-article-references__item js-c-reading-companion-references-item" data-counter="73."><p class="c-article-references__text" id="ref-CR73">UK Biobank. <a href="http://www.ukbiobank.ac.uk" data-track="click_references" data-track-action="external reference" data-track-value="external reference" data-track-label="http://www.ukbiobank.ac.uk">www.ukbiobank.ac.uk</a>. Accessed 22 Mar 2021.</p></li><li class="c-article-references__item js-c-reading-companion-references-item" data-counter="74."><p class="c-article-references__text" id="ref-CR74">Gong JJ, Naumann T, Szolovits P, Guttag JV. Predicting clinical outcomes across changing electronic health record systems. In: Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. Halifax: Association for Computing Machinery; 2017. p. 1497–505.</p><p class="c-article-references__links u-hide-print"><a data-track="click_references" rel="nofollow noopener" data-track-label="10.1145/3097983.3098064" data-track-item_id="10.1145/3097983.3098064" data-track-value="chapter reference" data-track-action="chapter reference" href="https://doi.org/10.1145%2F3097983.3098064" aria-label="Chapter reference 74" data-doi="10.1145/3097983.3098064">Chapter</a> <a data-track="click_references" data-track-action="google scholar reference" data-track-value="google scholar reference" data-track-label="link" data-track-item_id="link" rel="nofollow noopener" aria-label="Google Scholar reference 74" href="http://scholar.google.com/scholar_lookup?&title=Predicting%20clinical%20outcomes%20across%20changing%20electronic%20health%20record%20systems&doi=10.1145%2F3097983.3098064&pages=1497-1505&publication_year=2017&author=Gong%2CJJ&author=Naumann%2CT&author=Szolovits%2CP&author=Guttag%2CJV"> Google Scholar</a> </p></li><li class="c-article-references__item js-c-reading-companion-references-item" data-counter="75."><p class="c-article-references__text" id="ref-CR75">Beam AL, Manrai AK, Ghassemi M. Challenges to the reproducibility of machine learning models in health care. JAMA. 2020;323(4):305–6. <a href="https://doi.org/10.1001/jama.2019.20866" data-track="click_references" data-track-action="external reference" data-track-value="external reference" data-track-label="10.1001/jama.2019.20866">https://doi.org/10.1001/jama.2019.20866</a>.</p></li><li class="c-article-references__item js-c-reading-companion-references-item" data-counter="76."><p class="c-article-references__text" id="ref-CR76">Adebayo J, Gilmer J, Muelly M, Goodfellow I, Hardt M, Kim B. Sanity checks for saliency maps. In: Proceedings of the 32nd International Conference on Neural Information Processing Systems. Montréal: Curran Associates Inc.; 2018. p. 9525–36.</p><p class="c-article-references__links u-hide-print"><a data-track="click_references" data-track-action="google scholar reference" data-track-value="google scholar reference" data-track-label="link" data-track-item_id="link" rel="nofollow noopener" aria-label="Google Scholar reference 76" href="http://scholar.google.com/scholar_lookup?&title=Sanity%20checks%20for%20saliency%20maps&pages=9525-9536&publication_year=2018&author=Adebayo%2CJ&author=Gilmer%2CJ&author=Muelly%2CM&author=Goodfellow%2CI&author=Hardt%2CM&author=Kim%2CB"> Google Scholar</a> </p></li><li class="c-article-references__item js-c-reading-companion-references-item" data-counter="77."><p class="c-article-references__text" id="ref-CR77">Wiegreffe S, Pinter Y. Attention is not not explanation. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). Hong Kong, China: Association for Computational Linguistics; 2019.</p><p class="c-article-references__links u-hide-print"><a data-track="click_references" data-track-action="google scholar reference" data-track-value="google scholar reference" data-track-label="link" data-track-item_id="link" rel="nofollow noopener" aria-label="Google Scholar reference 77" href="http://scholar.google.com/scholar_lookup?&title=Attention%20is%20not%20not%20explanation.%20Proceedings%20of%20the%202019%20Conference%20on%20Empirical%20Methods%20in%20Natural%20Language%20Processing%20and%20the%209th%20International%20Joint%20Conference%20on%20Natural%20Language%20Processing%20%28EMNLP-IJCNLP%29&publication_year=2019&author=Wiegreffe%2CS&author=Pinter%2CY"> Google Scholar</a> </p></li><li class="c-article-references__item js-c-reading-companion-references-item" data-counter="78."><p class="c-article-references__text" id="ref-CR78">Jain S, Wallace BC. Attention is not explanation: NAACL-HLT; 2019.</p><p class="c-article-references__links u-hide-print"><a data-track="click_references" data-track-action="google scholar reference" data-track-value="google scholar reference" data-track-label="link" data-track-item_id="link" rel="nofollow noopener" aria-label="Google Scholar reference 78" href="http://scholar.google.com/scholar_lookup?&title=Attention%20is%20not%20explanation&publication_year=2019&author=Jain%2CS&author=Wallace%2CBC"> Google Scholar</a> </p></li><li class="c-article-references__item js-c-reading-companion-references-item" data-counter="79."><p class="c-article-references__text" id="ref-CR79">Serrano S, Smith NA. Is attention interpretable? In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 2931–2951, Stroudsburg, PA, USA, 2019. Association for Computational Linguistics.</p></li><li class="c-article-references__item js-c-reading-companion-references-item" data-counter="80."><p class="c-article-references__text" id="ref-CR80">Sendak M, Elish MC, Gao M, Futoma J, Ratliff W, Nichols M, et al. “The human body is a black box”: supporting clinical decision-making with deep learning. In: Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency. Barcelona: Association for Computing Machinery; 2020. p. 99–109.</p><p class="c-article-references__links u-hide-print"><a data-track="click_references" rel="nofollow noopener" data-track-label="10.1145/3351095.3372827" data-track-item_id="10.1145/3351095.3372827" data-track-value="chapter reference" data-track-action="chapter reference" href="https://doi.org/10.1145%2F3351095.3372827" aria-label="Chapter reference 80" data-doi="10.1145/3351095.3372827">Chapter</a> <a data-track="click_references" data-track-action="google scholar reference" data-track-value="google scholar reference" data-track-label="link" data-track-item_id="link" rel="nofollow noopener" aria-label="Google Scholar reference 80" href="http://scholar.google.com/scholar_lookup?&title=%E2%80%9CThe%20human%20body%20is%20a%20black%20box%E2%80%9D%3A%20supporting%20clinical%20decision-making%20with%20deep%20learning&doi=10.1145%2F3351095.3372827&pages=99-109&publication_year=2020&author=Sendak%2CM&author=Elish%2CMC&author=Gao%2CM&author=Futoma%2CJ&author=Ratliff%2CW&author=Nichols%2CM"> Google Scholar</a> </p></li><li class="c-article-references__item js-c-reading-companion-references-item" data-counter="81."><p class="c-article-references__text" id="ref-CR81">Angwin J LJ, Mattu S, Kirchner L. Machine bias. ProPublica. 2016 13 May 2020. Available from: <a href="https://www.propublica.org/article/machine-bias-risk-assessments-in-criminal-sentencing" data-track="click_references" data-track-action="external reference" data-track-value="external reference" data-track-label="https://www.propublica.org/article/machine-bias-risk-assessments-in-criminal-sentencing">https://www.propublica.org/article/machine-bias-risk-assessments-in-criminal-sentencing</a>.</p><p class="c-article-references__links u-hide-print"><a data-track="click_references" data-track-action="google scholar reference" data-track-value="google scholar reference" data-track-label="link" data-track-item_id="link" rel="nofollow noopener" aria-label="Google Scholar reference 81" href="http://scholar.google.com/scholar_lookup?&title=Machine%20bias.%20ProPublica&publication_year=2016&author=Angwin%2CJLJ&author=Mattu%2CS&author=Kirchner%2CL"> Google Scholar</a> </p></li><li class="c-article-references__item js-c-reading-companion-references-item" data-counter="82."><p class="c-article-references__text" id="ref-CR82">Qualls LG, Phillips TA, Hammill BG, Topping J, Louzao DM, Brown JS, et al. Evaluating foundational data quality in the National Patient-Centered Clinical Research Network (PCORnet(R)). EGEMS (Wash DC). 2018;6(1):3.</p><p class="c-article-references__links u-hide-print"><a data-track="click_references" rel="nofollow noopener" data-track-label="link" data-track-item_id="link" data-track-value="pubmed central reference" data-track-action="pubmed central reference" href="http://www.ncbi.nlm.nih.gov/pmc/articles/PMC5983028" aria-label="PubMed Central reference 82">PubMed Central</a> <a data-track="click_references" data-track-action="google scholar reference" data-track-value="google scholar reference" data-track-label="link" data-track-item_id="link" rel="nofollow noopener" aria-label="Google Scholar reference 82" href="http://scholar.google.com/scholar_lookup?&title=Evaluating%20foundational%20data%20quality%20in%20the%20National%20Patient-Centered%20Clinical%20Research%20Network%20%28PCORnet%28R%29%29&journal=EGEMS%20%28Wash%20DC%29&volume=6&issue=1&publication_year=2018&author=Qualls%2CLG&author=Phillips%2CTA&author=Hammill%2CBG&author=Topping%2CJ&author=Louzao%2CDM&author=Brown%2CJS"> Google Scholar</a> </p></li><li class="c-article-references__item js-c-reading-companion-references-item" data-counter="83."><p class="c-article-references__text" id="ref-CR83">Bosca D, Moner D, Maldonado JA, Robles M. Combining archetypes with fast health interoperability resources in future-proof health information systems. Stud Health Technol Inform. 2015;210:180–4.</p><p class="c-article-references__links u-hide-print"><a data-track="click_references" rel="nofollow noopener" data-track-label="link" data-track-item_id="link" data-track-value="pubmed reference" data-track-action="pubmed reference" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Abstract&list_uids=25991126" aria-label="PubMed reference 83">PubMed</a> <a data-track="click_references" data-track-action="google scholar reference" data-track-value="google scholar reference" data-track-label="link" data-track-item_id="link" rel="nofollow noopener" aria-label="Google Scholar reference 83" href="http://scholar.google.com/scholar_lookup?&title=Combining%20archetypes%20with%20fast%20health%20interoperability%20resources%20in%20future-proof%20health%20information%20systems&journal=Stud%20Health%20Technol%20Inform.&volume=210&pages=180-184&publication_year=2015&author=Bosca%2CD&author=Moner%2CD&author=Maldonado%2CJA&author=Robles%2CM"> Google Scholar</a> </p></li><li class="c-article-references__item js-c-reading-companion-references-item" data-counter="84."><p class="c-article-references__text" id="ref-CR84">Klann JG, Abend A, Raghavan VA, Mandl KD, Murphy SN. Data interchange using i2b2. J Am Med Inform Assoc. 2016;23(5):909–15. <a href="https://doi.org/10.1093/jamia/ocv188" data-track="click_references" data-track-action="external reference" data-track-value="external reference" data-track-label="10.1093/jamia/ocv188">https://doi.org/10.1093/jamia/ocv188</a>.</p><p class="c-article-references__links u-hide-print"><a data-track="click_references" rel="nofollow noopener" data-track-label="10.1093/jamia/ocv188" data-track-item_id="10.1093/jamia/ocv188" data-track-value="article reference" data-track-action="article reference" href="https://doi.org/10.1093%2Fjamia%2Focv188" aria-label="Article reference 84" data-doi="10.1093/jamia/ocv188">Article</a> <a data-track="click_references" rel="nofollow noopener" data-track-label="link" data-track-item_id="link" data-track-value="pubmed reference" data-track-action="pubmed reference" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Abstract&list_uids=26911824" aria-label="PubMed reference 84">PubMed</a> <a data-track="click_references" rel="nofollow noopener" data-track-label="link" data-track-item_id="link" data-track-value="pubmed central reference" data-track-action="pubmed central reference" href="http://www.ncbi.nlm.nih.gov/pmc/articles/PMC4997035" aria-label="PubMed Central reference 84">PubMed Central</a> <a data-track="click_references" data-track-action="google scholar reference" data-track-value="google scholar reference" data-track-label="link" data-track-item_id="link" rel="nofollow noopener" aria-label="Google Scholar reference 84" href="http://scholar.google.com/scholar_lookup?&title=Data%20interchange%20using%20i2b2&journal=J%20Am%20Med%20Inform%20Assoc.&doi=10.1093%2Fjamia%2Focv188&volume=23&issue=5&pages=909-915&publication_year=2016&author=Klann%2CJG&author=Abend%2CA&author=Raghavan%2CVA&author=Mandl%2CKD&author=Murphy%2CSN"> Google Scholar</a> </p></li><li class="c-article-references__item js-c-reading-companion-references-item" data-counter="85."><p class="c-article-references__text" id="ref-CR85">Overhage JM, Ryan PB, Reich CG, Hartzema AG, Stang PE. Validation of a common data model for active safety surveillance research. J Am Med Inform Assoc. 2012;19(1):54–60. <a href="https://doi.org/10.1136/amiajnl-2011-000376" data-track="click_references" data-track-action="external reference" data-track-value="external reference" data-track-label="10.1136/amiajnl-2011-000376">https://doi.org/10.1136/amiajnl-2011-000376</a>.</p><p class="c-article-references__links u-hide-print"><a data-track="click_references" rel="nofollow noopener" data-track-label="10.1136/amiajnl-2011-000376" data-track-item_id="10.1136/amiajnl-2011-000376" data-track-value="article reference" data-track-action="article reference" href="https://doi.org/10.1136%2Famiajnl-2011-000376" aria-label="Article reference 85" data-doi="10.1136/amiajnl-2011-000376">Article</a> <a data-track="click_references" rel="nofollow noopener" data-track-label="link" data-track-item_id="link" data-track-value="pubmed reference" data-track-action="pubmed reference" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Abstract&list_uids=22037893" aria-label="PubMed reference 85">PubMed</a> <a data-track="click_references" data-track-action="google scholar reference" data-track-value="google scholar reference" data-track-label="link" data-track-item_id="link" rel="nofollow noopener" aria-label="Google Scholar reference 85" href="http://scholar.google.com/scholar_lookup?&title=Validation%20of%20a%20common%20data%20model%20for%20active%20safety%20surveillance%20research&journal=J%20Am%20Med%20Inform%20Assoc.&doi=10.1136%2Famiajnl-2011-000376&volume=19&issue=1&pages=54-60&publication_year=2012&author=Overhage%2CJM&author=Ryan%2CPB&author=Reich%2CCG&author=Hartzema%2CAG&author=Stang%2CPE"> Google Scholar</a> </p></li><li class="c-article-references__item js-c-reading-companion-references-item" data-counter="86."><p class="c-article-references__text" id="ref-CR86">21st Century Cures Act: Interoperability, information blocking, and the ONC Health IT Certification Program [updated 1 May 2020]. Available from: <a href="https://www.federalregister.gov/documents/2020/05/01/2020-07419/21st-century-cures-act-interoperability-information-blocking-and-the-onc-health-it-certification" data-track="click_references" data-track-action="external reference" data-track-value="external reference" data-track-label="https://www.federalregister.gov/documents/2020/05/01/2020-07419/21st-century-cures-act-interoperability-information-blocking-and-the-onc-health-it-certification">https://www.federalregister.gov/documents/2020/05/01/2020-07419/21st-century-cures-act-interoperability-information-blocking-and-the-onc-health-it-certification</a>. Accessed 16 May 2020.</p></li><li class="c-article-references__item js-c-reading-companion-references-item" data-counter="87."><p class="c-article-references__text" id="ref-CR87">Oh M, Park S, Kim S, Chae H. Machine learning-based analysis of multi-omics data on the cloud for investigating gene regulations. Brief Bioinform. 2020. Epub 2020/04/01. <a href="https://doi.org/10.1093/bib/bbaa032" data-track="click_references" data-track-action="external reference" data-track-value="external reference" data-track-label="10.1093/bib/bbaa032">https://doi.org/10.1093/bib/bbaa032</a>.</p></li><li class="c-article-references__item js-c-reading-companion-references-item" data-counter="88."><p class="c-article-references__text" id="ref-CR88">Czeizler E, Wiessler W, Koester T, Hakala M, Basiri S, Jordan P, et al. Using federated data sources and Varian Learning Portal framework to train a neural network model for automatic organ segmentation. Phys Med. 2020;72:39–45. <a href="https://doi.org/10.1016/j.ejmp.2020.03.011" data-track="click_references" data-track-action="external reference" data-track-value="external reference" data-track-label="10.1016/j.ejmp.2020.03.011">https://doi.org/10.1016/j.ejmp.2020.03.011</a>.</p><p class="c-article-references__links u-hide-print"><a data-track="click_references" rel="nofollow noopener" data-track-label="10.1016/j.ejmp.2020.03.011" data-track-item_id="10.1016/j.ejmp.2020.03.011" data-track-value="article reference" data-track-action="article reference" href="https://doi.org/10.1016%2Fj.ejmp.2020.03.011" aria-label="Article reference 88" data-doi="10.1016/j.ejmp.2020.03.011">Article</a> <a data-track="click_references" rel="nofollow noopener" data-track-label="link" data-track-item_id="link" data-track-value="pubmed reference" data-track-action="pubmed reference" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Abstract&list_uids=32197221" aria-label="PubMed reference 88">PubMed</a> <a data-track="click_references" data-track-action="google scholar reference" data-track-value="google scholar reference" data-track-label="link" data-track-item_id="link" rel="nofollow noopener" aria-label="Google Scholar reference 88" href="http://scholar.google.com/scholar_lookup?&title=Using%20federated%20data%20sources%20and%20Varian%20Learning%20Portal%20framework%20to%20train%20a%20neural%20network%20model%20for%20automatic%20organ%20segmentation&journal=Phys%20Med.&doi=10.1016%2Fj.ejmp.2020.03.011&volume=72&pages=39-45&publication_year=2020&author=Czeizler%2CE&author=Wiessler%2CW&author=Koester%2CT&author=Hakala%2CM&author=Basiri%2CS&author=Jordan%2CP&author=Kuusela%2CE"> Google Scholar</a> </p></li><li class="c-article-references__item js-c-reading-companion-references-item" data-counter="89."><p class="c-article-references__text" id="ref-CR89">Zerka F, Barakat S, Walsh S, Bogowicz M, Leijenaar RTH, Jochems A, et al. Systematic review of privacy-preserving distributed machine learning from federated databases in health care. JCO Clin Cancer Inform. 2020;4:184–200. <a href="https://doi.org/10.1200/CCI.19.00047" data-track="click_references" data-track-action="external reference" data-track-value="external reference" data-track-label="10.1200/CCI.19.00047">https://doi.org/10.1200/CCI.19.00047</a>.</p><p class="c-article-references__links u-hide-print"><a data-track="click_references" rel="nofollow noopener" data-track-label="10.1200/CCI.19.00047" data-track-item_id="10.1200/CCI.19.00047" data-track-value="article reference" data-track-action="article reference" href="https://doi.org/10.1200%2FCCI.19.00047" aria-label="Article reference 89" data-doi="10.1200/CCI.19.00047">Article</a> <a data-track="click_references" rel="nofollow noopener" data-track-label="link" data-track-item_id="link" data-track-value="pubmed reference" data-track-action="pubmed reference" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Abstract&list_uids=32134684" aria-label="PubMed reference 89">PubMed</a> <a data-track="click_references" data-track-action="google scholar reference" data-track-value="google scholar reference" data-track-label="link" data-track-item_id="link" rel="nofollow noopener" aria-label="Google Scholar reference 89" href="http://scholar.google.com/scholar_lookup?&title=Systematic%20review%20of%20privacy-preserving%20distributed%20machine%20learning%20from%20federated%20databases%20in%20health%20care&journal=JCO%20Clin%20Cancer%20Inform.&doi=10.1200%2FCCI.19.00047&volume=4&pages=184-200&publication_year=2020&author=Zerka%2CF&author=Barakat%2CS&author=Walsh%2CS&author=Bogowicz%2CM&author=Leijenaar%2CRTH&author=Jochems%2CA&author=Miraglio%2CB&author=Townend%2CD&author=Lambin%2CP"> Google Scholar</a> </p></li><li class="c-article-references__item js-c-reading-companion-references-item" data-counter="90."><p class="c-article-references__text" id="ref-CR90">McCarty CA, Chisholm RL, Chute CG, Kullo IJ, Jarvik GP, Larson EB, et al. The eMERGE Network: a consortium of biorepositories linked to electronic medical records data for conducting genomic studies. BMC Med Genomics. 2011;4(1):13. <a href="https://doi.org/10.1186/1755-8794-4-13" data-track="click_references" data-track-action="external reference" data-track-value="external reference" data-track-label="10.1186/1755-8794-4-13">https://doi.org/10.1186/1755-8794-4-13</a>.</p><p class="c-article-references__links u-hide-print"><a data-track="click_references" rel="noopener" data-track-label="10.1186/1755-8794-4-13" data-track-item_id="10.1186/1755-8794-4-13" data-track-value="article reference" data-track-action="article reference" href="https://link.springer.com/doi/10.1186/1755-8794-4-13" aria-label="Article reference 90" data-doi="10.1186/1755-8794-4-13">Article</a> <a data-track="click_references" rel="nofollow noopener" data-track-label="link" data-track-item_id="link" data-track-value="pubmed reference" data-track-action="pubmed reference" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Abstract&list_uids=21269473" aria-label="PubMed reference 90">PubMed</a> <a data-track="click_references" rel="nofollow noopener" data-track-label="link" data-track-item_id="link" data-track-value="pubmed central reference" data-track-action="pubmed central reference" href="http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3038887" aria-label="PubMed Central reference 90">PubMed Central</a> <a data-track="click_references" data-track-action="google scholar reference" data-track-value="google scholar reference" data-track-label="link" data-track-item_id="link" rel="nofollow noopener" aria-label="Google Scholar reference 90" href="http://scholar.google.com/scholar_lookup?&title=The%20eMERGE%20Network%3A%20a%20consortium%20of%20biorepositories%20linked%20to%20electronic%20medical%20records%20data%20for%20conducting%20genomic%20studies&journal=BMC%20Med%20Genomics.&doi=10.1186%2F1755-8794-4-13&volume=4&issue=1&publication_year=2011&author=McCarty%2CCA&author=Chisholm%2CRL&author=Chute%2CCG&author=Kullo%2CIJ&author=Jarvik%2CGP&author=Larson%2CEB"> Google Scholar</a> </p></li><li class="c-article-references__item js-c-reading-companion-references-item" data-counter="91."><p class="c-article-references__text" id="ref-CR91">Boyce RD, Ryan PB, Noren GN, Schuemie MJ, Reich C, Duke J, et al. Bridging islands of information to establish an integrated knowledge base of drugs and health outcomes of interest. Drug Saf. 2014;37(8):557–67. <a href="https://doi.org/10.1007/s40264-014-0189-0" data-track="click_references" data-track-action="external reference" data-track-value="external reference" data-track-label="10.1007/s40264-014-0189-0">https://doi.org/10.1007/s40264-014-0189-0</a>.</p><p class="c-article-references__links u-hide-print"><a data-track="click_references" rel="noopener" data-track-label="10.1007/s40264-014-0189-0" data-track-item_id="10.1007/s40264-014-0189-0" data-track-value="article reference" data-track-action="article reference" href="https://link.springer.com/doi/10.1007/s40264-014-0189-0" aria-label="Article reference 91" data-doi="10.1007/s40264-014-0189-0">Article</a> <a data-track="click_references" rel="nofollow noopener" data-track-label="link" data-track-item_id="link" data-track-value="pubmed reference" data-track-action="pubmed reference" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Abstract&list_uids=24985530" aria-label="PubMed reference 91">PubMed</a> <a data-track="click_references" rel="nofollow noopener" data-track-label="link" data-track-item_id="link" data-track-value="pubmed central reference" data-track-action="pubmed central reference" href="http://www.ncbi.nlm.nih.gov/pmc/articles/PMC4134480" aria-label="PubMed Central reference 91">PubMed Central</a> <a data-track="click_references" rel="nofollow noopener" data-track-label="link" data-track-item_id="link" data-track-value="cas reference" data-track-action="cas reference" href="/articles/cas-redirect/1:CAS:528:DC%2BC2cXhtV2jur7M" aria-label="CAS reference 91">CAS</a> <a data-track="click_references" data-track-action="google scholar reference" data-track-value="google scholar reference" data-track-label="link" data-track-item_id="link" rel="nofollow noopener" aria-label="Google Scholar reference 91" href="http://scholar.google.com/scholar_lookup?&title=Bridging%20islands%20of%20information%20to%20establish%20an%20integrated%20knowledge%20base%20of%20drugs%20and%20health%20outcomes%20of%20interest&journal=Drug%20Saf.&doi=10.1007%2Fs40264-014-0189-0&volume=37&issue=8&pages=557-567&publication_year=2014&author=Boyce%2CRD&author=Ryan%2CPB&author=Noren%2CGN&author=Schuemie%2CMJ&author=Reich%2CC&author=Duke%2CJ"> Google Scholar</a> </p></li><li class="c-article-references__item js-c-reading-companion-references-item" data-counter="92."><p class="c-article-references__text" id="ref-CR92">van Klaveren D, Steyerberg EW, Serruys PW, Kent DM. The proposed ‘concordance-statistic for benefit’ provided a useful metric when modeling heterogeneous treatment effects. J Clin Epidemiol. 2018;94:59–68. <a href="https://doi.org/10.1016/j.jclinepi.2017.10.021" data-track="click_references" data-track-action="external reference" data-track-value="external reference" data-track-label="10.1016/j.jclinepi.2017.10.021">https://doi.org/10.1016/j.jclinepi.2017.10.021</a>.</p><p class="c-article-references__links u-hide-print"><a data-track="click_references" rel="nofollow noopener" data-track-label="10.1016/j.jclinepi.2017.10.021" data-track-item_id="10.1016/j.jclinepi.2017.10.021" data-track-value="article reference" data-track-action="article reference" href="https://doi.org/10.1016%2Fj.jclinepi.2017.10.021" aria-label="Article reference 92" data-doi="10.1016/j.jclinepi.2017.10.021">Article</a> <a data-track="click_references" rel="nofollow noopener" data-track-label="link" data-track-item_id="link" data-track-value="pubmed reference" data-track-action="pubmed reference" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Abstract&list_uids=29132832" aria-label="PubMed reference 92">PubMed</a> <a data-track="click_references" data-track-action="google scholar reference" data-track-value="google scholar reference" data-track-label="link" data-track-item_id="link" rel="nofollow noopener" aria-label="Google Scholar reference 92" href="http://scholar.google.com/scholar_lookup?&title=The%20proposed%20%E2%80%98concordance-statistic%20for%20benefit%E2%80%99%20provided%20a%20useful%20metric%20when%20modeling%20heterogeneous%20treatment%20effects&journal=J%20Clin%20Epidemiol.&doi=10.1016%2Fj.jclinepi.2017.10.021&volume=94&pages=59-68&publication_year=2018&author=Klaveren%2CD&author=Steyerberg%2CEW&author=Serruys%2CPW&author=Kent%2CDM"> Google Scholar</a> </p></li><li class="c-article-references__item js-c-reading-companion-references-item" data-counter="93."><p class="c-article-references__text" id="ref-CR93">Robbins RBE. An invisible hand: patients aren’t being told about the AI systems advising their care. STAT; 2020.</p><p class="c-article-references__links u-hide-print"><a data-track="click_references" data-track-action="google scholar reference" data-track-value="google scholar reference" data-track-label="link" data-track-item_id="link" rel="nofollow noopener" aria-label="Google Scholar reference 93" href="http://scholar.google.com/scholar_lookup?&title=An%20invisible%20hand%3A%20patients%20aren%E2%80%99t%20being%20told%20about%20the%20AI%20systems%20advising%20their%20care.%20STAT&publication_year=2020&author=Robbins%2CRBE"> Google Scholar</a> </p></li><li class="c-article-references__item js-c-reading-companion-references-item" data-counter="94."><p class="c-article-references__text" id="ref-CR94">Sterckx S, Rakic V, Cockbain J, Borry P. “You hoped we would sleep walk into accepting the collection of our data”: controversies surrounding the UK care.data scheme and their wider relevance for biomedical research. Med Health Care Philos. 2016;19(2):177–90. <a href="https://doi.org/10.1007/s11019-015-9661-6" data-track="click_references" data-track-action="external reference" data-track-value="external reference" data-track-label="10.1007/s11019-015-9661-6">https://doi.org/10.1007/s11019-015-9661-6</a>.</p><p class="c-article-references__links u-hide-print"><a data-track="click_references" rel="noopener" data-track-label="10.1007/s11019-015-9661-6" data-track-item_id="10.1007/s11019-015-9661-6" data-track-value="article reference" data-track-action="article reference" href="https://link.springer.com/doi/10.1007/s11019-015-9661-6" aria-label="Article reference 94" data-doi="10.1007/s11019-015-9661-6">Article</a> <a data-track="click_references" rel="nofollow noopener" data-track-label="link" data-track-item_id="link" data-track-value="pubmed reference" data-track-action="pubmed reference" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Abstract&list_uids=26280642" aria-label="PubMed reference 94">PubMed</a> <a data-track="click_references" data-track-action="google scholar reference" data-track-value="google scholar reference" data-track-label="link" data-track-item_id="link" rel="nofollow noopener" aria-label="Google Scholar reference 94" href="http://scholar.google.com/scholar_lookup?&title=%E2%80%9CYou%20hoped%20we%20would%20sleep%20walk%20into%20accepting%20the%20collection%20of%20our%20data%E2%80%9D%3A%20controversies%20surrounding%20the%20UK%20care.data%20scheme%20and%20their%20wider%20relevance%20for%20biomedical%20research&journal=Med%20Health%20Care%20Philos.&doi=10.1007%2Fs11019-015-9661-6&volume=19&issue=2&pages=177-190&publication_year=2016&author=Sterckx%2CS&author=Rakic%2CV&author=Cockbain%2CJ&author=Borry%2CP"> Google Scholar</a> </p></li><li class="c-article-references__item js-c-reading-companion-references-item" data-counter="95."><p class="c-article-references__text" id="ref-CR95">Committee on Understanding and Eliminating Racial and Ethnic Disparities in Health Care. Confronting racial and ethnic disparities in health care. Washington (DC): National Academies Press; 2003.</p><p class="c-article-references__links u-hide-print"><a data-track="click_references" data-track-action="google scholar reference" data-track-value="google scholar reference" data-track-label="link" data-track-item_id="link" rel="nofollow noopener" aria-label="Google Scholar reference 95" href="http://scholar.google.com/scholar_lookup?&title=Confronting%20racial%20and%20ethnic%20disparities%20in%20health%20care&publication_year=2003"> Google Scholar</a> </p></li><li class="c-article-references__item js-c-reading-companion-references-item" data-counter="96."><p class="c-article-references__text" id="ref-CR96">Criado PC. Invisible women. New York: Harry N. Abrams; 2019.</p><p class="c-article-references__links u-hide-print"><a data-track="click_references" data-track-action="google scholar reference" data-track-value="google scholar reference" data-track-label="link" data-track-item_id="link" rel="nofollow noopener" aria-label="Google Scholar reference 96" href="http://scholar.google.com/scholar_lookup?&title=Invisible%20women&publication_year=2019&author=Criado%2CPC"> Google Scholar</a> </p></li><li class="c-article-references__item js-c-reading-companion-references-item" data-counter="97."><p class="c-article-references__text" id="ref-CR97">Zhang H, Lu AX, Abdalla M, McDermott M, Ghassemi M. Hurtful words: quantifying biases in clinical contextual word embeddings. In: Proceedings of the ACM Conference on Health, Inference, and Learning. Toronto: Association for Computing Machinery; 2020. p. 110–20.</p><p class="c-article-references__links u-hide-print"><a data-track="click_references" rel="nofollow noopener" data-track-label="10.1145/3368555.3384448" data-track-item_id="10.1145/3368555.3384448" data-track-value="chapter reference" data-track-action="chapter reference" href="https://doi.org/10.1145%2F3368555.3384448" aria-label="Chapter reference 97" data-doi="10.1145/3368555.3384448">Chapter</a> <a data-track="click_references" data-track-action="google scholar reference" data-track-value="google scholar reference" data-track-label="link" data-track-item_id="link" rel="nofollow noopener" aria-label="Google Scholar reference 97" href="http://scholar.google.com/scholar_lookup?&title=Hurtful%20words%3A%20quantifying%20biases%20in%20clinical%20contextual%20word%20embeddings&doi=10.1145%2F3368555.3384448&pages=110-120&publication_year=2020&author=Zhang%2CH&author=Lu%2CAX&author=Abdalla%2CM&author=McDermott%2CM&author=Ghassemi%2CM"> Google Scholar</a> </p></li><li class="c-article-references__item js-c-reading-companion-references-item" data-counter="98."><p class="c-article-references__text" id="ref-CR98">Chen IY, Joshi S, Ghassemi M. Treating health disparities with artificial intelligence. Nat Med. 2020;26(1):16–7. <a href="https://doi.org/10.1038/s41591-019-0649-2" data-track="click_references" data-track-action="external reference" data-track-value="external reference" data-track-label="10.1038/s41591-019-0649-2">https://doi.org/10.1038/s41591-019-0649-2</a>.</p><p class="c-article-references__links u-hide-print"><a data-track="click_references" rel="nofollow noopener" data-track-label="10.1038/s41591-019-0649-2" data-track-item_id="10.1038/s41591-019-0649-2" data-track-value="article reference" data-track-action="article reference" href="https://doi.org/10.1038%2Fs41591-019-0649-2" aria-label="Article reference 98" data-doi="10.1038/s41591-019-0649-2">Article</a> <a data-track="click_references" rel="nofollow noopener" data-track-label="link" data-track-item_id="link" data-track-value="pubmed reference" data-track-action="pubmed reference" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Abstract&list_uids=31932779" aria-label="PubMed reference 98">PubMed</a> <a data-track="click_references" rel="nofollow noopener" data-track-label="link" data-track-item_id="link" data-track-value="cas reference" data-track-action="cas reference" href="/articles/cas-redirect/1:CAS:528:DC%2BB3cXotFOjtQ%3D%3D" aria-label="CAS reference 98">CAS</a> <a data-track="click_references" data-track-action="google scholar reference" data-track-value="google scholar reference" data-track-label="link" data-track-item_id="link" rel="nofollow noopener" aria-label="Google Scholar reference 98" href="http://scholar.google.com/scholar_lookup?&title=Treating%20health%20disparities%20with%20artificial%20intelligence&journal=Nat%20Med.&doi=10.1038%2Fs41591-019-0649-2&volume=26&issue=1&pages=16-17&publication_year=2020&author=Chen%2CIY&author=Joshi%2CS&author=Ghassemi%2CM"> Google Scholar</a> </p></li><li class="c-article-references__item js-c-reading-companion-references-item" data-counter="99."><p class="c-article-references__text" id="ref-CR99">Bolukbasi T, Chang K-W, Zou J, Saligrama V, Kalai A. Man is to computer programmer as woman is to homemaker? debiasing word embeddings. In: Proceedings of the 30th International Conference on Neural Information Processing Systems. Barcelona: Curran Associates Inc.; 2016. p. 4356–64.</p><p class="c-article-references__links u-hide-print"><a data-track="click_references" data-track-action="google scholar reference" data-track-value="google scholar reference" data-track-label="link" data-track-item_id="link" rel="nofollow noopener" aria-label="Google Scholar reference 99" href="http://scholar.google.com/scholar_lookup?&title=Man%20is%20to%20computer%20programmer%20as%20woman%20is%20to%20homemaker%3F%20debiasing%20word%20embeddings&pages=4356-4364&publication_year=2016&author=Bolukbasi%2CT&author=Chang%2CK-W&author=Zou%2CJ&author=Saligrama%2CV&author=Kalai%2CA"> Google Scholar</a> </p></li><li class="c-article-references__item js-c-reading-companion-references-item" data-counter="100."><p class="c-article-references__text" id="ref-CR100">Kusner, Matt, Loftus, Joshua, Russell, Chris and Silva, Ricardo. Counterfactual fairness Conference. Proceedings of the 31st International Conference on Neural Information Processing Systems Conference. Long Beach, California, USA Publisher: Curran Associates Inc; 2017:4069–4079.</p></li><li class="c-article-references__item js-c-reading-companion-references-item" data-counter="101."><p class="c-article-references__text" id="ref-CR101">Hardt M, Price E, Srebro N. Equality of opportunity in supervised learning. In: Proceedings of the 30th International Conference on Neural Information Processing Systems. Barcelona: Curran Associates Inc.; 2016. p. 3323–31.</p><p class="c-article-references__links u-hide-print"><a data-track="click_references" data-track-action="google scholar reference" data-track-value="google scholar reference" data-track-label="link" data-track-item_id="link" rel="nofollow noopener" aria-label="Google Scholar reference 101" href="http://scholar.google.com/scholar_lookup?&title=Equality%20of%20opportunity%20in%20supervised%20learning&pages=3323-3331&publication_year=2016&author=Hardt%2CM&author=Price%2CE&author=Srebro%2CN"> Google Scholar</a> </p></li><li class="c-article-references__item js-c-reading-companion-references-item" data-counter="102."><p class="c-article-references__text" id="ref-CR102">Ustun B, Liu Y, Parkes D. Fairness without harm: decoupled classifiers with preference guarantees. In: Kamalika C, Ruslan S, editors. Proceedings of the 36th International Conference on Machine Learning; Proceedings of Machine Learning Research: PMLR %J Proceedings of Machine Learning Research; 2019. p. 6373–82.</p><p class="c-article-references__links u-hide-print"><a data-track="click_references" data-track-action="google scholar reference" data-track-value="google scholar reference" data-track-label="link" data-track-item_id="link" rel="nofollow noopener" aria-label="Google Scholar reference 102" href="http://scholar.google.com/scholar_lookup?&title=Fairness%20without%20harm%3A%20decoupled%20classifiers%20with%20preference%20guarantees&pages=6373-6382&publication_year=2019&author=Ustun%2CB&author=Liu%2CY&author=Parkes%2CD"> Google Scholar</a> </p></li><li class="c-article-references__item js-c-reading-companion-references-item" data-counter="103."><p class="c-article-references__text" id="ref-CR103">Noseworthy PA, Attia ZI, Brewer LC, Hayes SN, Yao X, Kapa S, et al. Assessing and Mitigating bias in medical artificial intelligence: the effects of race and ethnicity on a deep learning model for ECG analysis. Circ Arrhythm Electrophysiol. 2020;13(3):e007988. <a href="https://doi.org/10.1161/CIRCEP.119.007988" data-track="click_references" data-track-action="external reference" data-track-value="external reference" data-track-label="10.1161/CIRCEP.119.007988">https://doi.org/10.1161/CIRCEP.119.007988</a>.</p><p class="c-article-references__links u-hide-print"><a data-track="click_references" rel="nofollow noopener" data-track-label="10.1161/CIRCEP.119.007988" data-track-item_id="10.1161/CIRCEP.119.007988" data-track-value="article reference" data-track-action="article reference" href="https://doi.org/10.1161%2FCIRCEP.119.007988" aria-label="Article reference 103" data-doi="10.1161/CIRCEP.119.007988">Article</a> <a data-track="click_references" rel="nofollow noopener" data-track-label="link" data-track-item_id="link" data-track-value="pubmed reference" data-track-action="pubmed reference" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Abstract&list_uids=32064914" aria-label="PubMed reference 103">PubMed</a> <a data-track="click_references" rel="nofollow noopener" data-track-label="link" data-track-item_id="link" data-track-value="pubmed central reference" data-track-action="pubmed central reference" href="http://www.ncbi.nlm.nih.gov/pmc/articles/PMC7158877" aria-label="PubMed Central reference 103">PubMed Central</a> <a data-track="click_references" data-track-action="google scholar reference" data-track-value="google scholar reference" data-track-label="link" data-track-item_id="link" rel="nofollow noopener" aria-label="Google Scholar reference 103" href="http://scholar.google.com/scholar_lookup?&title=Assessing%20and%20Mitigating%20bias%20in%20medical%20artificial%20intelligence%3A%20the%20effects%20of%20race%20and%20ethnicity%20on%20a%20deep%20learning%20model%20for%20ECG%20analysis&journal=Circ%20Arrhythm%20Electrophysiol.&doi=10.1161%2FCIRCEP.119.007988&volume=13&issue=3&publication_year=2020&author=Noseworthy%2CPA&author=Attia%2CZI&author=Brewer%2CLC&author=Hayes%2CSN&author=Yao%2CX&author=Kapa%2CS&author=Friedman%2CPA&author=Lopez-Jimenez%2CF"> Google Scholar</a> </p></li></ol><p class="c-article-references__download u-hide-print"><a data-track="click" data-track-action="download citation references" data-track-label="link" rel="nofollow" href="https://citation-needed.springer.com/v2/references/10.1186/s13063-021-05489-x?format=refman&flavour=references">Download references<svg width="16" height="16" focusable="false" role="img" aria-hidden="true" class="u-icon"><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#icon-eds-i-download-medium"></use></svg></a></p></div></div></div></section></div><section data-title="Acknowledgements"><div class="c-article-section" id="Ack1-section"><h2 class="c-article-section__title js-section-title js-c-reading-companion-sections-item" id="Ack1">Acknowledgements</h2><div class="c-article-section__content" id="Ack1-content"><p>The authors would like to acknowledge the contributions of Peter Hoffmann and Brooke Walker to the editing and preparation of this manuscript.</p></div></div></section><section data-title="Funding"><div class="c-article-section" id="Fun-section"><h2 class="c-article-section__title js-section-title js-c-reading-companion-sections-item" id="Fun">Funding</h2><div class="c-article-section__content" id="Fun-content"><p>Funding support for the meeting was provided through registration fees from Amgen Inc., AstraZeneca, Bayer AG, Boehringer-Ingelheim, Cytokinetics, Eli Lilly & Company, Evidation, IQVIA, Janssen, Microsoft, Pfizer, Sanofi, and Verily. No government funds were used for this meeting.</p></div></div></section><section aria-labelledby="author-information" data-title="Author information"><div class="c-article-section" id="author-information-section"><h2 class="c-article-section__title js-section-title js-c-reading-companion-sections-item" id="author-information">Author information</h2><div class="c-article-section__content" id="author-information-content"><h3 class="c-article__sub-heading" id="affiliations">Authors and Affiliations</h3><ol class="c-article-author-affiliation__list"><li id="Aff1"><p class="c-article-author-affiliation__address">Duke Clinical Research Institute, Duke University School of Medicine, Box 2834, Durham, NC, 27701, USA</p><p class="c-article-author-affiliation__authors-list">E. Hope Weissler, Scott H. Kollins, Lesley Curtis & Erich Huang</p></li><li id="Aff2"><p class="c-article-author-affiliation__address">Microsoft Research, Cambridge, MA, USA</p><p class="c-article-author-affiliation__authors-list">Tristan Naumann</p></li><li id="Aff3"><p class="c-article-author-affiliation__address">AstraZeneca, Gothenburg, Sweden</p><p class="c-article-author-affiliation__authors-list">Tomas Andersson, Faisal Khan, Khader Shameer & Emmette Hutchison</p></li><li id="Aff4"><p class="c-article-author-affiliation__address">Courant Institute of Mathematical Science, New York University, New York, NY, USA</p><p class="c-article-author-affiliation__authors-list">Rajesh Ranganath</p></li><li id="Aff5"><p class="c-article-author-affiliation__address">Englander Institute for Precision Medicine, Weill Cornell Medical College, New York, NY, USA</p><p class="c-article-author-affiliation__authors-list">Olivier Elemento</p></li><li id="Aff6"><p class="c-article-author-affiliation__address">Northwestern University Clinical and Translational Sciences Institute, Northwestern University, Chicago, IL, USA</p><p class="c-article-author-affiliation__authors-list">Yuan Luo</p></li><li id="Aff7"><p class="c-article-author-affiliation__address">Division Pharmaceuticals, Open Innovation and Digital Technologies, Bayer AG, Wuppertal, Germany</p><p class="c-article-author-affiliation__authors-list">Daniel F. Freitag</p></li><li id="Aff8"><p class="c-article-author-affiliation__address">University of Alberta, Edmonton, Alberta, Canada</p><p class="c-article-author-affiliation__authors-list">James Benoit</p></li><li id="Aff9"><p class="c-article-author-affiliation__address">Department of Computer Science, Tufts University, Medford, MA, USA</p><p class="c-article-author-affiliation__authors-list">Michael C. Hughes</p></li><li id="Aff10"><p class="c-article-author-affiliation__address">Billion Minds, Inc., Seattle, WA, USA</p><p class="c-article-author-affiliation__authors-list">Paul Slater</p></li><li id="Aff11"><p class="c-article-author-affiliation__address">Verana Health, San Francisco, CA, USA</p><p class="c-article-author-affiliation__authors-list">Matthew Roe</p></li><li id="Aff12"><p class="c-article-author-affiliation__address">Boehringer-Ingelheim, Burlington, Canada</p><p class="c-article-author-affiliation__authors-list">Uli Broedl</p></li><li id="Aff13"><p class="c-article-author-affiliation__address">Sanofi, Cambridge, MA, USA</p><p class="c-article-author-affiliation__authors-list">Zhaoling Meng</p></li><li id="Aff14"><p class="c-article-author-affiliation__address">Sanofi, Washington, DC, USA</p><p class="c-article-author-affiliation__authors-list">Jennifer L. Wong</p></li><li id="Aff15"><p class="c-article-author-affiliation__address">Duke Forge, Durham, NC, USA</p><p class="c-article-author-affiliation__authors-list">Erich Huang</p></li><li id="Aff16"><p class="c-article-author-affiliation__address">Vector Institute, University of Toronto, Toronto, Ontario, Canada</p><p class="c-article-author-affiliation__authors-list">Marzyeh Ghassemi</p></li><li id="Aff17"><p class="c-article-author-affiliation__address">Department of Electrical Engineering and Computer Science, Massachusetts Institute of Technology, Cambridge, Massachusetts, 02139, USA</p><p class="c-article-author-affiliation__authors-list">Marzyeh Ghassemi</p></li><li id="Aff18"><p class="c-article-author-affiliation__address">Institute for Medical Engineering and Science, Massachusetts Institute of Technology, Cambridge, Massachusetts, 02139, USA</p><p class="c-article-author-affiliation__authors-list">Marzyeh Ghassemi</p></li><li id="Aff19"><p class="c-article-author-affiliation__address">CIFAR AI Chair, Vector Institute, Toronto, Ontario, Canada</p><p class="c-article-author-affiliation__authors-list">Marzyeh Ghassemi</p></li></ol><div class="u-js-hide u-hide-print" data-test="author-info"><span class="c-article__sub-heading">Authors</span><ol class="c-article-authors-search u-list-reset"><li id="auth-E__Hope-Weissler-Aff1"><span class="c-article-authors-search__title u-h3 js-search-name">E. Hope Weissler</span><div class="c-article-authors-search__list"><div class="c-article-authors-search__item c-article-authors-search__list-item--left"><a href="https://www.biomedcentral.com/search?query=author%23E.%20Hope%20Weissler" class="c-article-button" data-track="click" data-track-action="author link - publication" data-track-label="link" rel="nofollow">View author publications</a></div><div class="c-article-authors-search__item c-article-authors-search__list-item--right"><p class="search-in-title-js c-article-authors-search__text">You can also search for this author in <span class="c-article-identifiers"><a class="c-article-identifiers__item" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=search&term=E.%20Hope%20Weissler" data-track="click" data-track-action="author link - pubmed" data-track-label="link" rel="nofollow">PubMed</a><span class="u-hide"> </span><a class="c-article-identifiers__item" href="http://scholar.google.co.uk/scholar?as_q=&num=10&btnG=Search+Scholar&as_epq=&as_oq=&as_eq=&as_occt=any&as_sauthors=%22E.%20Hope%20Weissler%22&as_publication=&as_ylo=&as_yhi=&as_allsubj=all&hl=en" data-track="click" data-track-action="author link - scholar" data-track-label="link" rel="nofollow">Google Scholar</a></span></p></div></div></li><li id="auth-Tristan-Naumann-Aff2"><span class="c-article-authors-search__title u-h3 js-search-name">Tristan Naumann</span><div class="c-article-authors-search__list"><div class="c-article-authors-search__item c-article-authors-search__list-item--left"><a href="https://www.biomedcentral.com/search?query=author%23Tristan%20Naumann" class="c-article-button" data-track="click" data-track-action="author link - publication" data-track-label="link" rel="nofollow">View author publications</a></div><div class="c-article-authors-search__item c-article-authors-search__list-item--right"><p class="search-in-title-js c-article-authors-search__text">You can also search for this author in <span class="c-article-identifiers"><a class="c-article-identifiers__item" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=search&term=Tristan%20Naumann" data-track="click" data-track-action="author link - pubmed" data-track-label="link" rel="nofollow">PubMed</a><span class="u-hide"> </span><a class="c-article-identifiers__item" href="http://scholar.google.co.uk/scholar?as_q=&num=10&btnG=Search+Scholar&as_epq=&as_oq=&as_eq=&as_occt=any&as_sauthors=%22Tristan%20Naumann%22&as_publication=&as_ylo=&as_yhi=&as_allsubj=all&hl=en" data-track="click" data-track-action="author link - scholar" data-track-label="link" rel="nofollow">Google Scholar</a></span></p></div></div></li><li id="auth-Tomas-Andersson-Aff3"><span class="c-article-authors-search__title u-h3 js-search-name">Tomas Andersson</span><div class="c-article-authors-search__list"><div class="c-article-authors-search__item c-article-authors-search__list-item--left"><a href="https://www.biomedcentral.com/search?query=author%23Tomas%20Andersson" class="c-article-button" data-track="click" data-track-action="author link - publication" data-track-label="link" rel="nofollow">View author publications</a></div><div class="c-article-authors-search__item c-article-authors-search__list-item--right"><p class="search-in-title-js c-article-authors-search__text">You can also search for this author in <span class="c-article-identifiers"><a class="c-article-identifiers__item" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=search&term=Tomas%20Andersson" data-track="click" data-track-action="author link - pubmed" data-track-label="link" rel="nofollow">PubMed</a><span class="u-hide"> </span><a class="c-article-identifiers__item" href="http://scholar.google.co.uk/scholar?as_q=&num=10&btnG=Search+Scholar&as_epq=&as_oq=&as_eq=&as_occt=any&as_sauthors=%22Tomas%20Andersson%22&as_publication=&as_ylo=&as_yhi=&as_allsubj=all&hl=en" data-track="click" data-track-action="author link - scholar" data-track-label="link" rel="nofollow">Google Scholar</a></span></p></div></div></li><li id="auth-Rajesh-Ranganath-Aff4"><span class="c-article-authors-search__title u-h3 js-search-name">Rajesh Ranganath</span><div class="c-article-authors-search__list"><div class="c-article-authors-search__item c-article-authors-search__list-item--left"><a href="https://www.biomedcentral.com/search?query=author%23Rajesh%20Ranganath" class="c-article-button" data-track="click" data-track-action="author link - publication" data-track-label="link" rel="nofollow">View author publications</a></div><div class="c-article-authors-search__item c-article-authors-search__list-item--right"><p class="search-in-title-js c-article-authors-search__text">You can also search for this author in <span class="c-article-identifiers"><a class="c-article-identifiers__item" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=search&term=Rajesh%20Ranganath" data-track="click" data-track-action="author link - pubmed" data-track-label="link" rel="nofollow">PubMed</a><span class="u-hide"> </span><a class="c-article-identifiers__item" href="http://scholar.google.co.uk/scholar?as_q=&num=10&btnG=Search+Scholar&as_epq=&as_oq=&as_eq=&as_occt=any&as_sauthors=%22Rajesh%20Ranganath%22&as_publication=&as_ylo=&as_yhi=&as_allsubj=all&hl=en" data-track="click" data-track-action="author link - scholar" data-track-label="link" rel="nofollow">Google Scholar</a></span></p></div></div></li><li id="auth-Olivier-Elemento-Aff5"><span class="c-article-authors-search__title u-h3 js-search-name">Olivier Elemento</span><div class="c-article-authors-search__list"><div class="c-article-authors-search__item c-article-authors-search__list-item--left"><a href="https://www.biomedcentral.com/search?query=author%23Olivier%20Elemento" class="c-article-button" data-track="click" data-track-action="author link - publication" data-track-label="link" rel="nofollow">View author publications</a></div><div class="c-article-authors-search__item c-article-authors-search__list-item--right"><p class="search-in-title-js c-article-authors-search__text">You can also search for this author in <span class="c-article-identifiers"><a class="c-article-identifiers__item" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=search&term=Olivier%20Elemento" data-track="click" data-track-action="author link - pubmed" data-track-label="link" rel="nofollow">PubMed</a><span class="u-hide"> </span><a class="c-article-identifiers__item" href="http://scholar.google.co.uk/scholar?as_q=&num=10&btnG=Search+Scholar&as_epq=&as_oq=&as_eq=&as_occt=any&as_sauthors=%22Olivier%20Elemento%22&as_publication=&as_ylo=&as_yhi=&as_allsubj=all&hl=en" data-track="click" data-track-action="author link - scholar" data-track-label="link" rel="nofollow">Google Scholar</a></span></p></div></div></li><li id="auth-Yuan-Luo-Aff6"><span class="c-article-authors-search__title u-h3 js-search-name">Yuan Luo</span><div class="c-article-authors-search__list"><div class="c-article-authors-search__item c-article-authors-search__list-item--left"><a href="https://www.biomedcentral.com/search?query=author%23Yuan%20Luo" class="c-article-button" data-track="click" data-track-action="author link - publication" data-track-label="link" rel="nofollow">View author publications</a></div><div class="c-article-authors-search__item c-article-authors-search__list-item--right"><p class="search-in-title-js c-article-authors-search__text">You can also search for this author in <span class="c-article-identifiers"><a class="c-article-identifiers__item" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=search&term=Yuan%20Luo" data-track="click" data-track-action="author link - pubmed" data-track-label="link" rel="nofollow">PubMed</a><span class="u-hide"> </span><a class="c-article-identifiers__item" href="http://scholar.google.co.uk/scholar?as_q=&num=10&btnG=Search+Scholar&as_epq=&as_oq=&as_eq=&as_occt=any&as_sauthors=%22Yuan%20Luo%22&as_publication=&as_ylo=&as_yhi=&as_allsubj=all&hl=en" data-track="click" data-track-action="author link - scholar" data-track-label="link" rel="nofollow">Google Scholar</a></span></p></div></div></li><li id="auth-Daniel_F_-Freitag-Aff7"><span class="c-article-authors-search__title u-h3 js-search-name">Daniel F. Freitag</span><div class="c-article-authors-search__list"><div class="c-article-authors-search__item c-article-authors-search__list-item--left"><a href="https://www.biomedcentral.com/search?query=author%23Daniel%20F.%20Freitag" class="c-article-button" data-track="click" data-track-action="author link - publication" data-track-label="link" rel="nofollow">View author publications</a></div><div class="c-article-authors-search__item c-article-authors-search__list-item--right"><p class="search-in-title-js c-article-authors-search__text">You can also search for this author in <span class="c-article-identifiers"><a class="c-article-identifiers__item" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=search&term=Daniel%20F.%20Freitag" data-track="click" data-track-action="author link - pubmed" data-track-label="link" rel="nofollow">PubMed</a><span class="u-hide"> </span><a class="c-article-identifiers__item" href="http://scholar.google.co.uk/scholar?as_q=&num=10&btnG=Search+Scholar&as_epq=&as_oq=&as_eq=&as_occt=any&as_sauthors=%22Daniel%20F.%20Freitag%22&as_publication=&as_ylo=&as_yhi=&as_allsubj=all&hl=en" data-track="click" data-track-action="author link - scholar" data-track-label="link" rel="nofollow">Google Scholar</a></span></p></div></div></li><li id="auth-James-Benoit-Aff8"><span class="c-article-authors-search__title u-h3 js-search-name">James Benoit</span><div class="c-article-authors-search__list"><div class="c-article-authors-search__item c-article-authors-search__list-item--left"><a href="https://www.biomedcentral.com/search?query=author%23James%20Benoit" class="c-article-button" data-track="click" data-track-action="author link - publication" data-track-label="link" rel="nofollow">View author publications</a></div><div class="c-article-authors-search__item c-article-authors-search__list-item--right"><p class="search-in-title-js c-article-authors-search__text">You can also search for this author in <span class="c-article-identifiers"><a class="c-article-identifiers__item" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=search&term=James%20Benoit" data-track="click" data-track-action="author link - pubmed" data-track-label="link" rel="nofollow">PubMed</a><span class="u-hide"> </span><a class="c-article-identifiers__item" href="http://scholar.google.co.uk/scholar?as_q=&num=10&btnG=Search+Scholar&as_epq=&as_oq=&as_eq=&as_occt=any&as_sauthors=%22James%20Benoit%22&as_publication=&as_ylo=&as_yhi=&as_allsubj=all&hl=en" data-track="click" data-track-action="author link - scholar" data-track-label="link" rel="nofollow">Google Scholar</a></span></p></div></div></li><li id="auth-Michael_C_-Hughes-Aff9"><span class="c-article-authors-search__title u-h3 js-search-name">Michael C. Hughes</span><div class="c-article-authors-search__list"><div class="c-article-authors-search__item c-article-authors-search__list-item--left"><a href="https://www.biomedcentral.com/search?query=author%23Michael%20C.%20Hughes" class="c-article-button" data-track="click" data-track-action="author link - publication" data-track-label="link" rel="nofollow">View author publications</a></div><div class="c-article-authors-search__item c-article-authors-search__list-item--right"><p class="search-in-title-js c-article-authors-search__text">You can also search for this author in <span class="c-article-identifiers"><a class="c-article-identifiers__item" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=search&term=Michael%20C.%20Hughes" data-track="click" data-track-action="author link - pubmed" data-track-label="link" rel="nofollow">PubMed</a><span class="u-hide"> </span><a class="c-article-identifiers__item" href="http://scholar.google.co.uk/scholar?as_q=&num=10&btnG=Search+Scholar&as_epq=&as_oq=&as_eq=&as_occt=any&as_sauthors=%22Michael%20C.%20Hughes%22&as_publication=&as_ylo=&as_yhi=&as_allsubj=all&hl=en" data-track="click" data-track-action="author link - scholar" data-track-label="link" rel="nofollow">Google Scholar</a></span></p></div></div></li><li id="auth-Faisal-Khan-Aff3"><span class="c-article-authors-search__title u-h3 js-search-name">Faisal Khan</span><div class="c-article-authors-search__list"><div class="c-article-authors-search__item c-article-authors-search__list-item--left"><a href="https://www.biomedcentral.com/search?query=author%23Faisal%20Khan" class="c-article-button" data-track="click" data-track-action="author link - publication" data-track-label="link" rel="nofollow">View author publications</a></div><div class="c-article-authors-search__item c-article-authors-search__list-item--right"><p class="search-in-title-js c-article-authors-search__text">You can also search for this author in <span class="c-article-identifiers"><a class="c-article-identifiers__item" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=search&term=Faisal%20Khan" data-track="click" data-track-action="author link - pubmed" data-track-label="link" rel="nofollow">PubMed</a><span class="u-hide"> </span><a class="c-article-identifiers__item" href="http://scholar.google.co.uk/scholar?as_q=&num=10&btnG=Search+Scholar&as_epq=&as_oq=&as_eq=&as_occt=any&as_sauthors=%22Faisal%20Khan%22&as_publication=&as_ylo=&as_yhi=&as_allsubj=all&hl=en" data-track="click" data-track-action="author link - scholar" data-track-label="link" rel="nofollow">Google Scholar</a></span></p></div></div></li><li id="auth-Paul-Slater-Aff10"><span class="c-article-authors-search__title u-h3 js-search-name">Paul Slater</span><div class="c-article-authors-search__list"><div class="c-article-authors-search__item c-article-authors-search__list-item--left"><a href="https://www.biomedcentral.com/search?query=author%23Paul%20Slater" class="c-article-button" data-track="click" data-track-action="author link - publication" data-track-label="link" rel="nofollow">View author publications</a></div><div class="c-article-authors-search__item c-article-authors-search__list-item--right"><p class="search-in-title-js c-article-authors-search__text">You can also search for this author in <span class="c-article-identifiers"><a class="c-article-identifiers__item" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=search&term=Paul%20Slater" data-track="click" data-track-action="author link - pubmed" data-track-label="link" rel="nofollow">PubMed</a><span class="u-hide"> </span><a class="c-article-identifiers__item" href="http://scholar.google.co.uk/scholar?as_q=&num=10&btnG=Search+Scholar&as_epq=&as_oq=&as_eq=&as_occt=any&as_sauthors=%22Paul%20Slater%22&as_publication=&as_ylo=&as_yhi=&as_allsubj=all&hl=en" data-track="click" data-track-action="author link - scholar" data-track-label="link" rel="nofollow">Google Scholar</a></span></p></div></div></li><li id="auth-Khader-Shameer-Aff3"><span class="c-article-authors-search__title u-h3 js-search-name">Khader Shameer</span><div class="c-article-authors-search__list"><div class="c-article-authors-search__item c-article-authors-search__list-item--left"><a href="https://www.biomedcentral.com/search?query=author%23Khader%20Shameer" class="c-article-button" data-track="click" data-track-action="author link - publication" data-track-label="link" rel="nofollow">View author publications</a></div><div class="c-article-authors-search__item c-article-authors-search__list-item--right"><p class="search-in-title-js c-article-authors-search__text">You can also search for this author in <span class="c-article-identifiers"><a class="c-article-identifiers__item" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=search&term=Khader%20Shameer" data-track="click" data-track-action="author link - pubmed" data-track-label="link" rel="nofollow">PubMed</a><span class="u-hide"> </span><a class="c-article-identifiers__item" href="http://scholar.google.co.uk/scholar?as_q=&num=10&btnG=Search+Scholar&as_epq=&as_oq=&as_eq=&as_occt=any&as_sauthors=%22Khader%20Shameer%22&as_publication=&as_ylo=&as_yhi=&as_allsubj=all&hl=en" data-track="click" data-track-action="author link - scholar" data-track-label="link" rel="nofollow">Google Scholar</a></span></p></div></div></li><li id="auth-Matthew-Roe-Aff11"><span class="c-article-authors-search__title u-h3 js-search-name">Matthew Roe</span><div class="c-article-authors-search__list"><div class="c-article-authors-search__item c-article-authors-search__list-item--left"><a href="https://www.biomedcentral.com/search?query=author%23Matthew%20Roe" class="c-article-button" data-track="click" data-track-action="author link - publication" data-track-label="link" rel="nofollow">View author publications</a></div><div class="c-article-authors-search__item c-article-authors-search__list-item--right"><p class="search-in-title-js c-article-authors-search__text">You can also search for this author in <span class="c-article-identifiers"><a class="c-article-identifiers__item" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=search&term=Matthew%20Roe" data-track="click" data-track-action="author link - pubmed" data-track-label="link" rel="nofollow">PubMed</a><span class="u-hide"> </span><a class="c-article-identifiers__item" href="http://scholar.google.co.uk/scholar?as_q=&num=10&btnG=Search+Scholar&as_epq=&as_oq=&as_eq=&as_occt=any&as_sauthors=%22Matthew%20Roe%22&as_publication=&as_ylo=&as_yhi=&as_allsubj=all&hl=en" data-track="click" data-track-action="author link - scholar" data-track-label="link" rel="nofollow">Google Scholar</a></span></p></div></div></li><li id="auth-Emmette-Hutchison-Aff3"><span class="c-article-authors-search__title u-h3 js-search-name">Emmette Hutchison</span><div class="c-article-authors-search__list"><div class="c-article-authors-search__item c-article-authors-search__list-item--left"><a href="https://www.biomedcentral.com/search?query=author%23Emmette%20Hutchison" class="c-article-button" data-track="click" data-track-action="author link - publication" data-track-label="link" rel="nofollow">View author publications</a></div><div class="c-article-authors-search__item c-article-authors-search__list-item--right"><p class="search-in-title-js c-article-authors-search__text">You can also search for this author in <span class="c-article-identifiers"><a class="c-article-identifiers__item" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=search&term=Emmette%20Hutchison" data-track="click" data-track-action="author link - pubmed" data-track-label="link" rel="nofollow">PubMed</a><span class="u-hide"> </span><a class="c-article-identifiers__item" href="http://scholar.google.co.uk/scholar?as_q=&num=10&btnG=Search+Scholar&as_epq=&as_oq=&as_eq=&as_occt=any&as_sauthors=%22Emmette%20Hutchison%22&as_publication=&as_ylo=&as_yhi=&as_allsubj=all&hl=en" data-track="click" data-track-action="author link - scholar" data-track-label="link" rel="nofollow">Google Scholar</a></span></p></div></div></li><li id="auth-Scott_H_-Kollins-Aff1"><span class="c-article-authors-search__title u-h3 js-search-name">Scott H. Kollins</span><div class="c-article-authors-search__list"><div class="c-article-authors-search__item c-article-authors-search__list-item--left"><a href="https://www.biomedcentral.com/search?query=author%23Scott%20H.%20Kollins" class="c-article-button" data-track="click" data-track-action="author link - publication" data-track-label="link" rel="nofollow">View author publications</a></div><div class="c-article-authors-search__item c-article-authors-search__list-item--right"><p class="search-in-title-js c-article-authors-search__text">You can also search for this author in <span class="c-article-identifiers"><a class="c-article-identifiers__item" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=search&term=Scott%20H.%20Kollins" data-track="click" data-track-action="author link - pubmed" data-track-label="link" rel="nofollow">PubMed</a><span class="u-hide"> </span><a class="c-article-identifiers__item" href="http://scholar.google.co.uk/scholar?as_q=&num=10&btnG=Search+Scholar&as_epq=&as_oq=&as_eq=&as_occt=any&as_sauthors=%22Scott%20H.%20Kollins%22&as_publication=&as_ylo=&as_yhi=&as_allsubj=all&hl=en" data-track="click" data-track-action="author link - scholar" data-track-label="link" rel="nofollow">Google Scholar</a></span></p></div></div></li><li id="auth-Uli-Broedl-Aff12"><span class="c-article-authors-search__title u-h3 js-search-name">Uli Broedl</span><div class="c-article-authors-search__list"><div class="c-article-authors-search__item c-article-authors-search__list-item--left"><a href="https://www.biomedcentral.com/search?query=author%23Uli%20Broedl" class="c-article-button" data-track="click" data-track-action="author link - publication" data-track-label="link" rel="nofollow">View author publications</a></div><div class="c-article-authors-search__item c-article-authors-search__list-item--right"><p class="search-in-title-js c-article-authors-search__text">You can also search for this author in <span class="c-article-identifiers"><a class="c-article-identifiers__item" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=search&term=Uli%20Broedl" data-track="click" data-track-action="author link - pubmed" data-track-label="link" rel="nofollow">PubMed</a><span class="u-hide"> </span><a class="c-article-identifiers__item" href="http://scholar.google.co.uk/scholar?as_q=&num=10&btnG=Search+Scholar&as_epq=&as_oq=&as_eq=&as_occt=any&as_sauthors=%22Uli%20Broedl%22&as_publication=&as_ylo=&as_yhi=&as_allsubj=all&hl=en" data-track="click" data-track-action="author link - scholar" data-track-label="link" rel="nofollow">Google Scholar</a></span></p></div></div></li><li id="auth-Zhaoling-Meng-Aff13"><span class="c-article-authors-search__title u-h3 js-search-name">Zhaoling Meng</span><div class="c-article-authors-search__list"><div class="c-article-authors-search__item c-article-authors-search__list-item--left"><a href="https://www.biomedcentral.com/search?query=author%23Zhaoling%20Meng" class="c-article-button" data-track="click" data-track-action="author link - publication" data-track-label="link" rel="nofollow">View author publications</a></div><div class="c-article-authors-search__item c-article-authors-search__list-item--right"><p class="search-in-title-js c-article-authors-search__text">You can also search for this author in <span class="c-article-identifiers"><a class="c-article-identifiers__item" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=search&term=Zhaoling%20Meng" data-track="click" data-track-action="author link - pubmed" data-track-label="link" rel="nofollow">PubMed</a><span class="u-hide"> </span><a class="c-article-identifiers__item" href="http://scholar.google.co.uk/scholar?as_q=&num=10&btnG=Search+Scholar&as_epq=&as_oq=&as_eq=&as_occt=any&as_sauthors=%22Zhaoling%20Meng%22&as_publication=&as_ylo=&as_yhi=&as_allsubj=all&hl=en" data-track="click" data-track-action="author link - scholar" data-track-label="link" rel="nofollow">Google Scholar</a></span></p></div></div></li><li id="auth-Jennifer_L_-Wong-Aff14"><span class="c-article-authors-search__title u-h3 js-search-name">Jennifer L. Wong</span><div class="c-article-authors-search__list"><div class="c-article-authors-search__item c-article-authors-search__list-item--left"><a href="https://www.biomedcentral.com/search?query=author%23Jennifer%20L.%20Wong" class="c-article-button" data-track="click" data-track-action="author link - publication" data-track-label="link" rel="nofollow">View author publications</a></div><div class="c-article-authors-search__item c-article-authors-search__list-item--right"><p class="search-in-title-js c-article-authors-search__text">You can also search for this author in <span class="c-article-identifiers"><a class="c-article-identifiers__item" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=search&term=Jennifer%20L.%20Wong" data-track="click" data-track-action="author link - pubmed" data-track-label="link" rel="nofollow">PubMed</a><span class="u-hide"> </span><a class="c-article-identifiers__item" href="http://scholar.google.co.uk/scholar?as_q=&num=10&btnG=Search+Scholar&as_epq=&as_oq=&as_eq=&as_occt=any&as_sauthors=%22Jennifer%20L.%20Wong%22&as_publication=&as_ylo=&as_yhi=&as_allsubj=all&hl=en" data-track="click" data-track-action="author link - scholar" data-track-label="link" rel="nofollow">Google Scholar</a></span></p></div></div></li><li id="auth-Lesley-Curtis-Aff1"><span class="c-article-authors-search__title u-h3 js-search-name">Lesley Curtis</span><div class="c-article-authors-search__list"><div class="c-article-authors-search__item c-article-authors-search__list-item--left"><a href="https://www.biomedcentral.com/search?query=author%23Lesley%20Curtis" class="c-article-button" data-track="click" data-track-action="author link - publication" data-track-label="link" rel="nofollow">View author publications</a></div><div class="c-article-authors-search__item c-article-authors-search__list-item--right"><p class="search-in-title-js c-article-authors-search__text">You can also search for this author in <span class="c-article-identifiers"><a class="c-article-identifiers__item" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=search&term=Lesley%20Curtis" data-track="click" data-track-action="author link - pubmed" data-track-label="link" rel="nofollow">PubMed</a><span class="u-hide"> </span><a class="c-article-identifiers__item" href="http://scholar.google.co.uk/scholar?as_q=&num=10&btnG=Search+Scholar&as_epq=&as_oq=&as_eq=&as_occt=any&as_sauthors=%22Lesley%20Curtis%22&as_publication=&as_ylo=&as_yhi=&as_allsubj=all&hl=en" data-track="click" data-track-action="author link - scholar" data-track-label="link" rel="nofollow">Google Scholar</a></span></p></div></div></li><li id="auth-Erich-Huang-Aff1-Aff15"><span class="c-article-authors-search__title u-h3 js-search-name">Erich Huang</span><div class="c-article-authors-search__list"><div class="c-article-authors-search__item c-article-authors-search__list-item--left"><a href="https://www.biomedcentral.com/search?query=author%23Erich%20Huang" class="c-article-button" data-track="click" data-track-action="author link - publication" data-track-label="link" rel="nofollow">View author publications</a></div><div class="c-article-authors-search__item c-article-authors-search__list-item--right"><p class="search-in-title-js c-article-authors-search__text">You can also search for this author in <span class="c-article-identifiers"><a class="c-article-identifiers__item" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=search&term=Erich%20Huang" data-track="click" data-track-action="author link - pubmed" data-track-label="link" rel="nofollow">PubMed</a><span class="u-hide"> </span><a class="c-article-identifiers__item" href="http://scholar.google.co.uk/scholar?as_q=&num=10&btnG=Search+Scholar&as_epq=&as_oq=&as_eq=&as_occt=any&as_sauthors=%22Erich%20Huang%22&as_publication=&as_ylo=&as_yhi=&as_allsubj=all&hl=en" data-track="click" data-track-action="author link - scholar" data-track-label="link" rel="nofollow">Google Scholar</a></span></p></div></div></li><li id="auth-Marzyeh-Ghassemi-Aff16-Aff17-Aff18-Aff19"><span class="c-article-authors-search__title u-h3 js-search-name">Marzyeh Ghassemi</span><div class="c-article-authors-search__list"><div class="c-article-authors-search__item c-article-authors-search__list-item--left"><a href="https://www.biomedcentral.com/search?query=author%23Marzyeh%20Ghassemi" class="c-article-button" data-track="click" data-track-action="author link - publication" data-track-label="link" rel="nofollow">View author publications</a></div><div class="c-article-authors-search__item c-article-authors-search__list-item--right"><p class="search-in-title-js c-article-authors-search__text">You can also search for this author in <span class="c-article-identifiers"><a class="c-article-identifiers__item" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=search&term=Marzyeh%20Ghassemi" data-track="click" data-track-action="author link - pubmed" data-track-label="link" rel="nofollow">PubMed</a><span class="u-hide"> </span><a class="c-article-identifiers__item" href="http://scholar.google.co.uk/scholar?as_q=&num=10&btnG=Search+Scholar&as_epq=&as_oq=&as_eq=&as_occt=any&as_sauthors=%22Marzyeh%20Ghassemi%22&as_publication=&as_ylo=&as_yhi=&as_allsubj=all&hl=en" data-track="click" data-track-action="author link - scholar" data-track-label="link" rel="nofollow">Google Scholar</a></span></p></div></div></li></ol></div><h3 class="c-article__sub-heading" id="contributions">Contributions</h3><p>All authors contributed to the conception and design of the work and the analysis and interpretation of the data consisting of reports (peer-reviewed and otherwise) concerning the development, performance, and use of ML in clinical research. EHW drafted the work. All authors substantively revised the work. The author(s) read and approved the final manuscript.</p><h3 class="c-article__sub-heading" id="corresponding-author">Corresponding author</h3><p id="corresponding-author-list">Correspondence to <a id="corresp-c1" href="mailto:Hope.weissler@duke.edu">E. Hope Weissler</a>.</p></div></div></section><section data-title="Ethics declarations"><div class="c-article-section" id="ethics-section"><h2 class="c-article-section__title js-section-title js-c-reading-companion-sections-item" id="ethics">Ethics declarations</h2><div class="c-article-section__content" id="ethics-content"> <h3 class="c-article__sub-heading" id="FPar1">Ethics approval and consent to participate</h3> <p>Not applicable.</p> <h3 class="c-article__sub-heading" id="FPar2">Consent for publication</h3> <p>Not applicable.</p> <h3 class="c-article__sub-heading" id="FPar3">Competing interests</h3> <p>HW has nothing to disclose.</p> <p>TN has nothing to disclose.</p> <p>TA is an employee of AstraZeneca.</p> <p>RR has nothing to disclose.</p> <p>OE is a co-founder of and holds equity in OneThree Biotech and Volastra Therapeutics and is scientific advisor for and holds equity in Freenome and Owkin,</p> <p>YL has nothing to disclose.</p> <p>DF is an employee of Bayer AG, Germany.</p> <p>JB has nothing to disclose.</p> <p>MH reports personal fees from Duke Clinical Research Institute, non-financial support from RGI Informatics, LLC, and grants from Oracle Labs.</p> <p>FK is an employee of AstraZeneca.</p> <p>PS has nothing to disclose.</p> <p>SK is an employee of AstraZeneca; has served as an advisor for Kencor Health and OccamzRazor; has received consulting fees from Google Cloud (Alphabet), McKinsey, and LEK Consulting; was an employee of Philips Healthcare; and has a patent (Diagnosis and Classification of Left Ventricular Diastolic Dysfunction Using a Computer) issued to MSIP.</p> <p>Dr. Roe reports grants from the American College of Cardiology, American Heart Association, Bayer Pharmaceuticals, Familial Hypercholesterolemia Foundation, Ferring Pharmaceuticals, Myokardia, and Patient Centered Outcomes Research Institute; grants and personal fees from Amgen, AstraZeneca, and Sanofi Aventis; personal fees from Janssen Pharmaceuticals, Elsevier Publishers, Regeneron, Roche-Genetech, Eli Lilly, Novo Nordisk, Pfizer, and Signal Path; and is an employee of Verana Health.</p> <p>EH is an employee of AstraZeneca.</p> <p>SK reports personal fees from Holmusk.</p> <p>UB is an employee of Boehringer-Ingelheim.</p> <p>ZM has nothing to disclose.</p> <p>JW reports being an employee of Sanofi US.</p> <p>LC has nothing to disclose.</p> <p>EH reports personal fees from Valo Health and is a founder of (with equity in) kelaHealth and Clinetic.</p> <p>MG has nothing to disclose.</p> </div></div></section><section data-title="Additional information"><div class="c-article-section" id="additional-information-section"><h2 class="c-article-section__title js-section-title js-c-reading-companion-sections-item" id="additional-information">Additional information</h2><div class="c-article-section__content" id="additional-information-content"><h3 class="c-article__sub-heading">Publisher’s Note</h3><p>Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.</p><p>The original online version of this article was revised:Following the publication of the original article, we were notified that current affiliations 17, 18 and 19 were erroneously added to the first author rather than the senior author (Marzyeh Ghassemi).</p></div></div></section><section data-title="Rights and permissions"><div class="c-article-section" id="rightslink-section"><h2 class="c-article-section__title js-section-title js-c-reading-companion-sections-item" id="rightslink">Rights and permissions</h2><div class="c-article-section__content" id="rightslink-content"> <p><b>Open Access</b> This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit <a href="http://creativecommons.org/licenses/by/4.0/" rel="license">http://creativecommons.org/licenses/by/4.0/</a>. The Creative Commons Public Domain Dedication waiver (<a href="http://creativecommons.org/publicdomain/zero/1.0/" rel="license">http://creativecommons.org/publicdomain/zero/1.0/</a>) applies to the data made available in this article, unless otherwise stated in a credit line to the data.</p> <p class="c-article-rights"><a data-track="click" data-track-action="view rights and permissions" data-track-label="link" href="https://s100.copyright.com/AppDispatchServlet?title=The%20role%20of%20machine%20learning%20in%20clinical%20research%3A%20transforming%20the%20future%20of%20evidence%20generation&author=E.%20Hope%20Weissler%20et%20al&contentID=10.1186%2Fs13063-021-05489-x&copyright=The%20Author%28s%29&publication=1745-6215&publicationDate=2021-08-16&publisherName=SpringerNature&orderBeanReset=true&oa=CC%20BY%20%2B%20CC0">Reprints and permissions</a></p></div></div></section><section aria-labelledby="article-info" data-title="About this article"><div class="c-article-section" id="article-info-section"><h2 class="c-article-section__title js-section-title js-c-reading-companion-sections-item" id="article-info">About this article</h2><div class="c-article-section__content" id="article-info-content"><div class="c-bibliographic-information"><div class="u-hide-print c-bibliographic-information__column c-bibliographic-information__column--border"><a data-crossmark="10.1186/s13063-021-05489-x" target="_blank" rel="noopener" href="https://crossmark.crossref.org/dialog/?doi=10.1186/s13063-021-05489-x" data-track="click" data-track-action="Click Crossmark" data-track-label="link" data-test="crossmark"><img loading="lazy" width="57" height="81" alt="Check for updates. Verify currency and authenticity via CrossMark" src="data:image/svg+xml;base64,PHN2ZyBoZWlnaHQ9IjgxIiB3aWR0aD0iNTciIHhtbG5zPSJodHRwOi8vd3d3LnczLm9yZy8yMDAwL3N2ZyI+PGcgZmlsbD0ibm9uZSIgZmlsbC1ydWxlPSJldmVub2RkIj48cGF0aCBkPSJtMTcuMzUgMzUuNDUgMjEuMy0xNC4ydi0xNy4wM2gtMjEuMyIgZmlsbD0iIzk4OTg5OCIvPjxwYXRoIGQ9Im0zOC42NSAzNS40NS0yMS4zLTE0LjJ2LTE3LjAzaDIxLjMiIGZpbGw9IiM3NDc0NzQiLz48cGF0aCBkPSJtMjggLjVjLTEyLjk4IDAtMjMuNSAxMC41Mi0yMy41IDIzLjVzMTAuNTIgMjMuNSAyMy41IDIzLjUgMjMuNS0xMC41MiAyMy41LTIzLjVjMC02LjIzLTIuNDgtMTIuMjEtNi44OC0xNi42Mi00LjQxLTQuNC0xMC4zOS02Ljg4LTE2LjYyLTYuODh6bTAgNDEuMjVjLTkuOCAwLTE3Ljc1LTcuOTUtMTcuNzUtMTcuNzVzNy45NS0xNy43NSAxNy43NS0xNy43NSAxNy43NSA3Ljk1IDE3Ljc1IDE3Ljc1YzAgNC43MS0xLjg3IDkuMjItNS4yIDEyLjU1cy03Ljg0IDUuMi0xMi41NSA1LjJ6IiBmaWxsPSIjNTM1MzUzIi8+PHBhdGggZD0ibTQxIDM2Yy01LjgxIDYuMjMtMTUuMjMgNy40NS0yMi40MyAyLjktNy4yMS00LjU1LTEwLjE2LTEzLjU3LTcuMDMtMjEuNWwtNC45Mi0zLjExYy00Ljk1IDEwLjctMS4xOSAyMy40MiA4Ljc4IDI5LjcxIDkuOTcgNi4zIDIzLjA3IDQuMjIgMzAuNi00Ljg2eiIgZmlsbD0iIzljOWM5YyIvPjxwYXRoIGQ9Im0uMiA1OC40NWMwLS43NS4xMS0xLjQyLjMzLTIuMDFzLjUyLTEuMDkuOTEtMS41Yy4zOC0uNDEuODMtLjczIDEuMzQtLjk0LjUxLS4yMiAxLjA2LS4zMiAxLjY1LS4zMi41NiAwIDEuMDYuMTEgMS41MS4zNS40NC4yMy44MS41IDEuMS44MWwtLjkxIDEuMDFjLS4yNC0uMjQtLjQ5LS40Mi0uNzUtLjU2LS4yNy0uMTMtLjU4LS4yLS45My0uMi0uMzkgMC0uNzMuMDgtMS4wNS4yMy0uMzEuMTYtLjU4LjM3LS44MS42Ni0uMjMuMjgtLjQxLjYzLS41MyAxLjA0LS4xMy40MS0uMTkuODgtLjE5IDEuMzkgMCAxLjA0LjIzIDEuODYuNjggMi40Ni40NS41OSAxLjA2Ljg4IDEuODQuODguNDEgMCAuNzctLjA3IDEuMDctLjIzcy41OS0uMzkuODUtLjY4bC45MSAxYy0uMzguNDMtLjguNzYtMS4yOC45OS0uNDcuMjItMSAuMzQtMS41OC4zNC0uNTkgMC0xLjEzLS4xLTEuNjQtLjMxLS41LS4yLS45NC0uNTEtMS4zMS0uOTEtLjM4LS40LS42Ny0uOS0uODgtMS40OC0uMjItLjU5LS4zMy0xLjI2LS4zMy0yLjAyem04LjQtNS4zM2gxLjYxdjIuNTRsLS4wNSAxLjMzYy4yOS0uMjcuNjEtLjUxLjk2LS43MnMuNzYtLjMxIDEuMjQtLjMxYy43MyAwIDEuMjcuMjMgMS42MS43MS4zMy40Ny41IDEuMTQuNSAyLjAydjQuMzFoLTEuNjF2LTQuMWMwLS41Ny0uMDgtLjk3LS4yNS0xLjIxLS4xNy0uMjMtLjQ1LS4zNS0uODMtLjM1LS4zIDAtLjU2LjA4LS43OS4yMi0uMjMuMTUtLjQ5LjM2LS43OC42NHY0LjhoLTEuNjF6bTcuMzcgNi40NWMwLS41Ni4wOS0xLjA2LjI2LTEuNTEuMTgtLjQ1LjQyLS44My43MS0xLjE0LjI5LS4zLjYzLS41NCAxLjAxLS43MS4zOS0uMTcuNzgtLjI1IDEuMTgtLjI1LjQ3IDAgLjg4LjA4IDEuMjMuMjQuMzYuMTYuNjUuMzguODkuNjdzLjQyLjYzLjU0IDEuMDNjLjEyLjQxLjE4Ljg0LjE4IDEuMzIgMCAuMzItLjAyLjU3LS4wNy43NmgtNC4zNmMuMDcuNjIuMjkgMS4xLjY1IDEuNDQuMzYuMzMuODIuNSAxLjM4LjUuMjkgMCAuNTctLjA0LjgzLS4xM3MuNTEtLjIxLjc2LS4zN2wuNTUgMS4wMWMtLjMzLjIxLS42OS4zOS0xLjA5LjUzLS40MS4xNC0uODMuMjEtMS4yNi4yMS0uNDggMC0uOTItLjA4LTEuMzQtLjI1LS40MS0uMTYtLjc2LS40LTEuMDctLjctLjMxLS4zMS0uNTUtLjY5LS43Mi0xLjEzLS4xOC0uNDQtLjI2LS45NS0uMjYtMS41MnptNC42LS42MmMwLS41NS0uMTEtLjk4LS4zNC0xLjI4LS4yMy0uMzEtLjU4LS40Ny0xLjA2LS40Ny0uNDEgMC0uNzcuMTUtMS4wNy40NS0uMzEuMjktLjUuNzMtLjU4IDEuM3ptMi41LjYyYzAtLjU3LjA5LTEuMDguMjgtMS41My4xOC0uNDQuNDMtLjgyLjc1LTEuMTNzLjY5LS41NCAxLjEtLjcxYy40Mi0uMTYuODUtLjI0IDEuMzEtLjI0LjQ1IDAgLjg0LjA4IDEuMTcuMjNzLjYxLjM0Ljg1LjU3bC0uNzcgMS4wMmMtLjE5LS4xNi0uMzgtLjI4LS41Ni0uMzctLjE5LS4wOS0uMzktLjE0LS42MS0uMTQtLjU2IDAtMS4wMS4yMS0xLjM1LjYzLS4zNS40MS0uNTIuOTctLjUyIDEuNjcgMCAuNjkuMTcgMS4yNC41MSAxLjY2LjM0LjQxLjc4LjYyIDEuMzIuNjIuMjggMCAuNTQtLjA2Ljc4LS4xNy4yNC0uMTIuNDUtLjI2LjY0LS40MmwuNjcgMS4wM2MtLjMzLjI5LS42OS41MS0xLjA4LjY1LS4zOS4xNS0uNzguMjMtMS4xOC4yMy0uNDYgMC0uOS0uMDgtMS4zMS0uMjQtLjQtLjE2LS43NS0uMzktMS4wNS0uN3MtLjUzLS42OS0uNy0xLjEzYy0uMTctLjQ1LS4yNS0uOTYtLjI1LTEuNTN6bTYuOTEtNi40NWgxLjU4djYuMTdoLjA1bDIuNTQtMy4xNmgxLjc3bC0yLjM1IDIuOCAyLjU5IDQuMDdoLTEuNzVsLTEuNzctMi45OC0xLjA4IDEuMjN2MS43NWgtMS41OHptMTMuNjkgMS4yN2MtLjI1LS4xMS0uNS0uMTctLjc1LS4xNy0uNTggMC0uODcuMzktLjg3IDEuMTZ2Ljc1aDEuMzR2MS4yN2gtMS4zNHY1LjZoLTEuNjF2LTUuNmgtLjkydi0xLjJsLjkyLS4wN3YtLjcyYzAtLjM1LjA0LS42OC4xMy0uOTguMDgtLjMxLjIxLS41Ny40LS43OXMuNDItLjM5LjcxLS41MWMuMjgtLjEyLjYzLS4xOCAxLjA0LS4xOC4yNCAwIC40OC4wMi42OS4wNy4yMi4wNS40MS4xLjU3LjE3em0uNDggNS4xOGMwLS41Ny4wOS0xLjA4LjI3LTEuNTMuMTctLjQ0LjQxLS44Mi43Mi0xLjEzLjMtLjMxLjY1LS41NCAxLjA0LS43MS4zOS0uMTYuOC0uMjQgMS4yMy0uMjRzLjg0LjA4IDEuMjQuMjRjLjQuMTcuNzQuNCAxLjA0Ljcxcy41NC42OS43MiAxLjEzYy4xOS40NS4yOC45Ni4yOCAxLjUzcy0uMDkgMS4wOC0uMjggMS41M2MtLjE4LjQ0LS40Mi44Mi0uNzIgMS4xM3MtLjY0LjU0LTEuMDQuNy0uODEuMjQtMS4yNC4yNC0uODQtLjA4LTEuMjMtLjI0LS43NC0uMzktMS4wNC0uN2MtLjMxLS4zMS0uNTUtLjY5LS43Mi0xLjEzLS4xOC0uNDUtLjI3LS45Ni0uMjctMS41M3ptMS42NSAwYzAgLjY5LjE0IDEuMjQuNDMgMS42Ni4yOC40MS42OC42MiAxLjE4LjYyLjUxIDAgLjktLjIxIDEuMTktLjYyLjI5LS40Mi40NC0uOTcuNDQtMS42NiAwLS43LS4xNS0xLjI2LS40NC0xLjY3LS4yOS0uNDItLjY4LS42My0xLjE5LS42My0uNSAwLS45LjIxLTEuMTguNjMtLjI5LjQxLS40My45Ny0uNDMgMS42N3ptNi40OC0zLjQ0aDEuMzNsLjEyIDEuMjFoLjA1Yy4yNC0uNDQuNTQtLjc5Ljg4LTEuMDIuMzUtLjI0LjctLjM2IDEuMDctLjM2LjMyIDAgLjU5LjA1Ljc4LjE0bC0uMjggMS40LS4zMy0uMDljLS4xMS0uMDEtLjIzLS4wMi0uMzgtLjAyLS4yNyAwLS41Ni4xLS44Ni4zMXMtLjU1LjU4LS43NyAxLjF2NC4yaC0xLjYxem0tNDcuODcgMTVoMS42MXY0LjFjMCAuNTcuMDguOTcuMjUgMS4yLjE3LjI0LjQ0LjM1LjgxLjM1LjMgMCAuNTctLjA3LjgtLjIyLjIyLS4xNS40Ny0uMzkuNzMtLjczdi00LjdoMS42MXY2Ljg3aC0xLjMybC0uMTItMS4wMWgtLjA0Yy0uMy4zNi0uNjMuNjQtLjk4Ljg2LS4zNS4yMS0uNzYuMzItMS4yNC4zMi0uNzMgMC0xLjI3LS4yNC0xLjYxLS43MS0uMzMtLjQ3LS41LTEuMTQtLjUtMi4wMnptOS40NiA3LjQzdjIuMTZoLTEuNjF2LTkuNTloMS4zM2wuMTIuNzJoLjA1Yy4yOS0uMjQuNjEtLjQ1Ljk3LS42My4zNS0uMTcuNzItLjI2IDEuMS0uMjYuNDMgMCAuODEuMDggMS4xNS4yNC4zMy4xNy42MS40Ljg0LjcxLjI0LjMxLjQxLjY4LjUzIDEuMTEuMTMuNDIuMTkuOTEuMTkgMS40NCAwIC41OS0uMDkgMS4xMS0uMjUgMS41Ny0uMTYuNDctLjM4Ljg1LS42NSAxLjE2LS4yNy4zMi0uNTguNTYtLjk0LjczLS4zNS4xNi0uNzIuMjUtMS4xLjI1LS4zIDAtLjYtLjA3LS45LS4ycy0uNTktLjMxLS44Ny0uNTZ6bTAtMi4zYy4yNi4yMi41LjM3LjczLjQ1LjI0LjA5LjQ2LjEzLjY2LjEzLjQ2IDAgLjg0LS4yIDEuMTUtLjYuMzEtLjM5LjQ2LS45OC40Ni0xLjc3IDAtLjY5LS4xMi0xLjIyLS4zNS0xLjYxLS4yMy0uMzgtLjYxLS41Ny0xLjEzLS41Ny0uNDkgMC0uOTkuMjYtMS41Mi43N3ptNS44Ny0xLjY5YzAtLjU2LjA4LTEuMDYuMjUtMS41MS4xNi0uNDUuMzctLjgzLjY1LTEuMTQuMjctLjMuNTgtLjU0LjkzLS43MXMuNzEtLjI1IDEuMDgtLjI1Yy4zOSAwIC43My4wNyAxIC4yLjI3LjE0LjU0LjMyLjgxLjU1bC0uMDYtMS4xdi0yLjQ5aDEuNjF2OS44OGgtMS4zM2wtLjExLS43NGgtLjA2Yy0uMjUuMjUtLjU0LjQ2LS44OC42NC0uMzMuMTgtLjY5LjI3LTEuMDYuMjctLjg3IDAtMS41Ni0uMzItMi4wNy0uOTVzLS43Ni0xLjUxLS43Ni0yLjY1em0xLjY3LS4wMWMwIC43NC4xMyAxLjMxLjQgMS43LjI2LjM4LjY1LjU4IDEuMTUuNTguNTEgMCAuOTktLjI2IDEuNDQtLjc3di0zLjIxYy0uMjQtLjIxLS40OC0uMzYtLjctLjQ1LS4yMy0uMDgtLjQ2LS4xMi0uNy0uMTItLjQ1IDAtLjgyLjE5LTEuMTMuNTktLjMxLjM5LS40Ni45NS0uNDYgMS42OHptNi4zNSAxLjU5YzAtLjczLjMyLTEuMy45Ny0xLjcxLjY0LS40IDEuNjctLjY4IDMuMDgtLjg0IDAtLjE3LS4wMi0uMzQtLjA3LS41MS0uMDUtLjE2LS4xMi0uMy0uMjItLjQzcy0uMjItLjIyLS4zOC0uM2MtLjE1LS4wNi0uMzQtLjEtLjU4LS4xLS4zNCAwLS42OC4wNy0xIC4ycy0uNjMuMjktLjkzLjQ3bC0uNTktMS4wOGMuMzktLjI0LjgxLS40NSAxLjI4LS42My40Ny0uMTcuOTktLjI2IDEuNTQtLjI2Ljg2IDAgMS41MS4yNSAxLjkzLjc2cy42MyAxLjI1LjYzIDIuMjF2NC4wN2gtMS4zMmwtLjEyLS43NmgtLjA1Yy0uMy4yNy0uNjMuNDgtLjk4LjY2cy0uNzMuMjctMS4xNC4yN2MtLjYxIDAtMS4xLS4xOS0xLjQ4LS41Ni0uMzgtLjM2LS41Ny0uODUtLjU3LTEuNDZ6bTEuNTctLjEyYzAgLjMuMDkuNTMuMjcuNjcuMTkuMTQuNDIuMjEuNzEuMjEuMjggMCAuNTQtLjA3Ljc3LS4ycy40OC0uMzEuNzMtLjU2di0xLjU0Yy0uNDcuMDYtLjg2LjEzLTEuMTguMjMtLjMxLjA5LS41Ny4xOS0uNzYuMzFzLS4zMy4yNS0uNDEuNGMtLjA5LjE1LS4xMy4zMS0uMTMuNDh6bTYuMjktMy42M2gtLjk4di0xLjJsMS4wNi0uMDcuMi0xLjg4aDEuMzR2MS44OGgxLjc1djEuMjdoLTEuNzV2My4yOGMwIC44LjMyIDEuMi45NyAxLjIuMTIgMCAuMjQtLjAxLjM3LS4wNC4xMi0uMDMuMjQtLjA3LjM0LS4xMWwuMjggMS4xOWMtLjE5LjA2LS40LjEyLS42NC4xNy0uMjMuMDUtLjQ5LjA4LS43Ni4wOC0uNCAwLS43NC0uMDYtMS4wMi0uMTgtLjI3LS4xMy0uNDktLjMtLjY3LS41Mi0uMTctLjIxLS4zLS40OC0uMzctLjc4LS4wOC0uMy0uMTItLjY0LS4xMi0xLjAxem00LjM2IDIuMTdjMC0uNTYuMDktMS4wNi4yNy0xLjUxcy40MS0uODMuNzEtMS4xNGMuMjktLjMuNjMtLjU0IDEuMDEtLjcxLjM5LS4xNy43OC0uMjUgMS4xOC0uMjUuNDcgMCAuODguMDggMS4yMy4yNC4zNi4xNi42NS4zOC44OS42N3MuNDIuNjMuNTQgMS4wM2MuMTIuNDEuMTguODQuMTggMS4zMiAwIC4zMi0uMDIuNTctLjA3Ljc2aC00LjM3Yy4wOC42Mi4yOSAxLjEuNjUgMS40NC4zNi4zMy44Mi41IDEuMzguNS4zIDAgLjU4LS4wNC44NC0uMTMuMjUtLjA5LjUxLS4yMS43Ni0uMzdsLjU0IDEuMDFjLS4zMi4yMS0uNjkuMzktMS4wOS41M3MtLjgyLjIxLTEuMjYuMjFjLS40NyAwLS45Mi0uMDgtMS4zMy0uMjUtLjQxLS4xNi0uNzctLjQtMS4wOC0uNy0uMy0uMzEtLjU0LS42OS0uNzItMS4xMy0uMTctLjQ0LS4yNi0uOTUtLjI2LTEuNTJ6bTQuNjEtLjYyYzAtLjU1LS4xMS0uOTgtLjM0LTEuMjgtLjIzLS4zMS0uNTgtLjQ3LTEuMDYtLjQ3LS40MSAwLS43Ny4xNS0xLjA4LjQ1LS4zMS4yOS0uNS43My0uNTcgMS4zem0zLjAxIDIuMjNjLjMxLjI0LjYxLjQzLjkyLjU3LjMuMTMuNjMuMi45OC4yLjM4IDAgLjY1LS4wOC44My0uMjNzLjI3LS4zNS4yNy0uNmMwLS4xNC0uMDUtLjI2LS4xMy0uMzctLjA4LS4xLS4yLS4yLS4zNC0uMjgtLjE0LS4wOS0uMjktLjE2LS40Ny0uMjNsLS41My0uMjJjLS4yMy0uMDktLjQ2LS4xOC0uNjktLjMtLjIzLS4xMS0uNDQtLjI0LS42Mi0uNHMtLjMzLS4zNS0uNDUtLjU1Yy0uMTItLjIxLS4xOC0uNDYtLjE4LS43NSAwLS42MS4yMy0xLjEuNjgtMS40OS40NC0uMzggMS4wNi0uNTcgMS44My0uNTcuNDggMCAuOTEuMDggMS4yOS4yNXMuNzEuMzYuOTkuNTdsLS43NC45OGMtLjI0LS4xNy0uNDktLjMyLS43My0uNDItLjI1LS4xMS0uNTEtLjE2LS43OC0uMTYtLjM1IDAtLjYuMDctLjc2LjIxLS4xNy4xNS0uMjUuMzMtLjI1LjU0IDAgLjE0LjA0LjI2LjEyLjM2cy4xOC4xOC4zMS4yNmMuMTQuMDcuMjkuMTQuNDYuMjFsLjU0LjE5Yy4yMy4wOS40Ny4xOC43LjI5cy40NC4yNC42NC40Yy4xOS4xNi4zNC4zNS40Ni41OC4xMS4yMy4xNy41LjE3LjgyIDAgLjMtLjA2LjU4LS4xNy44My0uMTIuMjYtLjI5LjQ4LS41MS42OC0uMjMuMTktLjUxLjM0LS44NC40NS0uMzQuMTEtLjcyLjE3LTEuMTUuMTctLjQ4IDAtLjk1LS4wOS0xLjQxLS4yNy0uNDYtLjE5LS44Ni0uNDEtMS4yLS42OHoiIGZpbGw9IiM1MzUzNTMiLz48L2c+PC9zdmc+"></a></div><div class="c-bibliographic-information__column"><h3 class="c-article__sub-heading" id="citeas">Cite this article</h3><p class="c-bibliographic-information__citation">Weissler, E.H., Naumann, T., Andersson, T. <i>et al.</i> The role of machine learning in clinical research: transforming the future of evidence generation. <i>Trials</i> <b>22</b>, 537 (2021). https://doi.org/10.1186/s13063-021-05489-x</p><p class="c-bibliographic-information__download-citation u-hide-print"><a data-test="citation-link" data-track="click" data-track-action="download article citation" data-track-label="link" data-track-external="" rel="nofollow" href="https://citation-needed.springer.com/v2/references/10.1186/s13063-021-05489-x?format=refman&flavour=citation">Download citation<svg width="16" height="16" focusable="false" role="img" aria-hidden="true" class="u-icon"><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#icon-eds-i-download-medium"></use></svg></a></p><ul class="c-bibliographic-information__list" data-test="publication-history"><li class="c-bibliographic-information__list-item"><p>Received<span class="u-hide">: </span><span class="c-bibliographic-information__value"><time datetime="2021-04-30">30 April 2021</time></span></p></li><li class="c-bibliographic-information__list-item"><p>Accepted<span class="u-hide">: </span><span class="c-bibliographic-information__value"><time datetime="2021-07-26">26 July 2021</time></span></p></li><li class="c-bibliographic-information__list-item"><p>Published<span class="u-hide">: </span><span class="c-bibliographic-information__value"><time datetime="2021-08-16">16 August 2021</time></span></p></li><li class="c-bibliographic-information__list-item c-bibliographic-information__list-item--full-width"><p><abbr title="Digital Object Identifier">DOI</abbr><span class="u-hide">: </span><span class="c-bibliographic-information__value">https://doi.org/10.1186/s13063-021-05489-x</span></p></li></ul><div data-component="share-box"><div class="c-article-share-box u-display-none" hidden=""><h3 class="c-article__sub-heading">Share this article</h3><p class="c-article-share-box__description">Anyone you share the following link with will be able to read this content:</p><button class="js-get-share-url c-article-share-box__button" type="button" id="get-share-url" data-track="click" data-track-label="button" data-track-external="" data-track-action="get shareable link">Get shareable link</button><div class="js-no-share-url-container u-display-none" hidden=""><p class="js-c-article-share-box__no-sharelink-info c-article-share-box__no-sharelink-info">Sorry, a shareable link is not currently available for this article.</p></div><div class="js-share-url-container u-display-none" hidden=""><p class="js-share-url c-article-share-box__only-read-input" id="share-url" data-track="click" data-track-label="button" data-track-action="select share url"></p><button class="js-copy-share-url c-article-share-box__button--link-like" type="button" id="copy-share-url" data-track="click" data-track-label="button" data-track-action="copy share url" data-track-external="">Copy to clipboard</button></div><p class="js-c-article-share-box__additional-info c-article-share-box__additional-info"> Provided by the Springer Nature SharedIt content-sharing initiative </p></div></div><h3 class="c-article__sub-heading">Keywords</h3><ul class="c-article-subject-list"><li class="c-article-subject-list__subject"><span><a href="/search?query=Clinical%20trials%20as%20topic%3B%20Machine%20learning&facet-discipline="Medicine%20%26%20Public%20Health"" data-track="click" data-track-action="view keyword" data-track-label="link">Clinical trials as topic; Machine learning</a></span></li><li class="c-article-subject-list__subject"><span><a href="/search?query=Artificial%20intelligence&facet-discipline="Medicine%20%26%20Public%20Health"" data-track="click" data-track-action="view keyword" data-track-label="link">Artificial intelligence</a></span></li><li class="c-article-subject-list__subject"><span><a href="/search?query=Research%20design&facet-discipline="Medicine%20%26%20Public%20Health"" data-track="click" data-track-action="view keyword" data-track-label="link">Research design</a></span></li><li class="c-article-subject-list__subject"><span><a href="/search?query=Research%20ethics&facet-discipline="Medicine%20%26%20Public%20Health"" data-track="click" data-track-action="view keyword" data-track-label="link">Research ethics</a></span></li></ul><div data-component="article-info-list"></div></div></div></div></div></section> </article> </main> <div class="c-article-extras u-text-sm u-hide-print" data-container-type="reading-companion" data-track-component="reading companion"> <aside> <div data-test="download-article-link-wrapper" class="js-context-bar-sticky-point-desktop" data-track-context="reading companion"> <div class="c-pdf-download u-clear-both"> <a href="//trialsjournal.biomedcentral.com/counter/pdf/10.1186/s13063-021-05489-x.pdf" class="u-button u-button--full-width u-button--primary u-justify-content-space-between c-pdf-download__link" data-article-pdf="true" data-readcube-pdf-url="true" data-test="pdf-link" data-draft-ignore="true" data-track="content_download" data-track-type="article pdf download" data-track-action="download pdf" data-track-label="link" data-track-external download> <span class="c-pdf-download__text">Download PDF</span> <svg aria-hidden="true" focusable="false" width="16" height="16" class="u-icon"><use xlink:href="#icon-download"/></svg> </a> </div> </div> <div class="c-reading-companion"> <div class="c-reading-companion__sticky" data-component="reading-companion-sticky" data-test="reading-companion-sticky"> <div class="c-reading-companion__panel c-reading-companion__sections c-reading-companion__panel--active" id="tabpanel-sections"> <div class="js-ad u-lazy-ad-wrapper u-mt-16 u-hide" data-component-mpu> <aside class="adsbox c-ad c-ad--300x250 u-mt-16" data-component-mpu> <div class="c-ad__inner"> <p class="c-ad__label">Advertisement</p> <div id="div-gpt-ad-MPU1" data-ad-type="MPU1" data-test="MPU1-ad" data-pa11y-ignore data-gpt data-gpt-unitpath="/270604982/bmc/trialsjournal/articles" data-gpt-sizes="300x250" data-gpt-targeting="pos=MPU1;doi=10.1186/s13063-021-05489-x;type=article;kwrd=Clinical trials as topic; Machine learning,Artificial intelligence,Research design,Research ethics;pmc=H00007,B0000X,S17030;" > <noscript> <a href="//pubads.g.doubleclick.net/gampad/jump?iu=/270604982/bmc/trialsjournal/articles&sz=300x250&pos=MPU1&doi=10.1186/s13063-021-05489-x&type=article&kwrd=Clinical trials as topic; Machine learning,Artificial intelligence,Research design,Research ethics&pmc=H00007,B0000X,S17030&"> <img data-test="gpt-advert-fallback-img" src="//pubads.g.doubleclick.net/gampad/ad?iu=/270604982/bmc/trialsjournal/articles&sz=300x250&pos=MPU1&doi=10.1186/s13063-021-05489-x&type=article&kwrd=Clinical trials as topic; Machine learning,Artificial intelligence,Research design,Research ethics&pmc=H00007,B0000X,S17030&" alt="Advertisement" width="300" height="250"> </a> </noscript> </div> </div> </aside> </div> </div> <div class="c-reading-companion__panel c-reading-companion__figures c-reading-companion__panel--full-width" id="tabpanel-figures"></div> <div class="c-reading-companion__panel c-reading-companion__references c-reading-companion__panel--full-width" id="tabpanel-references"></div> </div> </div> </aside> </div> </div> <div class="c-journal-footer"> <div class="c-journal-footer__inner"> <div class="c-journal-footer__summary"> <h4 class="c-journal-title c-journal-title--footer"> <img class="c-journal-title__logo" src="https://trialsjournal.biomedcentral.com/images/13063/logo.svg"/> <span class="c-journal-title__text">Trials</span> </h4> <p class="c-journal-footer__issn">ISSN: 1745-6215</p> </div> <div class="c-journal-footer__contact"> <h4 class="c-journal-footer__contact-title c-journal-footer__contact-title--adjust-for-logo">Contact us</h4> <ul class="c-journal-footer__contact-list"> <li class="c-journal-footer__contact-item">Submission enquiries: <a href="https://www.editorialmanager.com/trls/" target="_blank">Access here and click Contact Us</a></li> <li class="c-journal-footer__contact-item">General enquiries: <a href="mailto:journalsubmissions@springernature.com">journalsubmissions@springernature.com</a></li> </ul> </div> </div> </div> <img rel="nofollow" class='tracker' style='display:none' src='/track/article/10.1186/s13063-021-05489-x' alt=""/> <footer> <div class="c-publisher-footer u-color-inherit" data-test="publisher-footer"> <div class="u-container"> <div class="u-display-flex u-flex-wrap u-justify-content-space-between" data-test="publisher-footer-menu"> <div class="u-display-flex"> <ul class="c-list-group c-list-group--sm u-mr-24 u-mb-16"> <li class="c-list-group__item"> <a class="u-gray-link" href="https://blogs.biomedcentral.com/">Read more on our blogs</a> </li> <li class="c-list-group__item"> <a class="u-gray-link" href="//www.biomedcentral.com/login">Receive BMC newsletters</a> </li> <li class="c-list-group__item"> <a class="u-gray-link" href="//www.biomedcentral.com/account">Manage article alerts</a> </li> <li class="c-list-group__item"> <a class="u-gray-link" href="https://authorservices.springernature.com/go/sn/?utm_source=Website&utm_medium=BMC&utm_campaign=SNAS+Referrals+2022&utm_id=ref2022">Language editing for authors</a> </li> <li class="c-list-group__item"> <a class="u-gray-link" href="https://authorservices.springernature.com/go/sn/?utm_source=Website&utm_medium=BMC&utm_campaign=SNAS+Referrals+2022&utm_id=ref2022">Scientific editing for authors</a> </li> </ul> <ul class="c-list-group c-list-group--sm u-mr-24 u-mb-16"> <li class="c-list-group__item"> <a class="u-gray-link" href="//www.biomedcentral.com/about/policies">Policies</a> </li> <li class="c-list-group__item"> <a class="u-gray-link" href="//www.biomedcentral.com/accessibility">Accessibility</a> </li> <li class="c-list-group__item"> <a class="u-gray-link" href="//www.biomedcentral.com/about/press-centre">Press center</a> </li> </ul> <ul class="c-list-group c-list-group--sm u-mr-24 u-mb-16"> <li class="c-list-group__item"> <a class="u-gray-link" href="https://support.biomedcentral.com/support/home">Support and Contact</a> </li> <li class="c-list-group__item"> <a class="u-gray-link" href="https://biomedcentral.typeform.com/to/VLXboo">Leave feedback</a> </li> <li class="c-list-group__item"> <a class="u-gray-link" href="//www.biomedcentral.com/about/jobs">Careers</a> </li> </ul> </div> <div class="u-mb-24"> <h3 id="social-menu" class="u-text-sm u-reset-margin u-text-normal">Follow BMC</h3> <ul class="u-display-flex u-list-reset" data-test="footer-social-links"> <li class="u-mt-8 u-mr-8"> <a href="https://twitter.com/biomedcentral" data-track="click" data-track-category="Social" data-track-action="Clicked BMC Twitter" class="u-gray-link"> <span class="u-visually-hidden">BMC Twitter page</span> <svg class="u-icon u-text-lg" width="24" height="24" aria-hidden="true" focusable="false"> <use xlink:href="#icon-twitter-bordered"></use> </svg> </a> </li> <li class="u-mt-8 u-mr-8"> <a href="https://www.facebook.com/BioMedCentral" data-track="click" data-track-category="Social" data-track-action="Clicked BMC Facebook" class="u-gray-link"> <span class="u-visually-hidden">BMC Facebook page</span> <svg class="u-icon u-text-lg" width="24" height="24" aria-hidden="true" focusable="false"> <use xlink:href="#icon-facebook-bordered"></use> </svg> </a> </li> <li class="u-mt-8 u-mr-8"> <a href="http://www.weibo.com/biomedcentral" data-track="click" data-track-category="Social" data-track-action="Clicked BMC Weibo" class="u-gray-link"> <span class="u-visually-hidden">BMC Weibo page</span> <svg class="u-icon u-text-lg" width="24" height="24" aria-hidden="true" focusable="false"> <use xlink:href="#icon-weibo-bordered"></use> </svg> </a> </li> </ul> </div> </div> <p class="u-reset-margin"> By using this website, you agree to our <a class="u-gray-link" href="//www.biomedcentral.com/terms-and-conditions">Terms and Conditions</a>, <a class="u-gray-link" href="https://www.springernature.com/ccpa">Your US state privacy rights</a>, <a class="u-gray-link" href="//www.biomedcentral.com/privacy-statement">Privacy statement</a> and <a class="u-gray-link" href="//www.biomedcentral.com/cookies" data-test="cookie-link">Cookies</a> policy. <a class="u-gray-link" data-cc-action="preferences" href="javascript:void(0);">Your privacy choices/Manage cookies</a> we use in the preference centre. </p> </div> </div> <div class="c-corporate-footer"> <div class="u-container"> <img src=/static/images/logo-springernature-acb40b85fb.svg class="c-corporate-footer__logo" alt="Springer Nature" itemprop="logo" role="img"> <p class="c-corporate-footer__legal" data-test="copyright"> © 2024 BioMed Central Ltd unless otherwise stated. Part of <a class="c-corporate-footer__link" href="https://www.springernature.com" itemscope itemtype="http://schema.org/Organization" itemid="#parentOrganization">Springer Nature</a>. </p> </div> </div> </footer> </div> <div class="u-visually-hidden" aria-hidden="true"> <?xml version="1.0" encoding="UTF-8"?><!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN" "http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd"><svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink"><defs><path id="a" d="M0 .74h56.72v55.24H0z"/></defs><symbol id="icon-access" viewBox="0 0 18 18"><path d="m14 8c.5522847 0 1 .44771525 1 1v7h2.5c.2761424 0 .5.2238576.5.5v1.5h-18v-1.5c0-.2761424.22385763-.5.5-.5h2.5v-7c0-.55228475.44771525-1 1-1s1 .44771525 1 1v6.9996556h8v-6.9996556c0-.55228475.4477153-1 1-1zm-8 0 2 1v5l-2 1zm6 0v7l-2-1v-5zm-2.42653766-7.59857636 7.03554716 4.92488299c.4162533.29137735.5174853.86502537.226108 1.28127873-.1721584.24594054-.4534847.39241464-.7536934.39241464h-14.16284822c-.50810197 0-.92-.41189803-.92-.92 0-.30020869.1464741-.58153499.39241464-.75369337l7.03554714-4.92488299c.34432015-.2410241.80260453-.2410241 1.14692468 0zm-.57346234 2.03988748-3.65526982 2.55868888h7.31053962z" fill-rule="evenodd"/></symbol><symbol id="icon-account" viewBox="0 0 18 18"><path d="m10.2379028 16.9048051c1.3083556-.2032362 2.5118471-.7235183 3.5294683-1.4798399-.8731327-2.5141501-2.0638925-3.935978-3.7673711-4.3188248v-1.27684611c1.1651924-.41183641 2-1.52307546 2-2.82929429 0-1.65685425-1.3431458-3-3-3-1.65685425 0-3 1.34314575-3 3 0 1.30621883.83480763 2.41745788 2 2.82929429v1.27684611c-1.70347856.3828468-2.89423845 1.8046747-3.76737114 4.3188248 1.01762123.7563216 2.22111275 1.2766037 3.52946833 1.4798399.40563808.0629726.81921174.0951949 1.23790281.0951949s.83226473-.0322223 1.2379028-.0951949zm4.3421782-2.1721994c1.4927655-1.4532925 2.419919-3.484675 2.419919-5.7326057 0-4.418278-3.581722-8-8-8s-8 3.581722-8 8c0 2.2479307.92715352 4.2793132 2.41991895 5.7326057.75688473-2.0164459 1.83949951-3.6071894 3.48926591-4.3218837-1.14534283-.70360829-1.90918486-1.96796271-1.90918486-3.410722 0-2.209139 1.790861-4 4-4s4 1.790861 4 4c0 1.44275929-.763842 2.70711371-1.9091849 3.410722 1.6497664.7146943 2.7323812 2.3054378 3.4892659 4.3218837zm-5.580081 3.2673943c-4.97056275 0-9-4.0294373-9-9 0-4.97056275 4.02943725-9 9-9 4.9705627 0 9 4.02943725 9 9 0 4.9705627-4.0294373 9-9 9z" fill-rule="evenodd"/></symbol><symbol id="icon-alert" viewBox="0 0 18 18"><path d="m4 10h2.5c.27614237 0 .5.2238576.5.5s-.22385763.5-.5.5h-3.08578644l-1.12132034 1.1213203c-.18753638.1875364-.29289322.4418903-.29289322.7071068v.1715729h14v-.1715729c0-.2652165-.1053568-.5195704-.2928932-.7071068l-1.7071068-1.7071067v-3.4142136c0-2.76142375-2.2385763-5-5-5-2.76142375 0-5 2.23857625-5 5zm3 4c0 1.1045695.8954305 2 2 2s2-.8954305 2-2zm-5 0c-.55228475 0-1-.4477153-1-1v-.1715729c0-.530433.21071368-1.0391408.58578644-1.4142135l1.41421356-1.4142136v-3c0-3.3137085 2.6862915-6 6-6s6 2.6862915 6 6v3l1.4142136 1.4142136c.3750727.3750727.5857864.8837805.5857864 1.4142135v.1715729c0 .5522847-.4477153 1-1 1h-4c0 1.6568542-1.3431458 3-3 3-1.65685425 0-3-1.3431458-3-3z" fill-rule="evenodd"/></symbol><symbol id="icon-arrow-broad" viewBox="0 0 16 16"><path d="m6.10307866 2.97190702v7.69043288l2.44965196-2.44676915c.38776071-.38730439 1.0088052-.39493524 1.38498697-.01919617.38609051.38563612.38643641 1.01053024-.00013864 1.39665039l-4.12239817 4.11754683c-.38616704.3857126-1.01187344.3861062-1.39846576-.0000311l-4.12258206-4.11773056c-.38618426-.38572979-.39254614-1.00476697-.01636437-1.38050605.38609047-.38563611 1.01018509-.38751562 1.4012233.00306241l2.44985644 2.4469734v-8.67638639c0-.54139983.43698413-.98042709.98493125-.98159081l7.89910522-.0043627c.5451687 0 .9871152.44142642.9871152.98595351s-.4419465.98595351-.9871152.98595351z" fill-rule="evenodd" transform="matrix(-1 0 0 -1 14 15)"/></symbol><symbol id="icon-arrow-down" viewBox="0 0 16 16"><path d="m3.28337502 11.5302405 4.03074001 4.176208c.37758093.3912076.98937525.3916069 1.367372-.0000316l4.03091977-4.1763942c.3775978-.3912252.3838182-1.0190815.0160006-1.4001736-.3775061-.39113013-.9877245-.39303641-1.3700683.003106l-2.39538585 2.4818345v-11.6147896l-.00649339-.11662112c-.055753-.49733869-.46370161-.88337888-.95867408-.88337888-.49497246 0-.90292107.38604019-.95867408.88337888l-.00649338.11662112v11.6147896l-2.39518594-2.4816273c-.37913917-.39282218-.98637524-.40056175-1.35419292-.0194697-.37750607.3911302-.37784433 1.0249269.00013556 1.4165479z" fill-rule="evenodd"/></symbol><symbol id="icon-arrow-left" viewBox="0 0 16 16"><path d="m4.46975946 3.28337502-4.17620792 4.03074001c-.39120768.37758093-.39160691.98937525.0000316 1.367372l4.1763942 4.03091977c.39122514.3775978 1.01908149.3838182 1.40017357.0160006.39113012-.3775061.3930364-.9877245-.00310603-1.3700683l-2.48183446-2.39538585h11.61478958l.1166211-.00649339c.4973387-.055753.8833789-.46370161.8833789-.95867408 0-.49497246-.3860402-.90292107-.8833789-.95867408l-.1166211-.00649338h-11.61478958l2.4816273-2.39518594c.39282216-.37913917.40056173-.98637524.01946965-1.35419292-.39113012-.37750607-1.02492687-.37784433-1.41654791.00013556z" fill-rule="evenodd"/></symbol><symbol id="icon-arrow-right" viewBox="0 0 16 16"><path d="m11.5302405 12.716625 4.176208-4.03074003c.3912076-.37758093.3916069-.98937525-.0000316-1.367372l-4.1763942-4.03091981c-.3912252-.37759778-1.0190815-.38381821-1.4001736-.01600053-.39113013.37750607-.39303641.98772445.003106 1.37006824l2.4818345 2.39538588h-11.6147896l-.11662112.00649339c-.49733869.055753-.88337888.46370161-.88337888.95867408 0 .49497246.38604019.90292107.88337888.95867408l.11662112.00649338h11.6147896l-2.4816273 2.39518592c-.39282218.3791392-.40056175.9863753-.0194697 1.3541929.3911302.3775061 1.0249269.3778444 1.4165479-.0001355z" fill-rule="evenodd"/></symbol><symbol id="icon-arrow-sub" viewBox="0 0 16 16"><path d="m7.89692134 4.97190702v7.69043288l-2.44965196-2.4467692c-.38776071-.38730434-1.0088052-.39493519-1.38498697-.0191961-.38609047.3856361-.38643643 1.0105302.00013864 1.3966504l4.12239817 4.1175468c.38616704.3857126 1.01187344.3861062 1.39846576-.0000311l4.12258202-4.1177306c.3861843-.3857298.3925462-1.0047669.0163644-1.380506-.3860905-.38563612-1.0101851-.38751563-1.4012233.0030624l-2.44985643 2.4469734v-8.67638639c0-.54139983-.43698413-.98042709-.98493125-.98159081l-7.89910525-.0043627c-.54516866 0-.98711517.44142642-.98711517.98595351s.44194651.98595351.98711517.98595351z" fill-rule="evenodd"/></symbol><symbol id="icon-arrow-up" viewBox="0 0 16 16"><path d="m12.716625 4.46975946-4.03074003-4.17620792c-.37758093-.39120768-.98937525-.39160691-1.367372.0000316l-4.03091981 4.1763942c-.37759778.39122514-.38381821 1.01908149-.01600053 1.40017357.37750607.39113012.98772445.3930364 1.37006824-.00310603l2.39538588-2.48183446v11.61478958l.00649339.1166211c.055753.4973387.46370161.8833789.95867408.8833789.49497246 0 .90292107-.3860402.95867408-.8833789l.00649338-.1166211v-11.61478958l2.39518592 2.4816273c.3791392.39282216.9863753.40056173 1.3541929.01946965.3775061-.39113012.3778444-1.02492687-.0001355-1.41654791z" fill-rule="evenodd"/></symbol><symbol id="icon-article" viewBox="0 0 18 18"><path d="m13 15v-12.9906311c0-.0073595-.0019884-.0093689.0014977-.0093689l-11.00158888.00087166v13.00506804c0 .5482678.44615281.9940603.99415146.9940603h10.27350412c-.1701701-.2941734-.2675644-.6357129-.2675644-1zm-12 .0059397v-13.00506804c0-.5562408.44704472-1.00087166.99850233-1.00087166h11.00299537c.5510129 0 .9985023.45190985.9985023 1.0093689v2.9906311h3v9.9914698c0 1.1065798-.8927712 2.0085302-1.9940603 2.0085302h-12.01187942c-1.09954652 0-1.99406028-.8927712-1.99406028-1.9940603zm13-9.0059397v9c0 .5522847.4477153 1 1 1s1-.4477153 1-1v-9zm-10-2h7v4h-7zm1 1v2h5v-2zm-1 4h7v1h-7zm0 2h7v1h-7zm0 2h7v1h-7z" fill-rule="evenodd"/></symbol><symbol id="icon-audio" viewBox="0 0 18 18"><path d="m13.0957477 13.5588459c-.195279.1937043-.5119137.193729-.7072234.0000551-.1953098-.193674-.1953346-.5077061-.0000556-.7014104 1.0251004-1.0168342 1.6108711-2.3905226 1.6108711-3.85745208 0-1.46604976-.5850634-2.83898246-1.6090736-3.85566829-.1951894-.19379323-.1950192-.50782531.0003802-.70141028.1953993-.19358497.512034-.19341614.7072234.00037709 1.2094886 1.20083761 1.901635 2.8250555 1.901635 4.55670148 0 1.73268608-.6929822 3.35779608-1.9037571 4.55880738zm2.1233994 2.1025159c-.195234.193749-.5118687.1938462-.7072235.0002171-.1953548-.1936292-.1954528-.5076613-.0002189-.7014104 1.5832215-1.5711805 2.4881302-3.6939808 2.4881302-5.96012998 0-2.26581266-.9046382-4.3883241-2.487443-5.95944795-.1952117-.19377107-.1950777-.50780316.0002993-.70141031s.5120117-.19347426.7072234.00029682c1.7683321 1.75528196 2.7800854 4.12911258 2.7800854 6.66056144 0 2.53182498-1.0120556 4.90597838-2.7808529 6.66132328zm-14.21898205-3.6854911c-.5523759 0-1.00016505-.4441085-1.00016505-.991944v-3.96777631c0-.54783558.44778915-.99194407 1.00016505-.99194407h2.0003301l5.41965617-3.8393633c.44948677-.31842296 1.07413994-.21516983 1.39520191.23062232.12116339.16823446.18629727.36981184.18629727.57655577v12.01603479c0 .5478356-.44778914.9919441-1.00016505.9919441-.20845738 0-.41170538-.0645985-.58133413-.184766l-5.41965617-3.8393633zm0-.991944h2.32084805l5.68047235 4.0241292v-12.01603479l-5.68047235 4.02412928h-2.32084805z" fill-rule="evenodd"/></symbol><symbol id="icon-block" viewBox="0 0 24 24"><path d="m0 0h24v24h-24z" fill-rule="evenodd"/></symbol><symbol id="icon-book" viewBox="0 0 18 18"><path d="m4 13v-11h1v11h11v-11h-13c-.55228475 0-1 .44771525-1 1v10.2675644c.29417337-.1701701.63571286-.2675644 1-.2675644zm12 1h-13c-.55228475 0-1 .4477153-1 1s.44771525 1 1 1h13zm0 3h-13c-1.1045695 0-2-.8954305-2-2v-12c0-1.1045695.8954305-2 2-2h13c.5522847 0 1 .44771525 1 1v14c0 .5522847-.4477153 1-1 1zm-8.5-13h6c.2761424 0 .5.22385763.5.5s-.2238576.5-.5.5h-6c-.27614237 0-.5-.22385763-.5-.5s.22385763-.5.5-.5zm1 2h4c.2761424 0 .5.22385763.5.5s-.2238576.5-.5.5h-4c-.27614237 0-.5-.22385763-.5-.5s.22385763-.5.5-.5z" fill-rule="evenodd"/></symbol><symbol id="icon-broad" viewBox="0 0 24 24"><path d="m9.18274226 7.81v7.7999954l2.48162734-2.4816273c.3928221-.3928221 1.0219731-.4005617 1.4030652-.0194696.3911301.3911301.3914806 1.0249268-.0001404 1.4165479l-4.17620796 4.1762079c-.39120769.3912077-1.02508144.3916069-1.41671995-.0000316l-4.1763942-4.1763942c-.39122514-.3912251-.39767006-1.0190815-.01657798-1.4001736.39113012-.3911301 1.02337106-.3930364 1.41951349.0031061l2.48183446 2.4818344v-8.7999954c0-.54911294.4426881-.99439484.99778758-.99557515l8.00221246-.00442485c.5522847 0 1 .44771525 1 1s-.4477153 1-1 1z" fill-rule="evenodd" transform="matrix(-1 0 0 -1 20.182742 24.805206)"/></symbol><symbol id="icon-calendar" viewBox="0 0 18 18"><path d="m12.5 0c.2761424 0 .5.21505737.5.49047852v.50952148h2c1.1072288 0 2 .89451376 2 2v12c0 1.1072288-.8945138 2-2 2h-12c-1.1072288 0-2-.8945138-2-2v-12c0-1.1072288.89451376-2 2-2h1v1h-1c-.55393837 0-1 .44579254-1 1v3h14v-3c0-.55393837-.4457925-1-1-1h-2v1.50952148c0 .27088381-.2319336.49047852-.5.49047852-.2761424 0-.5-.21505737-.5-.49047852v-3.01904296c0-.27088381.2319336-.49047852.5-.49047852zm3.5 7h-14v8c0 .5539384.44579254 1 1 1h12c.5539384 0 1-.4457925 1-1zm-11 6v1h-1v-1zm3 0v1h-1v-1zm3 0v1h-1v-1zm-6-2v1h-1v-1zm3 0v1h-1v-1zm6 0v1h-1v-1zm-3 0v1h-1v-1zm-3-2v1h-1v-1zm6 0v1h-1v-1zm-3 0v1h-1v-1zm-5.5-9c.27614237 0 .5.21505737.5.49047852v.50952148h5v1h-5v1.50952148c0 .27088381-.23193359.49047852-.5.49047852-.27614237 0-.5-.21505737-.5-.49047852v-3.01904296c0-.27088381.23193359-.49047852.5-.49047852z" fill-rule="evenodd"/></symbol><symbol id="icon-cart" viewBox="0 0 18 18"><path d="m5 14c1.1045695 0 2 .8954305 2 2s-.8954305 2-2 2-2-.8954305-2-2 .8954305-2 2-2zm10 0c1.1045695 0 2 .8954305 2 2s-.8954305 2-2 2-2-.8954305-2-2 .8954305-2 2-2zm-10 1c-.55228475 0-1 .4477153-1 1s.44771525 1 1 1 1-.4477153 1-1-.44771525-1-1-1zm10 0c-.5522847 0-1 .4477153-1 1s.4477153 1 1 1 1-.4477153 1-1-.4477153-1-1-1zm-12.82032249-15c.47691417 0 .88746157.33678127.98070211.80449199l.23823144 1.19501025 13.36277974.00045554c.5522847.00001882.9999659.44774934.9999659 1.00004222 0 .07084994-.0075361.14150708-.022474.2107727l-1.2908094 5.98534344c-.1007861.46742419-.5432548.80388386-1.0571651.80388386h-10.24805106c-.59173366 0-1.07142857.4477153-1.07142857 1 0 .5128358.41361449.9355072.94647737.9932723l.1249512.0067277h10.35933776c.2749512 0 .4979349.2228539.4979349.4978051 0 .2749417-.2227336.4978951-.4976753.4980063l-10.35959736.0041886c-1.18346732 0-2.14285714-.8954305-2.14285714-2 0-.6625717.34520317-1.24989198.87690425-1.61383592l-1.63768102-8.19004794c-.01312273-.06561364-.01950005-.131011-.0196107-.19547395l-1.71961253-.00064219c-.27614237 0-.5-.22385762-.5-.5 0-.27614237.22385763-.5.5-.5zm14.53193359 2.99950224h-13.11300004l1.20580469 6.02530174c.11024034-.0163252.22327998-.02480398.33844139-.02480398h10.27064786z"/></symbol><symbol id="icon-chevron-less" viewBox="0 0 10 10"><path d="m5.58578644 4-3.29289322-3.29289322c-.39052429-.39052429-.39052429-1.02368927 0-1.41421356s1.02368927-.39052429 1.41421356 0l4 4c.39052429.39052429.39052429 1.02368927 0 1.41421356l-4 4c-.39052429.39052429-1.02368927.39052429-1.41421356 0s-.39052429-1.02368927 0-1.41421356z" fill-rule="evenodd" transform="matrix(0 -1 -1 0 9 9)"/></symbol><symbol id="icon-chevron-more" viewBox="0 0 10 10"><path d="m5.58578644 6-3.29289322-3.29289322c-.39052429-.39052429-.39052429-1.02368927 0-1.41421356s1.02368927-.39052429 1.41421356 0l4 4c.39052429.39052429.39052429 1.02368927 0 1.41421356l-4 4.00000002c-.39052429.3905243-1.02368927.3905243-1.41421356 0s-.39052429-1.02368929 0-1.41421358z" fill-rule="evenodd" transform="matrix(0 1 -1 0 11 1)"/></symbol><symbol id="icon-chevron-right" viewBox="0 0 10 10"><path d="m5.96738168 4.70639573 2.39518594-2.41447274c.37913917-.38219212.98637524-.38972225 1.35419292-.01894278.37750606.38054586.37784436.99719163-.00013556 1.37821513l-4.03074001 4.06319683c-.37758093.38062133-.98937525.38100976-1.367372-.00003075l-4.03091981-4.06337806c-.37759778-.38063832-.38381821-.99150444-.01600053-1.3622839.37750607-.38054587.98772445-.38240057 1.37006824.00302197l2.39538588 2.4146743.96295325.98624457z" fill-rule="evenodd" transform="matrix(0 -1 1 0 0 10)"/></symbol><symbol id="icon-circle-fill" viewBox="0 0 16 16"><path d="m8 14c-3.3137085 0-6-2.6862915-6-6s2.6862915-6 6-6 6 2.6862915 6 6-2.6862915 6-6 6z" fill-rule="evenodd"/></symbol><symbol id="icon-circle" viewBox="0 0 16 16"><path d="m8 12c2.209139 0 4-1.790861 4-4s-1.790861-4-4-4-4 1.790861-4 4 1.790861 4 4 4zm0 2c-3.3137085 0-6-2.6862915-6-6s2.6862915-6 6-6 6 2.6862915 6 6-2.6862915 6-6 6z" fill-rule="evenodd"/></symbol><symbol id="icon-citation" viewBox="0 0 18 18"><path d="m8.63593473 5.99995183c2.20913897 0 3.99999997 1.79084375 3.99999997 3.99996146 0 1.40730761-.7267788 2.64486871-1.8254829 3.35783281 1.6240224.6764218 2.8754442 2.0093871 3.4610603 3.6412466l-1.0763845.000006c-.5310008-1.2078237-1.5108121-2.1940153-2.7691712-2.7181346l-.79002167-.329052v-1.023992l.63016577-.4089232c.8482885-.5504661 1.3698342-1.4895187 1.3698342-2.51898361 0-1.65683828-1.3431457-2.99996146-2.99999997-2.99996146-1.65685425 0-3 1.34312318-3 2.99996146 0 1.02946491.52154569 1.96851751 1.36983419 2.51898361l.63016581.4089232v1.023992l-.79002171.329052c-1.25835905.5241193-2.23817037 1.5103109-2.76917113 2.7181346l-1.07638453-.000006c.58561612-1.6318595 1.8370379-2.9648248 3.46106024-3.6412466-1.09870405-.7129641-1.82548287-1.9505252-1.82548287-3.35783281 0-2.20911771 1.790861-3.99996146 4-3.99996146zm7.36897597-4.99995183c1.1018574 0 1.9950893.89353404 1.9950893 2.00274083v5.994422c0 1.10608317-.8926228 2.00274087-1.9950893 2.00274087l-3.0049107-.0009037v-1l3.0049107.00091329c.5490631 0 .9950893-.44783123.9950893-1.00275046v-5.994422c0-.55646537-.4450595-1.00275046-.9950893-1.00275046h-14.00982141c-.54906309 0-.99508929.44783123-.99508929 1.00275046v5.9971821c0 .66666024.33333333.99999036 1 .99999036l2-.00091329v1l-2 .0009037c-1 0-2-.99999041-2-1.99998077v-5.9971821c0-1.10608322.8926228-2.00274083 1.99508929-2.00274083zm-8.5049107 2.9999711c.27614237 0 .5.22385547.5.5 0 .2761349-.22385763.5-.5.5h-4c-.27614237 0-.5-.2238651-.5-.5 0-.27614453.22385763-.5.5-.5zm3 0c.2761424 0 .5.22385547.5.5 0 .2761349-.2238576.5-.5.5h-1c-.27614237 0-.5-.2238651-.5-.5 0-.27614453.22385763-.5.5-.5zm4 0c.2761424 0 .5.22385547.5.5 0 .2761349-.2238576.5-.5.5h-2c-.2761424 0-.5-.2238651-.5-.5 0-.27614453.2238576-.5.5-.5z" fill-rule="evenodd"/></symbol><symbol id="icon-close" viewBox="0 0 16 16"><path d="m2.29679575 12.2772478c-.39658757.3965876-.39438847 1.0328109-.00062148 1.4265779.39651227.3965123 1.03246768.3934888 1.42657791-.0006214l4.27724782-4.27724787 4.2772478 4.27724787c.3965876.3965875 1.0328109.3943884 1.4265779.0006214.3965123-.3965122.3934888-1.0324677-.0006214-1.4265779l-4.27724787-4.2772478 4.27724787-4.27724782c.3965875-.39658757.3943884-1.03281091.0006214-1.42657791-.3965122-.39651226-1.0324677-.39348875-1.4265779.00062148l-4.2772478 4.27724782-4.27724782-4.27724782c-.39658757-.39658757-1.03281091-.39438847-1.42657791-.00062148-.39651226.39651227-.39348875 1.03246768.00062148 1.42657791l4.27724782 4.27724782z" fill-rule="evenodd"/></symbol><symbol id="icon-collections" viewBox="0 0 18 18"><path d="m15 4c1.1045695 0 2 .8954305 2 2v9c0 1.1045695-.8954305 2-2 2h-8c-1.1045695 0-2-.8954305-2-2h1c0 .5128358.38604019.9355072.88337887.9932723l.11662113.0067277h8c.5128358 0 .9355072-.3860402.9932723-.8833789l.0067277-.1166211v-9c0-.51283584-.3860402-.93550716-.8833789-.99327227l-.1166211-.00672773h-1v-1zm-4-3c1.1045695 0 2 .8954305 2 2v9c0 1.1045695-.8954305 2-2 2h-8c-1.1045695 0-2-.8954305-2-2v-9c0-1.1045695.8954305-2 2-2zm0 1h-8c-.51283584 0-.93550716.38604019-.99327227.88337887l-.00672773.11662113v9c0 .5128358.38604019.9355072.88337887.9932723l.11662113.0067277h8c.5128358 0 .9355072-.3860402.9932723-.8833789l.0067277-.1166211v-9c0-.51283584-.3860402-.93550716-.8833789-.99327227zm-1.5 7c.27614237 0 .5.22385763.5.5s-.22385763.5-.5.5h-5c-.27614237 0-.5-.22385763-.5-.5s.22385763-.5.5-.5zm0-2c.27614237 0 .5.22385763.5.5s-.22385763.5-.5.5h-5c-.27614237 0-.5-.22385763-.5-.5s.22385763-.5.5-.5zm0-2c.27614237 0 .5.22385763.5.5s-.22385763.5-.5.5h-5c-.27614237 0-.5-.22385763-.5-.5s.22385763-.5.5-.5z" fill-rule="evenodd"/></symbol><symbol id="icon-compare" viewBox="0 0 18 18"><path d="m12 3c3.3137085 0 6 2.6862915 6 6s-2.6862915 6-6 6c-1.0928452 0-2.11744941-.2921742-2.99996061-.8026704-.88181407.5102749-1.90678042.8026704-3.00003939.8026704-3.3137085 0-6-2.6862915-6-6s2.6862915-6 6-6c1.09325897 0 2.11822532.29239547 3.00096303.80325037.88158756-.51107621 1.90619177-.80325037 2.99903697-.80325037zm-6 1c-2.76142375 0-5 2.23857625-5 5 0 2.7614237 2.23857625 5 5 5 .74397391 0 1.44999672-.162488 2.08451611-.4539116-1.27652344-1.1000812-2.08451611-2.7287264-2.08451611-4.5460884s.80799267-3.44600721 2.08434391-4.5463015c-.63434719-.29121054-1.34037-.4536985-2.08434391-.4536985zm6 0c-.7439739 0-1.4499967.16248796-2.08451611.45391156 1.27652341 1.10008123 2.08451611 2.72872644 2.08451611 4.54608844s-.8079927 3.4460072-2.08434391 4.5463015c.63434721.2912105 1.34037001.4536985 2.08434391.4536985 2.7614237 0 5-2.2385763 5-5 0-2.76142375-2.2385763-5-5-5zm-1.4162763 7.0005324h-3.16744736c.15614659.3572676.35283837.6927622.58425872 1.0006671h1.99892988c.23142036-.3079049.42811216-.6433995.58425876-1.0006671zm.4162763-2.0005324h-4c0 .34288501.0345146.67770871.10025909 1.0011864h3.79948181c.0657445-.32347769.1002591-.65830139.1002591-1.0011864zm-.4158423-1.99953894h-3.16831543c-.13859957.31730812-.24521946.651783-.31578599.99935097h3.79988742c-.0705665-.34756797-.1771864-.68204285-.315786-.99935097zm-1.58295822-1.999926-.08316107.06199199c-.34550042.27081213-.65446126.58611297-.91825862.93727862h2.00044041c-.28418626-.37830727-.6207872-.71499149-.99902072-.99927061z" fill-rule="evenodd"/></symbol><symbol id="icon-download-file" viewBox="0 0 18 18"><path d="m10.0046024 0c.5497429 0 1.3179837.32258606 1.707238.71184039l4.5763192 4.57631922c.3931386.39313859.7118404 1.16760135.7118404 1.71431368v8.98899651c0 1.1092806-.8945138 2.0085302-1.9940603 2.0085302h-12.01187942c-1.10128908 0-1.99406028-.8926228-1.99406028-1.9950893v-14.00982141c0-1.10185739.88743329-1.99508929 1.99961498-1.99508929zm0 1h-7.00498742c-.55709576 0-.99961498.44271433-.99961498.99508929v14.00982141c0 .5500396.44491393.9950893.99406028.9950893h12.01187942c.5463747 0 .9940603-.4506622.9940603-1.0085302v-8.98899651c0-.28393444-.2150684-.80332809-.4189472-1.0072069l-4.5763192-4.57631922c-.2038461-.20384606-.718603-.41894717-1.0001312-.41894717zm-1.5046024 4c.27614237 0 .5.21637201.5.49209595v6.14827645l1.7462789-1.77990922c.1933927-.1971171.5125222-.19455839.7001689-.0069117.1932998.19329992.1910058.50899492-.0027774.70277812l-2.59089271 2.5908927c-.19483374.1948337-.51177825.1937771-.70556873-.0000133l-2.59099079-2.5909908c-.19484111-.1948411-.19043735-.5151448-.00279066-.70279146.19329987-.19329987.50465175-.19237083.70018565.00692852l1.74638684 1.78001764v-6.14827695c0-.27177709.23193359-.49209595.5-.49209595z" fill-rule="evenodd"/></symbol><symbol id="icon-download" viewBox="0 0 16 16"><path d="m12.9975267 12.999368c.5467123 0 1.0024733.4478567 1.0024733 1.000316 0 .5563109-.4488226 1.000316-1.0024733 1.000316h-9.99505341c-.54671233 0-1.00247329-.4478567-1.00247329-1.000316 0-.5563109.44882258-1.000316 1.00247329-1.000316zm-4.9975267-11.999368c.55228475 0 1 .44497754 1 .99589209v6.80214418l2.4816273-2.48241149c.3928222-.39294628 1.0219732-.4006883 1.4030652-.01947579.3911302.39125371.3914806 1.02525073-.0001404 1.41699553l-4.17620792 4.17752758c-.39120769.3913313-1.02508144.3917306-1.41671995-.0000316l-4.17639421-4.17771394c-.39122513-.39134876-.39767006-1.01940351-.01657797-1.40061601.39113012-.39125372 1.02337105-.3931606 1.41951349.00310701l2.48183446 2.48261871v-6.80214418c0-.55001601.44386482-.99589209 1-.99589209z" fill-rule="evenodd"/></symbol><symbol id="icon-editors" viewBox="0 0 18 18"><path d="m8.72592184 2.54588137c-.48811714-.34391207-1.08343326-.54588137-1.72592184-.54588137-1.65685425 0-3 1.34314575-3 3 0 1.02947485.5215457 1.96853646 1.3698342 2.51900785l.6301658.40892721v1.02400182l-.79002171.32905522c-1.93395773.8055207-3.20997829 2.7024791-3.20997829 4.8180274v.9009805h-1v-.9009805c0-2.5479714 1.54557359-4.79153984 3.82548288-5.7411543-1.09870406-.71297106-1.82548288-1.95054399-1.82548288-3.3578652 0-2.209139 1.790861-4 4-4 1.09079823 0 2.07961816.43662103 2.80122451 1.1446278-.37707584.09278571-.7373238.22835063-1.07530267.40125357zm-2.72592184 14.45411863h-1v-.9009805c0-2.5479714 1.54557359-4.7915398 3.82548288-5.7411543-1.09870406-.71297106-1.82548288-1.95054399-1.82548288-3.3578652 0-2.209139 1.790861-4 4-4s4 1.790861 4 4c0 1.40732121-.7267788 2.64489414-1.8254829 3.3578652 2.2799093.9496145 3.8254829 3.1931829 3.8254829 5.7411543v.9009805h-1v-.9009805c0-2.1155483-1.2760206-4.0125067-3.2099783-4.8180274l-.7900217-.3290552v-1.02400184l.6301658-.40892721c.8482885-.55047139 1.3698342-1.489533 1.3698342-2.51900785 0-1.65685425-1.3431458-3-3-3-1.65685425 0-3 1.34314575-3 3 0 1.02947485.5215457 1.96853646 1.3698342 2.51900785l.6301658.40892721v1.02400184l-.79002171.3290552c-1.93395773.8055207-3.20997829 2.7024791-3.20997829 4.8180274z" fill-rule="evenodd"/></symbol><symbol id="icon-email" viewBox="0 0 18 18"><path d="m16.0049107 2c1.1018574 0 1.9950893.89706013 1.9950893 2.00585866v9.98828264c0 1.1078052-.8926228 2.0058587-1.9950893 2.0058587h-14.00982141c-1.10185739 0-1.99508929-.8970601-1.99508929-2.0058587v-9.98828264c0-1.10780515.8926228-2.00585866 1.99508929-2.00585866zm0 1h-14.00982141c-.54871518 0-.99508929.44887827-.99508929 1.00585866v9.98828264c0 .5572961.44630695 1.0058587.99508929 1.0058587h14.00982141c.5487152 0 .9950893-.4488783.9950893-1.0058587v-9.98828264c0-.55729607-.446307-1.00585866-.9950893-1.00585866zm-.0049107 2.55749512v1.44250488l-7 4-7-4v-1.44250488l7 4z" fill-rule="evenodd"/></symbol><symbol id="icon-error" viewBox="0 0 18 18"><path d="m9 0c4.9705627 0 9 4.02943725 9 9 0 4.9705627-4.0294373 9-9 9-4.97056275 0-9-4.0294373-9-9 0-4.97056275 4.02943725-9 9-9zm2.8630343 4.71100931-2.8630343 2.86303426-2.86303426-2.86303426c-.39658757-.39658757-1.03281091-.39438847-1.4265779-.00062147-.39651227.39651226-.39348876 1.03246767.00062147 1.4265779l2.86303426 2.86303426-2.86303426 2.8630343c-.39658757.3965875-.39438847 1.0328109-.00062147 1.4265779.39651226.3965122 1.03246767.3934887 1.4265779-.0006215l2.86303426-2.8630343 2.8630343 2.8630343c.3965875.3965876 1.0328109.3943885 1.4265779.0006215.3965122-.3965123.3934887-1.0324677-.0006215-1.4265779l-2.8630343-2.8630343 2.8630343-2.86303426c.3965876-.39658757.3943885-1.03281091.0006215-1.4265779-.3965123-.39651227-1.0324677-.39348876-1.4265779.00062147z" fill-rule="evenodd"/></symbol><symbol id="icon-ethics" viewBox="0 0 18 18"><path d="m6.76384967 1.41421356.83301651-.8330165c.77492941-.77492941 2.03133823-.77492941 2.80626762 0l.8330165.8330165c.3750728.37507276.8837806.58578644 1.4142136.58578644h1.3496361c1.1045695 0 2 .8954305 2 2v1.34963611c0 .53043298.2107137 1.03914081.5857864 1.41421356l.8330165.83301651c.7749295.77492941.7749295 2.03133823 0 2.80626762l-.8330165.8330165c-.3750727.3750728-.5857864.8837806-.5857864 1.4142136v1.3496361c0 1.1045695-.8954305 2-2 2h-1.3496361c-.530433 0-1.0391408.2107137-1.4142136.5857864l-.8330165.8330165c-.77492939.7749295-2.03133821.7749295-2.80626762 0l-.83301651-.8330165c-.37507275-.3750727-.88378058-.5857864-1.41421356-.5857864h-1.34963611c-1.1045695 0-2-.8954305-2-2v-1.3496361c0-.530433-.21071368-1.0391408-.58578644-1.4142136l-.8330165-.8330165c-.77492941-.77492939-.77492941-2.03133821 0-2.80626762l.8330165-.83301651c.37507276-.37507275.58578644-.88378058.58578644-1.41421356v-1.34963611c0-1.1045695.8954305-2 2-2h1.34963611c.53043298 0 1.03914081-.21071368 1.41421356-.58578644zm-1.41421356 1.58578644h-1.34963611c-.55228475 0-1 .44771525-1 1v1.34963611c0 .79564947-.31607052 1.55871121-.87867966 2.12132034l-.8330165.83301651c-.38440512.38440512-.38440512 1.00764896 0 1.39205408l.8330165.83301646c.56260914.5626092.87867966 1.3256709.87867966 2.1213204v1.3496361c0 .5522847.44771525 1 1 1h1.34963611c.79564947 0 1.55871121.3160705 2.12132034.8786797l.83301651.8330165c.38440512.3844051 1.00764896.3844051 1.39205408 0l.83301646-.8330165c.5626092-.5626092 1.3256709-.8786797 2.1213204-.8786797h1.3496361c.5522847 0 1-.4477153 1-1v-1.3496361c0-.7956495.3160705-1.5587112.8786797-2.1213204l.8330165-.83301646c.3844051-.38440512.3844051-1.00764896 0-1.39205408l-.8330165-.83301651c-.5626092-.56260913-.8786797-1.32567087-.8786797-2.12132034v-1.34963611c0-.55228475-.4477153-1-1-1h-1.3496361c-.7956495 0-1.5587112-.31607052-2.1213204-.87867966l-.83301646-.8330165c-.38440512-.38440512-1.00764896-.38440512-1.39205408 0l-.83301651.8330165c-.56260913.56260914-1.32567087.87867966-2.12132034.87867966zm3.58698944 11.4960218c-.02081224.002155-.04199226.0030286-.06345763.002542-.98766446-.0223875-1.93408568-.3063547-2.75885125-.8155622-.23496767-.1450683-.30784554-.4531483-.16277726-.688116.14506827-.2349677.45314827-.3078455.68811595-.1627773.67447084.4164161 1.44758575.6483839 2.25617384.6667123.01759529.0003988.03495764.0017019.05204365.0038639.01713363-.0017748.03452416-.0026845.05212715-.0026845 2.4852814 0 4.5-2.0147186 4.5-4.5 0-1.04888973-.3593547-2.04134635-1.0074477-2.83787157-.1742817-.21419731-.1419238-.5291218.0722736-.70340353.2141973-.17428173.5291218-.14192375.7034035.07227357.7919032.97327203 1.2317706 2.18808682 1.2317706 3.46900153 0 3.0375661-2.4624339 5.5-5.5 5.5-.02146768 0-.04261937-.0013529-.06337445-.0039782zm1.57975095-10.78419583c.2654788.07599731.419084.35281842.3430867.61829728-.0759973.26547885-.3528185.419084-.6182973.3430867-.37560116-.10752146-.76586237-.16587951-1.15568824-.17249193-2.5587807-.00064534-4.58547766 2.00216524-4.58547766 4.49928198 0 .62691557.12797645 1.23496.37274865 1.7964426.11035133.2531347-.0053975.5477984-.25853224.6581497-.25313473.1103514-.54779841-.0053975-.65814974-.2585322-.29947131-.6869568-.45606667-1.43097603-.45606667-2.1960601 0-3.05211432 2.47714695-5.50006595 5.59399617-5.49921198.48576182.00815502.96289603.0795037 1.42238033.21103795zm-1.9766658 6.41091303 2.69835-2.94655317c.1788432-.21040373.4943901-.23598862.7047939-.05714545.2104037.17884318.2359886.49439014.0571454.70479387l-3.01637681 3.34277395c-.18039088.1999106-.48669547.2210637-.69285412.0478478l-1.93095347-1.62240047c-.21213845-.17678204-.24080048-.49206439-.06401844-.70420284.17678204-.21213844.49206439-.24080048.70420284-.06401844z" fill-rule="evenodd"/></symbol><symbol id="icon-expand"><path d="M7.498 11.918a.997.997 0 0 0-.003-1.411.995.995 0 0 0-1.412-.003l-4.102 4.102v-3.51A1 1 0 0 0 .98 10.09.992.992 0 0 0 0 11.092V17c0 .554.448 1.002 1.002 1.002h5.907c.554 0 1.002-.45 1.002-1.003 0-.539-.45-.978-1.006-.978h-3.51zm3.005-5.835a.997.997 0 0 0 .003 1.412.995.995 0 0 0 1.411.003l4.103-4.103v3.51a1 1 0 0 0 1.001 1.006A.992.992 0 0 0 18 6.91V1.002A1 1 0 0 0 17 0h-5.907a1.003 1.003 0 0 0-1.002 1.003c0 .539.45.978 1.006.978h3.51z" fill-rule="evenodd"/></symbol><symbol id="icon-explore" viewBox="0 0 18 18"><path d="m9 17c4.418278 0 8-3.581722 8-8s-3.581722-8-8-8-8 3.581722-8 8 3.581722 8 8 8zm0 1c-4.97056275 0-9-4.0294373-9-9 0-4.97056275 4.02943725-9 9-9 4.9705627 0 9 4.02943725 9 9 0 4.9705627-4.0294373 9-9 9zm0-2.5c-.27614237 0-.5-.2238576-.5-.5s.22385763-.5.5-.5c2.969509 0 5.400504-2.3575119 5.497023-5.31714844.0090007-.27599565.2400359-.49243782.5160315-.48343711.2759957.0090007.4924378.2400359.4834371.51603155-.114093 3.4985237-2.9869632 6.284554-6.4964916 6.284554zm-.29090657-12.99359748c.27587424-.01216621.50937715.20161139.52154336.47748563.01216621.27587423-.20161139.50937715-.47748563.52154336-2.93195733.12930094-5.25315116 2.54886451-5.25315116 5.49456849 0 .27614237-.22385763.5-.5.5s-.5-.22385763-.5-.5c0-3.48142406 2.74307146-6.34074398 6.20909343-6.49359748zm1.13784138 8.04763908-1.2004882-1.20048821c-.19526215-.19526215-.19526215-.51184463 0-.70710678s.51184463-.19526215.70710678 0l1.20048821 1.2004882 1.6006509-4.00162734-4.50670359 1.80268144-1.80268144 4.50670359zm4.10281269-6.50378907-2.6692597 6.67314927c-.1016411.2541026-.3029834.4554449-.557086.557086l-6.67314927 2.6692597 2.66925969-6.67314926c.10164107-.25410266.30298336-.45544495.55708602-.55708602z" fill-rule="evenodd"/></symbol><symbol id="icon-filter" viewBox="0 0 16 16"><path d="m14.9738641 0c.5667192 0 1.0261359.4477136 1.0261359 1 0 .24221858-.0902161.47620768-.2538899.65849851l-5.6938314 6.34147206v5.49997973c0 .3147562-.1520673.6111434-.4104543.7999971l-2.05227171 1.4999945c-.45337535.3313696-1.09655869.2418269-1.4365902-.1999993-.13321514-.1730955-.20522717-.3836284-.20522717-.5999978v-6.99997423l-5.69383133-6.34147206c-.3731872-.41563511-.32996891-1.0473954.09653074-1.41107611.18705584-.15950448.42716133-.2474224.67571519-.2474224zm-5.9218641 8.5h-2.105v6.491l.01238459.0070843.02053271.0015705.01955278-.0070558 2.0532976-1.4990996zm-8.02585008-7.5-.01564945.00240169 5.83249953 6.49759831h2.313l5.836-6.499z"/></symbol><symbol id="icon-home" viewBox="0 0 18 18"><path d="m9 5-6 6v5h4v-4h4v4h4v-5zm7 6.5857864v4.4142136c0 .5522847-.4477153 1-1 1h-5v-4h-2v4h-5c-.55228475 0-1-.4477153-1-1v-4.4142136c-.25592232 0-.51184464-.097631-.70710678-.2928932l-.58578644-.5857864c-.39052429-.3905243-.39052429-1.02368929 0-1.41421358l8.29289322-8.29289322 8.2928932 8.29289322c.3905243.39052429.3905243 1.02368928 0 1.41421358l-.5857864.5857864c-.1952622.1952622-.4511845.2928932-.7071068.2928932zm-7-9.17157284-7.58578644 7.58578644.58578644.5857864 7-6.99999996 7 6.99999996.5857864-.5857864z" fill-rule="evenodd"/></symbol><symbol id="icon-image" viewBox="0 0 18 18"><path d="m10.0046024 0c.5497429 0 1.3179837.32258606 1.707238.71184039l4.5763192 4.57631922c.3931386.39313859.7118404 1.16760135.7118404 1.71431368v8.98899651c0 1.1092806-.8945138 2.0085302-1.9940603 2.0085302h-12.01187942c-1.10128908 0-1.99406028-.8926228-1.99406028-1.9950893v-14.00982141c0-1.10185739.88743329-1.99508929 1.99961498-1.99508929zm-3.49645283 10.1752453-3.89407257 6.7495552c.11705545.048464.24538859.0751995.37998328.0751995h10.60290092l-2.4329715-4.2154691-1.57494129 2.7288098zm8.49779013 6.8247547c.5463747 0 .9940603-.4506622.9940603-1.0085302v-8.98899651c0-.28393444-.2150684-.80332809-.4189472-1.0072069l-4.5763192-4.57631922c-.2038461-.20384606-.718603-.41894717-1.0001312-.41894717h-7.00498742c-.55709576 0-.99961498.44271433-.99961498.99508929v13.98991071l4.50814957-7.81026689 3.08089884 5.33809539 1.57494129-2.7288097 3.5875735 6.2159812zm-3.0059397-11c1.1045695 0 2 .8954305 2 2s-.8954305 2-2 2-2-.8954305-2-2 .8954305-2 2-2zm0 1c-.5522847 0-1 .44771525-1 1s.4477153 1 1 1 1-.44771525 1-1-.4477153-1-1-1z" fill-rule="evenodd"/></symbol><symbol id="icon-info" viewBox="0 0 18 18"><path d="m9 0c4.9705627 0 9 4.02943725 9 9 0 4.9705627-4.0294373 9-9 9-4.97056275 0-9-4.0294373-9-9 0-4.97056275 4.02943725-9 9-9zm0 7h-1.5l-.11662113.00672773c-.49733868.05776511-.88337887.48043643-.88337887.99327227 0 .47338693.32893365.86994729.77070917.97358929l.1126697.01968298.11662113.00672773h.5v3h-.5l-.11662113.0067277c-.42082504.0488782-.76196299.3590206-.85696816.7639815l-.01968298.1126697-.00672773.1166211.00672773.1166211c.04887817.4208251.35902055.761963.76398144.8569682l.1126697.019683.11662113.0067277h3l.1166211-.0067277c.4973387-.0577651.8833789-.4804365.8833789-.9932723 0-.4733869-.3289337-.8699473-.7707092-.9735893l-.1126697-.019683-.1166211-.0067277h-.5v-4l-.00672773-.11662113c-.04887817-.42082504-.35902055-.76196299-.76398144-.85696816l-.1126697-.01968298zm0-3.25c-.69035594 0-1.25.55964406-1.25 1.25s.55964406 1.25 1.25 1.25 1.25-.55964406 1.25-1.25-.55964406-1.25-1.25-1.25z" fill-rule="evenodd"/></symbol><symbol id="icon-institution" viewBox="0 0 18 18"><path d="m7 16.9998189v-2.0003623h4v2.0003623h2v-3.0005434h-8v3.0005434zm-3-10.00181122h-1.52632364c-.27614237 0-.5-.22389817-.5-.50009056 0-.13995446.05863589-.27350497.16166338-.36820841l1.23156713-1.13206327h-2.36690687v12.00217346h3v-2.0003623h-3v-1.0001811h3v-1.0001811h1v-4.00072448h-1zm10 0v2.00036224h-1v4.00072448h1v1.0001811h3v1.0001811h-3v2.0003623h3v-12.00217346h-2.3695309l1.2315671 1.13206327c.2033191.186892.2166633.50325042.0298051.70660631-.0946863.10304615-.2282126.16169266-.3681417.16169266zm3-3.00054336c.5522847 0 1 .44779634 1 1.00018112v13.00235456h-18v-13.00235456c0-.55238478.44771525-1.00018112 1-1.00018112h3.45499992l4.20535144-3.86558216c.19129876-.17584288.48537447-.17584288.67667324 0l4.2053514 3.86558216zm-4 3.00054336h-8v1.00018112h8zm-2 6.00108672h1v-4.00072448h-1zm-1 0v-4.00072448h-2v4.00072448zm-3 0v-4.00072448h-1v4.00072448zm8-4.00072448c.5522847 0 1 .44779634 1 1.00018112v2.00036226h-2v-2.00036226c0-.55238478.4477153-1.00018112 1-1.00018112zm-12 0c.55228475 0 1 .44779634 1 1.00018112v2.00036226h-2v-2.00036226c0-.55238478.44771525-1.00018112 1-1.00018112zm5.99868798-7.81907007-5.24205601 4.81852671h10.48411203zm.00131202 3.81834559c-.55228475 0-1-.44779634-1-1.00018112s.44771525-1.00018112 1-1.00018112 1 .44779634 1 1.00018112-.44771525 1.00018112-1 1.00018112zm-1 11.00199236v1.0001811h2v-1.0001811z" fill-rule="evenodd"/></symbol><symbol id="icon-location" viewBox="0 0 18 18"><path d="m9.39521328 16.2688008c.79596342-.7770119 1.59208152-1.6299956 2.33285652-2.5295081 1.4020032-1.7024324 2.4323601-3.3624519 2.9354918-4.871847.2228715-.66861448.3364384-1.29323246.3364384-1.8674457 0-3.3137085-2.6862915-6-6-6-3.36356866 0-6 2.60156856-6 6 0 .57421324.11356691 1.19883122.3364384 1.8674457.50313169 1.5093951 1.53348863 3.1694146 2.93549184 4.871847.74077492.8995125 1.53689309 1.7524962 2.33285648 2.5295081.13694479.1336842.26895677.2602648.39521328.3793207.12625651-.1190559.25826849-.2456365.39521328-.3793207zm-.39521328 1.7311992s-7-6-7-11c0-4 3.13400675-7 7-7 3.8659932 0 7 3.13400675 7 7 0 5-7 11-7 11zm0-8c-1.65685425 0-3-1.34314575-3-3s1.34314575-3 3-3c1.6568542 0 3 1.34314575 3 3s-1.3431458 3-3 3zm0-1c1.1045695 0 2-.8954305 2-2s-.8954305-2-2-2-2 .8954305-2 2 .8954305 2 2 2z" fill-rule="evenodd"/></symbol><symbol id="icon-minus" viewBox="0 0 16 16"><path d="m2.00087166 7h11.99825664c.5527662 0 1.0008717.44386482 1.0008717 1 0 .55228475-.4446309 1-1.0008717 1h-11.99825664c-.55276616 0-1.00087166-.44386482-1.00087166-1 0-.55228475.44463086-1 1.00087166-1z" fill-rule="evenodd"/></symbol><symbol id="icon-newsletter" viewBox="0 0 18 18"><path d="m9 11.8482489 2-1.1428571v-1.7053918h-4v1.7053918zm-3-1.7142857v-2.1339632h6v2.1339632l3-1.71428574v-6.41967746h-12v6.41967746zm10-5.3839632 1.5299989.95624934c.2923814.18273835.4700011.50320827.4700011.8479983v8.44575236c0 1.1045695-.8954305 2-2 2h-14c-1.1045695 0-2-.8954305-2-2v-8.44575236c0-.34479003.1776197-.66525995.47000106-.8479983l1.52999894-.95624934v-2.75c0-.55228475.44771525-1 1-1h12c.5522847 0 1 .44771525 1 1zm0 1.17924764v3.07075236l-7 4-7-4v-3.07075236l-1 .625v8.44575236c0 .5522847.44771525 1 1 1h14c.5522847 0 1-.4477153 1-1v-8.44575236zm-10-1.92924764h6v1h-6zm-1 2h8v1h-8z" fill-rule="evenodd"/></symbol><symbol id="icon-orcid" viewBox="0 0 18 18"><path d="m9 1c4.418278 0 8 3.581722 8 8s-3.581722 8-8 8-8-3.581722-8-8 3.581722-8 8-8zm-2.90107518 5.2732337h-1.41865256v7.1712107h1.41865256zm4.55867178.02508949h-2.99247027v7.14612121h2.91062487c.7673039 0 1.4476365-.1483432 2.0410182-.445034s1.0511995-.7152915 1.3734671-1.2558144c.3222677-.540523.4833991-1.1603247.4833991-1.85942385 0-.68545815-.1602789-1.30270225-.4808414-1.85175082-.3205625-.54904856-.7707074-.97532211-1.3504481-1.27883343-.5797408-.30351132-1.2413173-.45526471-1.9847495-.45526471zm-.1892674 1.07933542c.7877654 0 1.4143875.22336734 1.8798852.67010873.4654977.44674138.698243 1.05546001.698243 1.82617415 0 .74343221-.2310402 1.34447791-.6931277 1.80315511-.4620874.4586773-1.0750688.6880124-1.8389625.6880124h-1.46810075v-4.98745039zm-5.08652545-3.71099194c-.21825533 0-.410525.08444276-.57681478.25333081-.16628977.16888806-.24943341.36245684-.24943341.58071218 0 .22345188.08314364.41961891.24943341.58850696.16628978.16888806.35855945.25333082.57681478.25333082.233845 0 .43390938-.08314364.60019916-.24943342.16628978-.16628977.24943342-.36375592.24943342-.59240436 0-.233845-.08314364-.43131115-.24943342-.59240437s-.36635416-.24163862-.60019916-.24163862z" fill-rule="evenodd"/></symbol><symbol id="icon-plus" viewBox="0 0 16 16"><path d="m2.00087166 7h4.99912834v-4.99912834c0-.55276616.44386482-1.00087166 1-1.00087166.55228475 0 1 .44463086 1 1.00087166v4.99912834h4.9991283c.5527662 0 1.0008717.44386482 1.0008717 1 0 .55228475-.4446309 1-1.0008717 1h-4.9991283v4.9991283c0 .5527662-.44386482 1.0008717-1 1.0008717-.55228475 0-1-.4446309-1-1.0008717v-4.9991283h-4.99912834c-.55276616 0-1.00087166-.44386482-1.00087166-1 0-.55228475.44463086-1 1.00087166-1z" fill-rule="evenodd"/></symbol><symbol id="icon-print" viewBox="0 0 18 18"><path d="m16.0049107 5h-14.00982141c-.54941618 0-.99508929.4467783-.99508929.99961498v6.00077002c0 .5570958.44271433.999615.99508929.999615h1.00491071v-3h12v3h1.0049107c.5494162 0 .9950893-.4467783.9950893-.999615v-6.00077002c0-.55709576-.4427143-.99961498-.9950893-.99961498zm-2.0049107-1v-2.00208688c0-.54777062-.4519464-.99791312-1.0085302-.99791312h-7.9829396c-.55661731 0-1.0085302.44910695-1.0085302.99791312v2.00208688zm1 10v2.0018986c0 1.103521-.9019504 1.9981014-2.0085302 1.9981014h-7.9829396c-1.1092806 0-2.0085302-.8867064-2.0085302-1.9981014v-2.0018986h-1.00491071c-1.10185739 0-1.99508929-.8874333-1.99508929-1.999615v-6.00077002c0-1.10435686.8926228-1.99961498 1.99508929-1.99961498h1.00491071v-2.00208688c0-1.10341695.90195036-1.99791312 2.0085302-1.99791312h7.9829396c1.1092806 0 2.0085302.89826062 2.0085302 1.99791312v2.00208688h1.0049107c1.1018574 0 1.9950893.88743329 1.9950893 1.99961498v6.00077002c0 1.1043569-.8926228 1.999615-1.9950893 1.999615zm-1-3h-10v5.0018986c0 .5546075.44702548.9981014 1.0085302.9981014h7.9829396c.5565964 0 1.0085302-.4491701 1.0085302-.9981014zm-9 1h8v1h-8zm0 2h5v1h-5zm9-5c-.5522847 0-1-.44771525-1-1s.4477153-1 1-1 1 .44771525 1 1-.4477153 1-1 1z" fill-rule="evenodd"/></symbol><symbol id="icon-search" viewBox="0 0 22 22"><path d="M21.697 20.261a1.028 1.028 0 01.01 1.448 1.034 1.034 0 01-1.448-.01l-4.267-4.267A9.812 9.811 0 010 9.812a9.812 9.811 0 1117.43 6.182zM9.812 18.222A8.41 8.41 0 109.81 1.403a8.41 8.41 0 000 16.82z" fill-rule="evenodd"/></symbol><symbol id="icon-social-facebook" viewBox="0 0 24 24"><path d="m6.00368507 20c-1.10660471 0-2.00368507-.8945138-2.00368507-1.9940603v-12.01187942c0-1.10128908.89451376-1.99406028 1.99406028-1.99406028h12.01187942c1.1012891 0 1.9940603.89451376 1.9940603 1.99406028v12.01187942c0 1.1012891-.88679 1.9940603-2.0032184 1.9940603h-2.9570132v-6.1960818h2.0797387l.3114113-2.414723h-2.39115v-1.54164807c0-.69911803.1941355-1.1755439 1.1966615-1.1755439l1.2786739-.00055875v-2.15974763l-.2339477-.02492088c-.3441234-.03134957-.9500153-.07025255-1.6293054-.07025255-1.8435726 0-3.1057323 1.12531866-3.1057323 3.19187953v1.78079225h-2.0850778v2.414723h2.0850778v6.1960818z" fill-rule="evenodd"/></symbol><symbol id="icon-social-twitter" viewBox="0 0 24 24"><path d="m18.8767135 6.87445248c.7638174-.46908424 1.351611-1.21167363 1.6250764-2.09636345-.7135248.43394112-1.50406.74870123-2.3464594.91677702-.6695189-.73342162-1.6297913-1.19486605-2.6922204-1.19486605-2.0399895 0-3.6933555 1.69603749-3.6933555 3.78628909 0 .29642457.0314329.58673729.0942985.8617704-3.06469922-.15890802-5.78835241-1.66547825-7.60988389-3.9574208-.3174714.56076194-.49978171 1.21167363-.49978171 1.90536824 0 1.31404706.65223085 2.47224203 1.64236444 3.15218497-.60350999-.0198635-1.17401554-.1925232-1.67222562-.47366811v.04583885c0 1.83355406 1.27302891 3.36609966 2.96411421 3.71294696-.31118484.0886217-.63651445.1329326-.97441718.1329326-.2357461 0-.47149219-.0229194-.69466516-.0672303.47149219 1.5065703 1.83253297 2.6036468 3.44975116 2.632678-1.2651707 1.0160946-2.85724264 1.6196394-4.5891906 1.6196394-.29861172 0-.59093688-.0152796-.88011875-.0504227 1.63450624 1.0726291 3.57548241 1.6990934 5.66104951 1.6990934 6.79263079 0 10.50641749-5.7711113 10.50641749-10.7751859l-.0094298-.48894775c.7229547-.53478659 1.3516109-1.20250585 1.8419628-1.96190282-.6632323.30100846-1.3751855.50422736-2.1217148.59590507z" fill-rule="evenodd"/></symbol><symbol id="icon-social-youtube" viewBox="0 0 24 24"><path d="m10.1415 14.3973208-.0005625-5.19318431 4.863375 2.60554491zm9.963-7.92753362c-.6845625-.73643756-1.4518125-.73990314-1.803375-.7826454-2.518875-.18714178-6.2971875-.18714178-6.2971875-.18714178-.007875 0-3.7861875 0-6.3050625.18714178-.352125.04274226-1.1188125.04620784-1.8039375.7826454-.5394375.56084773-.7149375 1.8344515-.7149375 1.8344515s-.18 1.49597903-.18 2.99138042v1.4024082c0 1.495979.18 2.9913804.18 2.9913804s.1755 1.2736038.7149375 1.8344515c.685125.7364376 1.5845625.7133337 1.9850625.7901542 1.44.1420891 6.12.1859866 6.12.1859866s3.78225-.005776 6.301125-.1929178c.3515625-.0433198 1.1188125-.0467854 1.803375-.783223.5394375-.5608477.7155-1.8344515.7155-1.8344515s.18-1.4954014.18-2.9913804v-1.4024082c0-1.49540139-.18-2.99138042-.18-2.99138042s-.1760625-1.27360377-.7155-1.8344515z" fill-rule="evenodd"/></symbol><symbol id="icon-subject-medicine" viewBox="0 0 18 18"><path d="m12.5 8h-6.5c-1.65685425 0-3 1.34314575-3 3v1c0 1.6568542 1.34314575 3 3 3h1v-2h-.5c-.82842712 0-1.5-.6715729-1.5-1.5s.67157288-1.5 1.5-1.5h1.5 2 1 2c1.6568542 0 3-1.34314575 3-3v-1c0-1.65685425-1.3431458-3-3-3h-2v2h1.5c.8284271 0 1.5.67157288 1.5 1.5s-.6715729 1.5-1.5 1.5zm-5.5-1v-1h-3.5c-1.38071187 0-2.5-1.11928813-2.5-2.5s1.11928813-2.5 2.5-2.5h1.02786405c.46573528 0 .92507448.10843528 1.34164078.31671843l1.13382424.56691212c.06026365-1.05041141.93116291-1.88363055 1.99667093-1.88363055 1.1045695 0 2 .8954305 2 2h2c2.209139 0 4 1.790861 4 4v1c0 2.209139-1.790861 4-4 4h-2v1h2c1.1045695 0 2 .8954305 2 2s-.8954305 2-2 2h-2c0 1.1045695-.8954305 2-2 2s-2-.8954305-2-2h-1c-2.209139 0-4-1.790861-4-4v-1c0-2.209139 1.790861-4 4-4zm0-2v-2.05652691c-.14564246-.03538148-.28733393-.08714006-.42229124-.15461871l-1.15541752-.57770876c-.27771087-.13885544-.583937-.21114562-.89442719-.21114562h-1.02786405c-.82842712 0-1.5.67157288-1.5 1.5s.67157288 1.5 1.5 1.5zm4 1v1h1.5c.2761424 0 .5-.22385763.5-.5s-.2238576-.5-.5-.5zm-1 1v-5c0-.55228475-.44771525-1-1-1s-1 .44771525-1 1v5zm-2 4v5c0 .5522847.44771525 1 1 1s1-.4477153 1-1v-5zm3 2v2h2c.5522847 0 1-.4477153 1-1s-.4477153-1-1-1zm-4-1v-1h-.5c-.27614237 0-.5.2238576-.5.5s.22385763.5.5.5zm-3.5-9h1c.27614237 0 .5.22385763.5.5s-.22385763.5-.5.5h-1c-.27614237 0-.5-.22385763-.5-.5s.22385763-.5.5-.5z" fill-rule="evenodd"/></symbol><symbol id="icon-success" viewBox="0 0 18 18"><path d="m9 0c4.9705627 0 9 4.02943725 9 9 0 4.9705627-4.0294373 9-9 9-4.97056275 0-9-4.0294373-9-9 0-4.97056275 4.02943725-9 9-9zm3.4860198 4.98163161-4.71802968 5.50657859-2.62834168-2.02300024c-.42862421-.36730544-1.06564993-.30775346-1.42283677.13301307-.35718685.44076653-.29927542 1.0958383.12934879 1.46314377l3.40735508 2.7323063c.42215801.3385221 1.03700951.2798252 1.38749189-.1324571l5.38450527-6.33394549c.3613513-.43716226.3096573-1.09278382-.115462-1.46437175-.4251192-.37158792-1.0626796-.31842941-1.4240309.11873285z" fill-rule="evenodd"/></symbol><symbol id="icon-table" viewBox="0 0 18 18"><path d="m16.0049107 2c1.1018574 0 1.9950893.89706013 1.9950893 2.00585866v9.98828264c0 1.1078052-.8926228 2.0058587-1.9950893 2.0058587l-4.0059107-.001.001.001h-1l-.001-.001h-5l.001.001h-1l-.001-.001-3.00391071.001c-1.10185739 0-1.99508929-.8970601-1.99508929-2.0058587v-9.98828264c0-1.10780515.8926228-2.00585866 1.99508929-2.00585866zm-11.0059107 5h-3.999v6.9941413c0 .5572961.44630695 1.0058587.99508929 1.0058587h3.00391071zm6 0h-5v8h5zm5.0059107-4h-4.0059107v3h5.001v1h-5.001v7.999l4.0059107.001c.5487152 0 .9950893-.4488783.9950893-1.0058587v-9.98828264c0-.55729607-.446307-1.00585866-.9950893-1.00585866zm-12.5049107 9c.27614237 0 .5.2238576.5.5s-.22385763.5-.5.5h-1c-.27614237 0-.5-.2238576-.5-.5s.22385763-.5.5-.5zm12 0c.2761424 0 .5.2238576.5.5s-.2238576.5-.5.5h-2c-.2761424 0-.5-.2238576-.5-.5s.2238576-.5.5-.5zm-6 0c.27614237 0 .5.2238576.5.5s-.22385763.5-.5.5h-2c-.27614237 0-.5-.2238576-.5-.5s.22385763-.5.5-.5zm-6-2c.27614237 0 .5.2238576.5.5s-.22385763.5-.5.5h-1c-.27614237 0-.5-.2238576-.5-.5s.22385763-.5.5-.5zm12 0c.2761424 0 .5.2238576.5.5s-.2238576.5-.5.5h-2c-.2761424 0-.5-.2238576-.5-.5s.2238576-.5.5-.5zm-6 0c.27614237 0 .5.2238576.5.5s-.22385763.5-.5.5h-2c-.27614237 0-.5-.2238576-.5-.5s.22385763-.5.5-.5zm-6-2c.27614237 0 .5.22385763.5.5s-.22385763.5-.5.5h-1c-.27614237 0-.5-.22385763-.5-.5s.22385763-.5.5-.5zm12 0c.2761424 0 .5.22385763.5.5s-.2238576.5-.5.5h-2c-.2761424 0-.5-.22385763-.5-.5s.2238576-.5.5-.5zm-6 0c.27614237 0 .5.22385763.5.5s-.22385763.5-.5.5h-2c-.27614237 0-.5-.22385763-.5-.5s.22385763-.5.5-.5zm1.499-5h-5v3h5zm-6 0h-3.00391071c-.54871518 0-.99508929.44887827-.99508929 1.00585866v1.99414134h3.999z" fill-rule="evenodd"/></symbol><symbol id="icon-tick-circle" viewBox="0 0 24 24"><path d="m12 2c5.5228475 0 10 4.4771525 10 10s-4.4771525 10-10 10-10-4.4771525-10-10 4.4771525-10 10-10zm0 1c-4.97056275 0-9 4.02943725-9 9 0 4.9705627 4.02943725 9 9 9 4.9705627 0 9-4.0294373 9-9 0-4.97056275-4.0294373-9-9-9zm4.2199868 5.36606669c.3613514-.43716226.9989118-.49032077 1.424031-.11873285s.4768133 1.02720949.115462 1.46437175l-6.093335 6.94397871c-.3622945.4128716-.9897871.4562317-1.4054264.0971157l-3.89719065-3.3672071c-.42862421-.3673054-.48653564-1.0223772-.1293488-1.4631437s.99421256-.5003185 1.42283677-.1330131l3.11097438 2.6987741z" fill-rule="evenodd"/></symbol><symbol id="icon-tick" viewBox="0 0 16 16"><path d="m6.76799012 9.21106946-3.1109744-2.58349728c-.42862421-.35161617-1.06564993-.29460792-1.42283677.12733148s-.29927541 1.04903009.1293488 1.40064626l3.91576307 3.23873978c.41034319.3393961 1.01467563.2976897 1.37450571-.0948578l6.10568327-6.660841c.3613513-.41848908.3096572-1.04610608-.115462-1.4018218-.4251192-.35571573-1.0626796-.30482786-1.424031.11366122z" fill-rule="evenodd"/></symbol><symbol id="icon-update" viewBox="0 0 18 18"><path d="m1 13v1c0 .5522847.44771525 1 1 1h14c.5522847 0 1-.4477153 1-1v-1h-1v-10h-14v10zm16-1h1v2c0 1.1045695-.8954305 2-2 2h-14c-1.1045695 0-2-.8954305-2-2v-2h1v-9c0-.55228475.44771525-1 1-1h14c.5522847 0 1 .44771525 1 1zm-1 0v1h-4.5857864l-1 1h-2.82842716l-1-1h-4.58578644v-1h5l1 1h2l1-1zm-13-8h12v7h-12zm1 1v5h10v-5zm1 1h4v1h-4zm0 2h4v1h-4z" fill-rule="evenodd"/></symbol><symbol id="icon-upload" viewBox="0 0 18 18"><path d="m10.0046024 0c.5497429 0 1.3179837.32258606 1.707238.71184039l4.5763192 4.57631922c.3931386.39313859.7118404 1.16760135.7118404 1.71431368v8.98899651c0 1.1092806-.8945138 2.0085302-1.9940603 2.0085302h-12.01187942c-1.10128908 0-1.99406028-.8926228-1.99406028-1.9950893v-14.00982141c0-1.10185739.88743329-1.99508929 1.99961498-1.99508929zm0 1h-7.00498742c-.55709576 0-.99961498.44271433-.99961498.99508929v14.00982141c0 .5500396.44491393.9950893.99406028.9950893h12.01187942c.5463747 0 .9940603-.4506622.9940603-1.0085302v-8.98899651c0-.28393444-.2150684-.80332809-.4189472-1.0072069l-4.5763192-4.57631922c-.2038461-.20384606-.718603-.41894717-1.0001312-.41894717zm-1.85576936 4.14572769c.19483374-.19483375.51177826-.19377714.70556874.00001334l2.59099082 2.59099079c.1948411.19484112.1904373.51514474.0027906.70279143-.1932998.19329987-.5046517.19237083-.7001856-.00692852l-1.74638687-1.7800176v6.14827687c0 .2717771-.23193359.492096-.5.492096-.27614237 0-.5-.216372-.5-.492096v-6.14827641l-1.74627892 1.77990922c-.1933927.1971171-.51252214.19455839-.70016883.0069117-.19329987-.19329988-.19100584-.50899493.00277731-.70277808z" fill-rule="evenodd"/></symbol><symbol id="icon-video" viewBox="0 0 18 18"><path d="m16.0049107 2c1.1018574 0 1.9950893.89706013 1.9950893 2.00585866v9.98828264c0 1.1078052-.8926228 2.0058587-1.9950893 2.0058587h-14.00982141c-1.10185739 0-1.99508929-.8970601-1.99508929-2.0058587v-9.98828264c0-1.10780515.8926228-2.00585866 1.99508929-2.00585866zm0 1h-14.00982141c-.54871518 0-.99508929.44887827-.99508929 1.00585866v9.98828264c0 .5572961.44630695 1.0058587.99508929 1.0058587h14.00982141c.5487152 0 .9950893-.4488783.9950893-1.0058587v-9.98828264c0-.55729607-.446307-1.00585866-.9950893-1.00585866zm-8.30912922 2.24944486 4.60460462 2.73982242c.9365543.55726659.9290753 1.46522435 0 2.01804082l-4.60460462 2.7398224c-.93655425.5572666-1.69578148.1645632-1.69578148-.8937585v-5.71016863c0-1.05087579.76670616-1.446575 1.69578148-.89375851zm-.67492769.96085624v5.5750128c0 .2995102-.10753745.2442517.16578928.0847713l4.58452283-2.67497259c.3050619-.17799716.3051624-.21655446 0-.39461026l-4.58452283-2.67497264c-.26630747-.15538481-.16578928-.20699944-.16578928.08477139z" fill-rule="evenodd"/></symbol><symbol id="icon-warning" viewBox="0 0 18 18"><path d="m9 11.75c.69035594 0 1.25.5596441 1.25 1.25s-.55964406 1.25-1.25 1.25-1.25-.5596441-1.25-1.25.55964406-1.25 1.25-1.25zm.41320045-7.75c.55228475 0 1.00000005.44771525 1.00000005 1l-.0034543.08304548-.3333333 4c-.043191.51829212-.47645714.91695452-.99654578.91695452h-.15973424c-.52008864 0-.95335475-.3986624-.99654576-.91695452l-.33333333-4c-.04586475-.55037702.36312325-1.03372649.91350028-1.07959124l.04148683-.00259031zm-.41320045 14c-4.97056275 0-9-4.0294373-9-9 0-4.97056275 4.02943725-9 9-9 4.9705627 0 9 4.02943725 9 9 0 4.9705627-4.0294373 9-9 9z" fill-rule="evenodd"/></symbol><symbol id="icon-checklist-banner" viewBox="0 0 56.69 56.69"><path style="fill:none" d="M0 0h56.69v56.69H0z"/><clipPath id="b"><use xlink:href="#a" style="overflow:visible"/></clipPath><path d="M21.14 34.46c0-6.77 5.48-12.26 12.24-12.26s12.24 5.49 12.24 12.26-5.48 12.26-12.24 12.26c-6.76-.01-12.24-5.49-12.24-12.26zm19.33 10.66 10.23 9.22s1.21 1.09 2.3-.12l2.09-2.32s1.09-1.21-.12-2.3l-10.23-9.22m-19.29-5.92c0-4.38 3.55-7.94 7.93-7.94s7.93 3.55 7.93 7.94c0 4.38-3.55 7.94-7.93 7.94-4.38-.01-7.93-3.56-7.93-7.94zm17.58 12.99 4.14-4.81" style="clip-path:url(#b);fill:none;stroke:#01324b;stroke-width:2;stroke-linecap:round"/><path d="M8.26 9.75H28.6M8.26 15.98H28.6m-20.34 6.2h12.5m14.42-5.2V4.86s0-2.93-2.93-2.93H4.13s-2.93 0-2.93 2.93v37.57s0 2.93 2.93 2.93h15.01M8.26 9.75H28.6M8.26 15.98H28.6m-20.34 6.2h12.5" style="clip-path:url(#b);fill:none;stroke:#01324b;stroke-width:2;stroke-linecap:round;stroke-linejoin:round"/></symbol><symbol id="icon-chevron-down" viewBox="0 0 16 16"><path d="m5.58578644 3-3.29289322-3.29289322c-.39052429-.39052429-.39052429-1.02368927 0-1.41421356s1.02368927-.39052429 1.41421356 0l4 4c.39052429.39052429.39052429 1.02368927 0 1.41421356l-4 4c-.39052429.39052429-1.02368927.39052429-1.41421356 0s-.39052429-1.02368927 0-1.41421356z" fill-rule="evenodd" transform="matrix(0 1 -1 0 11 1)"/></symbol><symbol id="icon-eds-i-arrow-right-medium" viewBox="0 0 24 24"><path d="m12.728 3.293 7.98 7.99a.996.996 0 0 1 .281.561l.011.157c0 .32-.15.605-.384.788l-7.908 7.918a1 1 0 0 1-1.416-1.414L17.576 13H4a1 1 0 0 1 0-2h13.598l-6.285-6.293a1 1 0 0 1-.082-1.32l.083-.095a1 1 0 0 1 1.414.001Z"/></symbol><symbol id="icon-eds-i-chevron-down-medium" viewBox="0 0 16 16"><path d="m2.00087166 7h4.99912834v-4.99912834c0-.55276616.44386482-1.00087166 1-1.00087166.55228475 0 1 .44463086 1 1.00087166v4.99912834h4.9991283c.5527662 0 1.0008717.44386482 1.0008717 1 0 .55228475-.4446309 1-1.0008717 1h-4.9991283v4.9991283c0 .5527662-.44386482 1.0008717-1 1.0008717-.55228475 0-1-.4446309-1-1.0008717v-4.9991283h-4.99912834c-.55276616 0-1.00087166-.44386482-1.00087166-1 0-.55228475.44463086-1 1.00087166-1z" fill-rule="evenodd"/></symbol><symbol id="icon-eds-i-chevron-down-small" viewBox="0 0 16 16"><path d="M13.692 5.278a1 1 0 0 1 .03 1.414L9.103 11.51a1.491 1.491 0 0 1-2.188.019L2.278 6.692a1 1 0 0 1 1.444-1.384L8 9.771l4.278-4.463a1 1 0 0 1 1.318-.111l.096.081Z"/></symbol><symbol id="icon-eds-i-chevron-right-medium" viewBox="0 0 10 10"><path d="m5.96738168 4.70639573 2.39518594-2.41447274c.37913917-.38219212.98637524-.38972225 1.35419292-.01894278.37750606.38054586.37784436.99719163-.00013556 1.37821513l-4.03074001 4.06319683c-.37758093.38062133-.98937525.38100976-1.367372-.00003075l-4.03091981-4.06337806c-.37759778-.38063832-.38381821-.99150444-.01600053-1.3622839.37750607-.38054587.98772445-.38240057 1.37006824.00302197l2.39538588 2.4146743.96295325.98624457z" fill-rule="evenodd" transform="matrix(0 -1 1 0 0 10)"/></symbol><symbol id="icon-eds-i-chevron-right-small" viewBox="0 0 10 10"><path d="m5.96738168 4.70639573 2.39518594-2.41447274c.37913917-.38219212.98637524-.38972225 1.35419292-.01894278.37750606.38054586.37784436.99719163-.00013556 1.37821513l-4.03074001 4.06319683c-.37758093.38062133-.98937525.38100976-1.367372-.00003075l-4.03091981-4.06337806c-.37759778-.38063832-.38381821-.99150444-.01600053-1.3622839.37750607-.38054587.98772445-.38240057 1.37006824.00302197l2.39538588 2.4146743.96295325.98624457z" fill-rule="evenodd" transform="matrix(0 -1 1 0 0 10)"/></symbol><symbol id="icon-eds-i-chevron-up-medium" viewBox="0 0 16 16"><path d="m2.00087166 7h11.99825664c.5527662 0 1.0008717.44386482 1.0008717 1 0 .55228475-.4446309 1-1.0008717 1h-11.99825664c-.55276616 0-1.00087166-.44386482-1.00087166-1 0-.55228475.44463086-1 1.00087166-1z" fill-rule="evenodd"/></symbol><symbol id="icon-eds-i-close-medium" viewBox="0 0 16 16"><path d="m2.29679575 12.2772478c-.39658757.3965876-.39438847 1.0328109-.00062148 1.4265779.39651227.3965123 1.03246768.3934888 1.42657791-.0006214l4.27724782-4.27724787 4.2772478 4.27724787c.3965876.3965875 1.0328109.3943884 1.4265779.0006214.3965123-.3965122.3934888-1.0324677-.0006214-1.4265779l-4.27724787-4.2772478 4.27724787-4.27724782c.3965875-.39658757.3943884-1.03281091.0006214-1.42657791-.3965122-.39651226-1.0324677-.39348875-1.4265779.00062148l-4.2772478 4.27724782-4.27724782-4.27724782c-.39658757-.39658757-1.03281091-.39438847-1.42657791-.00062148-.39651226.39651227-.39348875 1.03246768.00062148 1.42657791l4.27724782 4.27724782z" fill-rule="evenodd"/></symbol><symbol id="icon-eds-i-download-medium" viewBox="0 0 16 16"><path d="m12.9975267 12.999368c.5467123 0 1.0024733.4478567 1.0024733 1.000316 0 .5563109-.4488226 1.000316-1.0024733 1.000316h-9.99505341c-.54671233 0-1.00247329-.4478567-1.00247329-1.000316 0-.5563109.44882258-1.000316 1.00247329-1.000316zm-4.9975267-11.999368c.55228475 0 1 .44497754 1 .99589209v6.80214418l2.4816273-2.48241149c.3928222-.39294628 1.0219732-.4006883 1.4030652-.01947579.3911302.39125371.3914806 1.02525073-.0001404 1.41699553l-4.17620792 4.17752758c-.39120769.3913313-1.02508144.3917306-1.41671995-.0000316l-4.17639421-4.17771394c-.39122513-.39134876-.39767006-1.01940351-.01657797-1.40061601.39113012-.39125372 1.02337105-.3931606 1.41951349.00310701l2.48183446 2.48261871v-6.80214418c0-.55001601.44386482-.99589209 1-.99589209z" fill-rule="evenodd"/></symbol><symbol id="icon-eds-i-info-filled-medium" viewBox="0 0 18 18"><path d="m9 0c4.9705627 0 9 4.02943725 9 9 0 4.9705627-4.0294373 9-9 9-4.97056275 0-9-4.0294373-9-9 0-4.97056275 4.02943725-9 9-9zm0 7h-1.5l-.11662113.00672773c-.49733868.05776511-.88337887.48043643-.88337887.99327227 0 .47338693.32893365.86994729.77070917.97358929l.1126697.01968298.11662113.00672773h.5v3h-.5l-.11662113.0067277c-.42082504.0488782-.76196299.3590206-.85696816.7639815l-.01968298.1126697-.00672773.1166211.00672773.1166211c.04887817.4208251.35902055.761963.76398144.8569682l.1126697.019683.11662113.0067277h3l.1166211-.0067277c.4973387-.0577651.8833789-.4804365.8833789-.9932723 0-.4733869-.3289337-.8699473-.7707092-.9735893l-.1126697-.019683-.1166211-.0067277h-.5v-4l-.00672773-.11662113c-.04887817-.42082504-.35902055-.76196299-.76398144-.85696816l-.1126697-.01968298zm0-3.25c-.69035594 0-1.25.55964406-1.25 1.25s.55964406 1.25 1.25 1.25 1.25-.55964406 1.25-1.25-.55964406-1.25-1.25-1.25z" fill-rule="evenodd"/></symbol><symbol id="icon-eds-i-mail-medium" viewBox="0 0 24 24"><path d="m19.462 0c1.413 0 2.538 1.184 2.538 2.619v12.762c0 1.435-1.125 2.619-2.538 2.619h-16.924c-1.413 0-2.538-1.184-2.538-2.619v-12.762c0-1.435 1.125-2.619 2.538-2.619zm.538 5.158-7.378 6.258a2.549 2.549 0 0 1 -3.253-.008l-7.369-6.248v10.222c0 .353.253.619.538.619h16.924c.285 0 .538-.266.538-.619zm-.538-3.158h-16.924c-.264 0-.5.228-.534.542l8.65 7.334c.2.165.492.165.684.007l8.656-7.342-.001-.025c-.044-.3-.274-.516-.531-.516z"/></symbol><symbol id="icon-eds-i-menu-medium" viewBox="0 0 24 24"><path d="M21 4a1 1 0 0 1 0 2H3a1 1 0 1 1 0-2h18Zm-4 7a1 1 0 0 1 0 2H3a1 1 0 0 1 0-2h14Zm4 7a1 1 0 0 1 0 2H3a1 1 0 0 1 0-2h18Z"/></symbol><symbol id="icon-eds-i-search-medium" viewBox="0 0 24 24"><path d="M11 1c5.523 0 10 4.477 10 10 0 2.4-.846 4.604-2.256 6.328l3.963 3.965a1 1 0 0 1-1.414 1.414l-3.965-3.963A9.959 9.959 0 0 1 11 21C5.477 21 1 16.523 1 11S5.477 1 11 1Zm0 2a8 8 0 1 0 0 16 8 8 0 0 0 0-16Z"/></symbol><symbol id="icon-eds-i-user-single-medium" viewBox="0 0 24 24"><path d="M12 1a5 5 0 1 1 0 10 5 5 0 0 1 0-10Zm0 2a3 3 0 1 0 0 6 3 3 0 0 0 0-6Zm-.406 9.008a8.965 8.965 0 0 1 6.596 2.494A9.161 9.161 0 0 1 21 21.025V22a1 1 0 0 1-1 1H4a1 1 0 0 1-1-1v-.985c.05-4.825 3.815-8.777 8.594-9.007Zm.39 1.992-.299.006c-3.63.175-6.518 3.127-6.678 6.775L5 21h13.998l-.009-.268a7.157 7.157 0 0 0-1.97-4.573l-.214-.213A6.967 6.967 0 0 0 11.984 14Z"/></symbol><symbol id="icon-eds-i-warning-filled-medium" viewBox="0 0 18 18"><path d="m9 11.75c.69035594 0 1.25.5596441 1.25 1.25s-.55964406 1.25-1.25 1.25-1.25-.5596441-1.25-1.25.55964406-1.25 1.25-1.25zm.41320045-7.75c.55228475 0 1.00000005.44771525 1.00000005 1l-.0034543.08304548-.3333333 4c-.043191.51829212-.47645714.91695452-.99654578.91695452h-.15973424c-.52008864 0-.95335475-.3986624-.99654576-.91695452l-.33333333-4c-.04586475-.55037702.36312325-1.03372649.91350028-1.07959124l.04148683-.00259031zm-.41320045 14c-4.97056275 0-9-4.0294373-9-9 0-4.97056275 4.02943725-9 9-9 4.9705627 0 9 4.02943725 9 9 0 4.9705627-4.0294373 9-9 9z" fill-rule="evenodd"/></symbol><symbol id="icon-expand-image" viewBox="0 0 18 18"><path d="m7.49754099 11.9178212c.38955542-.3895554.38761957-1.0207846-.00290473-1.4113089-.39324695-.3932469-1.02238878-.3918247-1.41130883-.0029047l-4.10273549 4.1027355.00055454-3.5103985c.00008852-.5603185-.44832171-1.006032-1.00155062-1.0059446-.53903074.0000852-.97857527.4487442-.97866268 1.0021075l-.00093318 5.9072465c-.00008751.553948.44841131 1.001882 1.00174994 1.0017946l5.906983-.0009331c.5539233-.0000875 1.00197907-.4486389 1.00206646-1.0018679.00008515-.5390307-.45026621-.9784332-1.00588841-.9783454l-3.51010549.0005545zm3.00571741-5.83449376c-.3895554.38955541-.3876196 1.02078454.0029047 1.41130883.393247.39324696 1.0223888.39182478 1.4113089.00290473l4.1027355-4.10273549-.0005546 3.5103985c-.0000885.56031852.4483217 1.006032 1.0015506 1.00594461.5390308-.00008516.9785753-.44874418.9786627-1.00210749l.0009332-5.9072465c.0000875-.553948-.4484113-1.00188204-1.0017499-1.00179463l-5.906983.00093313c-.5539233.00008751-1.0019791.44863892-1.0020665 1.00186784-.0000852.53903074.4502662.97843325 1.0058884.97834547l3.5101055-.00055449z" fill-rule="evenodd"/></symbol><symbol id="icon-github" viewBox="0 0 100 100"><path fill-rule="evenodd" clip-rule="evenodd" d="M48.854 0C21.839 0 0 22 0 49.217c0 21.756 13.993 40.172 33.405 46.69 2.427.49 3.316-1.059 3.316-2.362 0-1.141-.08-5.052-.08-9.127-13.59 2.934-16.42-5.867-16.42-5.867-2.184-5.704-5.42-7.17-5.42-7.17-4.448-3.015.324-3.015.324-3.015 4.934.326 7.523 5.052 7.523 5.052 4.367 7.496 11.404 5.378 14.235 4.074.404-3.178 1.699-5.378 3.074-6.6-10.839-1.141-22.243-5.378-22.243-24.283 0-5.378 1.94-9.778 5.014-13.2-.485-1.222-2.184-6.275.486-13.038 0 0 4.125-1.304 13.426 5.052a46.97 46.97 0 0 1 12.214-1.63c4.125 0 8.33.571 12.213 1.63 9.302-6.356 13.427-5.052 13.427-5.052 2.67 6.763.97 11.816.485 13.038 3.155 3.422 5.015 7.822 5.015 13.2 0 18.905-11.404 23.06-22.324 24.283 1.78 1.548 3.316 4.481 3.316 9.126 0 6.6-.08 11.897-.08 13.526 0 1.304.89 2.853 3.316 2.364 19.412-6.52 33.405-24.935 33.405-46.691C97.707 22 75.788 0 48.854 0z"/></symbol><symbol id="icon-springer-arrow-left"><path d="M15 7a1 1 0 000-2H3.385l2.482-2.482a.994.994 0 00.02-1.403 1.001 1.001 0 00-1.417 0L.294 5.292a1.001 1.001 0 000 1.416l4.176 4.177a.991.991 0 001.4.016 1 1 0 00-.003-1.42L3.385 7H15z"/></symbol><symbol id="icon-springer-arrow-right"><path d="M1 7a1 1 0 010-2h11.615l-2.482-2.482a.994.994 0 01-.02-1.403 1.001 1.001 0 011.417 0l4.176 4.177a1.001 1.001 0 010 1.416l-4.176 4.177a.991.991 0 01-1.4.016 1 1 0 01.003-1.42L12.615 7H1z"/></symbol><symbol id="icon-submit-open" viewBox="0 0 16 17"><path d="M12 0c1.10457 0 2 .895431 2 2v5c0 .276142-.223858.5-.5.5S13 7.276142 13 7V2c0-.512836-.38604-.935507-.883379-.993272L12 1H6v3c0 1.10457-.89543 2-2 2H1v8c0 .512836.38604.935507.883379.993272L2 15h6.5c.276142 0 .5.223858.5.5s-.223858.5-.5.5H2c-1.104569 0-2-.89543-2-2V5.828427c0-.530433.210714-1.039141.585786-1.414213L4.414214.585786C4.789286.210714 5.297994 0 5.828427 0H12Zm3.41 11.14c.250899.250899.250274.659726 0 .91-.242954.242954-.649606.245216-.9-.01l-1.863671-1.900337.001043 5.869492c0 .356992-.289839.637138-.647372.637138-.347077 0-.647371-.285256-.647371-.637138l-.001043-5.869492L9.5 12.04c-.253166.258042-.649726.260274-.9.01-.242954-.242954-.252269-.657731 0-.91l2.942184-2.951303c.250908-.250909.66127-.252277.91353-.000017L15.41 11.14ZM5 1.413 1.413 5H4c.552285 0 1-.447715 1-1V1.413ZM11 3c.276142 0 .5.223858.5.5s-.223858.5-.5.5H7.5c-.276142 0-.5-.223858-.5-.5s.223858-.5.5-.5H11Zm0 2c.276142 0 .5.223858.5.5s-.223858.5-.5.5H7.5c-.276142 0-.5-.223858-.5-.5s.223858-.5.5-.5H11Z" fill-rule="nonzero"/></symbol></svg> </div> </body> </html>