CINXE.COM

<!DOCTYPE html> <html class="no-js" lang="en" dir="ltr" prefix="content: http://purl.org/rss/1.0/modules/content/ dc: http://purl.org/dc/terms/ foaf: http://xmlns.com/foaf/0.1/ rdfs: http://www.w3.org/2000/01/rdf-schema# sioc: http://rdfs.org/sioc/ns# sioct: http://rdfs.org/sioc/types# skos: http://www.w3.org/2004/02/skos/core# xsd: http://www.w3.org/2001/XMLSchema#"> <head> <title>Addressing Rigor in Scientific Studies | National Institutes of Health (NIH)</title> <meta charset="utf-8" /> <script type="text/javascript"> //<![CDATA[ var usasearch_config = { siteHandle:"nih" }; var script = document.createElement("script"); script.type = "text/javascript"; script.src = "//search.usa.gov/javascripts/remote.loader.js"; document.getElementsByTagName("head")[0].appendChild(script); //]]> </script><link rel="shortcut icon" href="https://www.nih.gov/favicon.ico" type="image/vnd.microsoft.icon" /> <meta name="google-site-verification" content="rjIoO4vW0AYgltSYvxM1L_CzGIkq28kHf9gHIASKx8s" /> <link rel="profile" href="http://www.w3.org/1999/xhtml/vocab" /> <meta http-equiv="cleartype" content="on" /> <link rel="apple-touch-icon" href="https://www.nih.gov/sites/all/themes/nih/apple-touch-icon.png" /> <meta name="description" content="Not all scientific studies are created equal. Communicators need to understand how to assess whether studies have robust findings and how to communicate this to audiences." /> <meta name="keywords" content="Scientific rigor, reproducibility, transparency, research integrity, sample size estimation, blinding, masking, randomization, data handling, CONSORT statement, ARRIVE guidelines, PRISMA statement, Devon C. Crawford" /> <meta name="robots" content="follow, index" /> <meta name="viewport" content="width=device-width, initial-scale=1.0" /> <meta name="generator" content="Drupal 7 (http://drupal.org)" /> <link rel="canonical" href="https://www.nih.gov/about-nih/what-we-do/science-health-public-trust/perspectives/addressing-rigor-scientific-studies" /> <link rel="shortlink" href="https://www.nih.gov/node/242796" /> <meta http-equiv="content-language" content="EN" /> <meta property="og:site_name" content="National Institutes of Health (NIH)" /> <meta property="og:type" content="article" /> <meta property="og:title" content="Addressing Rigor in Scientific Studies" /> <meta property="og:url" content="https://www.nih.gov/about-nih/what-we-do/science-health-public-trust/perspectives/addressing-rigor-scientific-studies" /> <meta property="og:description" content="Not all scientific studies are created equal. Communicators need to understand how to assess whether studies have robust findings and how to communicate this to audiences." /> <meta property="og:updated_time" content="2023-09-06T12:51:29-04:00" /> <meta property="og:image" content="https://www.nih.gov/sites/default/files/about-nih/public-trust/20230731-crawford.jpg" /> <meta name="twitter:card" content="summary" /> <meta name="twitter:title" content="Addressing Rigor in Scientific Studies" /> <meta name="twitter:url" content="https://www.nih.gov/about-nih/what-we-do/science-health-public-trust/perspectives/addressing-rigor-scientific-studies" /> <meta name="twitter:description" content="Not all scientific studies are created equal. Communicators need to understand how to assess whether studies have robust findings and how to communicate this to audiences." /> <meta name="twitter:image" content="https://www.nih.gov/sites/default/files/about-nih/public-trust/20230731-crawford.jpg" /> <meta property="article:published_time" content="2023-07-31T11:56:06-04:00" /> <meta property="article:modified_time" content="2023-09-06T12:51:29-04:00" /> <meta name="owner" content="OD" /> <meta name="published_date" content="2023-07-31T11:56:06-04:00" /> <meta name="last_revision" content="2023-09-06T12:51:29-04:00" /> <link type="text/css" rel="stylesheet" href="https://www.nih.gov/sites/default/files/css/css_PfvPCWanzn7FjDwZqwAHhPoOxYKwkQP-RZSK1pFFG0s.css" media="all" /> <link type="text/css" rel="stylesheet" href="https://www.nih.gov/sites/default/files/css/css__mIWP9gcStq74isjZVoAAsiih8VXwJoN75Nyw_BXrj0.css" media="screen" /> <style media="print">#sb-container{position:relative;}#sb-overlay{display:none;}#sb-wrapper{position:relative;top:0;left:0;}#sb-loading{display:none;} </style> <link type="text/css" rel="stylesheet" href="https://www.nih.gov/sites/default/files/css/css_jENQOQqZCRFGZK__DTefxYiXR2RUf03k1ugj3_Vg7vk.css" media="all" /> <link type="text/css" rel="stylesheet" href="//cdn.jsdelivr.net/qtip2/3.0.3/jquery.qtip.min.css" media="all" /> <link type="text/css" rel="stylesheet" href="https://www.nih.gov/sites/default/files/css/css_3yt-b0M1HNJsvKsXwYjh5TEtX2lJGAvJvtmVAfORYZs.css" media="all" /> <link type="text/css" rel="stylesheet" href="//fonts.googleapis.com/css?family=Droid+Sans:700,regular|Droid+Serif:italic,regular&subset=latin" media="all" /> <link type="text/css" rel="stylesheet" href="https://www.nih.gov/sites/default/files/css/css_E4VlyxyL3JMcT6dslWLGb0uQdoI2bh_Xj0N65V-bcWo.css" media="all" /> <link type="text/css" rel="stylesheet" href="https://www.nih.gov/sites/default/files/css/css_5rDMv37-JxnNA70cRr_1jlAjVs56DCb8EN4XB2QLl0c.css" media="all" />  <link type="text/css" rel="stylesheet" href="https://www.nih.gov/sites/default/files/css/css_KviiBystpJcQq4MplcQZvCFxnu9V763f5bPq_AKh4xM.css" media="all" /> <script src="https://www.nih.gov/sites/default/files/js/js_mQAv54bz2JVXcwS5QWlz0fVY9Os67zeu9bgNqtlVLPY.js"></script> <script src="//code.jquery.com/jquery-2.2.4.min.js"></script> <script>window.jQuery || document.write("<script src='/sites/all/modules/contrib/jquery_update/replace/jquery/2.2/jquery.min.js'>\x3C/script>")</script> <script>jQuery.migrateMute=true;jQuery.migrateTrace=false;</script> <script src="//code.jquery.com/jquery-migrate-1.4.1.min.js"></script> <script>window.jQuery && window.jQuery.migrateWarnings || document.write("<script src='/sites/all/modules/contrib/jquery_update/replace/jquery-migrate/1.4.1/jquery-migrate.min.js'>\x3C/script>")</script> <script src="https://www.nih.gov/sites/default/files/js/js_9sGTPBGcOGZjLiSLGrYEnuUf8VdvLmL_DEZNNNbnHXk.js"></script> <script src="https://www.nih.gov/sites/default/files/js/js_hNnF7a_sbk6nZ2-mOamHzalYND7HBHDLRInsaU93HDk.js"></script> <script>document.createElement( "picture" );</script> <script src="https://www.nih.gov/sites/default/files/js/js_gHk2gWJ_Qw_jU2qRiUmSl7d8oly1Cx7lQFrqcp3RXcI.js"></script> <script src="//cdn.jsdelivr.net/qtip2/3.0.3/jquery.qtip.min.js"></script> <script defer="defer" src="//script.crazyegg.com/pages/scripts/0011/2616.js"></script> <script src="https://www.nih.gov/sites/default/files/js/js_M6Y5x-IfJsaieldoN31qdrl5lv71Z8XEmxbJr6rsD1Q.js"></script> <script> Shadowbox.path = "/sites/all/libraries/shadowbox/"; </script> <script>jQuery.extend(Drupal.settings, {"basePath":"\/","pathPrefix":"","setHasJsCookie":0,"instances":"{\u0022default\u0022:{\u0022content\u0022:{\u0022text\u0022:\u0022\u0022},\u0022style\u0022:{\u0022tip\u0022:false,\u0022classes\u0022:\u0022\u0022},\u0022position\u0022:{\u0022at\u0022:\u0022bottom right\u0022,\u0022adjust\u0022:{\u0022method\u0022:\u0022\u0022},\u0022my\u0022:\u0022top left\u0022,\u0022viewport\u0022:false},\u0022show\u0022:{\u0022event\u0022:\u0022mouseenter \u0022},\u0022hide\u0022:{\u0022event\u0022:\u0022mouseleave \u0022}}}","qtipDebug":"{\u0022leaveElement\u0022:0}","shadowbox":{"animate":true,"animateFade":true,"animSequence":"wh","auto_enable_all_images":1,"auto_gallery":1,"autoplayMovies":true,"continuous":false,"counterLimit":10,"counterType":"default","displayCounter":true,"displayNav":true,"enableKeys":true,"fadeDuration":0.34999999999999998,"handleOversize":"resize","handleUnsupported":"link","initialHeight":160,"initialWidth":320,"language":"en","modal":false,"overlayColor":"#000","overlayOpacity":0.80000000000000004,"resizeDuration":0.55000000000000004,"showMovieControls":true,"slideshowDelay":0,"viewportPadding":20,"useSizzle":false},"picture":{"mappings":[["Sidebar media mappings","sidebar_media_mappings"],["Floated media mappings","floated_media_mappings"],["Not Set","not_set"]],"label":"Image size (required)"},"extlink":{"extTarget":"_blank","extClass":"ext","extLabel":"(link is external)","extImgClass":0,"extIconPlacement":0,"extSubdomains":1,"extExclude":"(.\\.nih\\.gov.)|(.\\.cancer\\.gov.)|(.\\.genome\\.gov.)|(.\\.drugabuse\\.gov.)|(locatorplus\\.gov.)|(.\\.govdelivery\\.com.)|(.\\.hatchbuck\\.com.)|(nihseniorhealth\\.gov.)|(clinicaltrials\\.gov.)|(search\\.usa\\.gov\\\/search\\?affiliate\\=hip.)|(search\\.gov\\\/search\\?affiliate\\=hip.)|(drugabuse\\.gov.)|(medlineplus\\.gov.)|(collegedrinkingprevention\\.gov.)|(smokefree\\.gov.)|(arpa-h\\.gov.)|(nih\\.sharepoint\\.com)","extInclude":"(.\\.pagefreezer\\.com.)","extCssExclude":".media, .emergency-message, .slideshow-main, .bean-home-slideshow-2l-block","extCssExplicit":"","extAlert":"_blank","extAlertText":"You are leaving the NIH website.\r\n\r\nThis external link provides additional information that is consistent with the intended purpose of this site. NIH cannot attest to the accuracy of a non-federal site.\r\n\r\nLinking to a non-federal site does not constitute an endorsement by NIH or any of its employees of the sponsors or the information and products presented on the site. You will be subject to the destination site\u2019s privacy policy when you follow the link.","mailtoClass":"mailto","mailtoLabel":"(link sends e-mail)","extUseFontAwesome":0}});</script> <script type="text/javascript"> // ForeSee Production Embed Script v2.01 // DO NOT MODIFY BELOW THIS LINE ***************************************** ;(function (g) { var d = document, am = d.createElement('script'), h = d.head || d.getElementsByTagName("head")[0], fsr = 'fsReady', aex = { "src": "//gateway.foresee.com/sites/nih-gov/production/gateway.min.js", "type": "text/javascript", "async": "true", "data-vendor": "fs", "data-role": "gateway" }; for (var attr in aex) { am.setAttribute(attr, aex[attr]); } h.appendChild(am); g[fsr] || (g[fsr] = function () { var aT = '__' + fsr + '_stk__'; g[aT] = g[aT] || []; g[aT].push(arguments); }); })(window); // DO NOT MODIFY ABOVE THIS LINE ***************************************** // Un-comment out the function below when you are ready to input your variable /*fsReady(function() { FSR.CPPS.set('name','value'); // use single quotes when passing a static-value FSR.CPPS.set('name2',somevariable); // don't use quotes for a dynamic value FSR.CPPS.set('name3',othervariable); // add as many CPPs as you like in this way });*/ </script> </head> <body class="html not-front not-logged-in page-node page-node- page-node-242796 node-type-nih-initiatives- section-about-nih"> <a href="#main-content" class="element-invisible element-focusable">Skip to main content</a> <noscript aria-hidden="true"><iframe src="https://www.googletagmanager.com/ns.html?id=GTM-5CBCNW2" height="0" width="0" style="display:none;visibility:hidden"></iframe></noscript> <div class="l-page has-two-sidebars has-sidebar-first has-sidebar-third"> <div class="l-page-content"> <header class="l-header" role="banner"> <div class="l-ribbon-wrapper"> <ul class="l-ribbon"> <li><span class="ribbon-hhs"></span><a href="https://www.hhs.gov">U.S. Department of Health & Human Services</a><span class="ribbon-right"></span></li> </ul> </div> <div class="l-banner-wrapper"> <div class="l-banner"> <div class="l-branding"> <a class="nih-logo icon-nih-logo" href="/" title="National Institutes of Health (NIH) - Turning Discovery into Health" rel="home"><img src="/sites/all/themes/nih/images/nih-logo-color.png" width="390" height="60" alt="National Institutes of Health (NIH) - Turning Discovery into Health" /></a> </div>  <form accept-charset="UTF-8" action="https://search.nih.gov/search" id="search" class="search-form site-search" method="get"> <div style="margin:0;padding:0;display:inline"><input name="utf8" type="hidden" value="✓" /></div> <input id="affiliate" name="affiliate" type="hidden" value="nih" /> <label for="query" class="hide">Search the NIH Website</label> <input autocomplete="on" class="usagov-search-autocomplete form-field form-field-site-search" id="query" name="query" type="text" placeholder="Search NIH" /><input type="submit" name="commit" value="Search" class="form-button form-button-site-search" /> </form> <ul class="banner-utilities"> <li><a href="/virtual-tour">Virtual Tour</a></li> <li><a href="https://ned.nih.gov/">Staff Directory</a></li> <li><a href="https://salud.nih.gov">En Español</a></li> </ul> </div> </div> </header> <div class="l-navigation-wrapper">  <h2 class="element-invisible">Site Menu</h2> <nav class="l-menu-wrapper"> <ul class="l-menu">  <li class="tab-wrapper tab-wrapper-home"> <div class="tab home"><a href="/"><span>Home</span></a></div> </li>  <li class="tab-wrapper tab-wrapper-healthinfo tab-wrapper-left"> <div class="tab"><a href="/health-information"><span>Health Information</span></a></div> <div class="megamenu"> <div class="megamenu-col megamenu-leftcol"> <ul class="sub-menu"> <li><a href="https://www.hhs.gov/programs/social-services/health-care-facilities/index.html">Health Care Providers & Facilities</a></li> <li><a href="/health-information/health-info-lines">Health Info Lines</a></li> <li><a href="https://www.healthcare.gov">HealthCare.gov</a></li> <li><a href="/research-training/science-education">Science Education Resources</a></li> <li><a href="/health-information/nih-clinical-research-trials-you">NIH Clinical Research Trials and You</a></li> <li><a href="/institutes-nih/nih-office-director/office-communications-public-liaison/clear-communication/talking-your-doctor">Talking to Your Doctor</a></li>  </ul> <p class="more-wrapper"><a class="button gold" href="/health-information" title="More Health Information">More »</a></p> </div> <div class="megamenu-col megamenu-rightcol"> <form id="hipsearch" accept-charset="UTF-8" action="https://search.nih.gov/search" method="get" class="search-form"> <p class="subhead"><label for="query2">Search Health Topics</label></p> <div class="input-wrapper"> <div style="margin:0;padding:0;display:inline"><input name="utf8" type="hidden" value="✓" /></div> <input id="affiliate" name="affiliate" type="hidden" value="hip" /> <input type="hidden" name="dc" value="940" /> <input autocomplete="on" maxlength="50" class="usagov-search-autocomplete form-field form-field-search" id="query2" name="query" type="text" /> <input name="commit" type="submit" value="Go" id="searchbutton" class="form-button" /> </div> </form> <p class="subhead">Quick Links</p> <ul class="sub-menu"> <li><a class="minibadge minibadge-medlineplus" href="https://medlineplus.gov/"><span class="minibadge-text">MedlinePlus Health Info</span></a></li> <li><a class="minibadge minibadge-newsinhealth" href="https://newsinhealth.nih.gov/"><span class="minibadge-text">NIH News in Health</span></a></li> <li><a class="minibadge minibadge-wellnesstoolkits" href="/health-information/your-healthiest-self-wellness-toolkits"><span class="minibadge-text">Wellness Toolkits</span></a></li> </ul> </div> </div> </li>  <li class="tab-wrapper tab-wrapper-grants tab-wrapper-left"> <div class="tab"><a href="/grants-funding"><span>Grants & Funding</span></a></div> <div class="megamenu"> <div class="megamenu-col megamenu-leftcol"> <ul> <li><a href="https://grants.nih.gov">Grants Home Page</a></li> <li><a href="https://grants.nih.gov/funding/index.htm">Find Funding</a></li> <li><a href="https://grants.nih.gov/grants/how-to-apply-application-guide/due-dates-and-submission-policies/due-dates.htm">Due Dates</a></li> <li><a href="https://grants.nih.gov/grants/how-to-apply-application-guide.html">How to Apply</a></li> <li><a href="https://grants.nih.gov/grants/about_grants.htm">About Grants</a></li> <li><a href="https://grants.nih.gov/policy/index.htm">Policy & Compliance</a></li> <li><a href="https://nexus.od.nih.gov/all/">Grants News/Blog</a></li> <li><a href="https://grants.nih.gov/funding/contracts.htm">Contracts</a></li> <li><a href="https://www.lrp.nih.gov/">Loan Repayment</a></li> </ul> <p class="more-wrapper"><a class="button gold" href="/grants-funding" title="More Grants & Funding Information">More »</a></p> </div> <div class="megamenu-col megamenu-rightcol"> <p class="subhead">Quick Links</p> <ul> <li><a class="minibadge minibadge-report" href="https://report.nih.gov/"><span class="minibadge-text">RePORT</span></a></li> <li><a class="minibadge minibadge-era" href="https://commons.era.nih.gov/commons/"><span class="minibadge-text">eRA Commons</span></a></li> <li><a class="minibadge minibadge-commonfund" href="https://commonfund.nih.gov/"><span class="minibadge-text">NIH Common Fund</span></a></li> </ul> </div> </div> </li>  <li class="tab-wrapper tab-wrapper-news tab-wrapper-left"> <div class="tab"><a href="/news-events"><span>News & Events</span></a></div> <div class="megamenu"> <div class="megamenu-col megamenu-leftcol"> <ul> <li><a href="/news-events/news-releases">News Releases</a></li> <li><a href="/news-events/digital-media-kits">Digital Media Kits</a></li> <li><a href="/news-events/media-resources">Media Resources</a></li> <li><a href="/news-events/media-contacts">Media Contacts</a></li> <li><a href="/news-events/images-b-roll">Images and B-roll</a></li> <li><a href="/news-events/events">Events</a></li> <li><a href="/news-events/social-media-outreach">Social Media</a></li> </ul> <p class="more-wrapper"><a class="button gold" href="/news-events" title="More News & Events">More »</a></p> </div> <div class="megamenu-col megamenu-rightcol"> <p class="subhead">Quick Links</p> <ul> <li><a class="minibadge minibadge-newsinhealth" href="https://newsinhealth.nih.gov/"><span class="minibadge-text">NIH News in Health</span></a></li> <li><a class="minibadge minibadge-researchmatters" href="/news-events/nih-research-matters"><span class="minibadge-text">NIH Research Matters</span></a></li> <li><a class="minibadge minibadge-nihrecord" href="https://nihrecord.nih.gov/"><span class="minibadge-text">NIH Record</span></a></li> </ul> </div> </div> </li>  <li class="tab-wrapper tab-wrapper-research tab-wrapper-right"> <div class="tab"><a href="/research-training"><span>Research & Training</span></a></div> <div class="megamenu"> <div class="megamenu-col megamenu-leftcol"> <ul> <li><a href="/research-training/medical-research-initiatives">Medical Research Initiatives</a></li> <li><a href="/research-training/science-highlights">Science Highlights</a></li> <li><a href="/research-training/science-education">Science Education</a></li> <li><a href="/research-training/research-nih-labs-clinics">Research in NIH Labs & Clinics</a></li> <li><a href="/research-training/training-opportunities">Training Opportunities</a></li> <li><a href="/research-training/library-resources">Library Resources</a></li> <li><a href="/research-training/research-resources">Research Resources</a></li> <li><a href="/research-training/clinical-research-resources">Clinical Research Resources</a></li> <li><a href="/research-training/safety-regulation-guidance">Safety, Regulation and Guidance</a></li> </ul> <p class="more-wrapper"><a class="button gold" href="/research-training" title="More Research & Training">More »</a></p> </div> <div class="megamenu-col megamenu-rightcol"> <p class="subhead">Quick Links</p> <ul> <li><a class="minibadge minibadge-pubmed" href="https://pubmed.ncbi.nlm.nih.gov"><span class="minibadge-text">PubMed</span></a></li> <li><a class="minibadge minibadge-stemcell" href="https://stemcells.nih.gov"><span class="minibadge-text">Stem Cell Information</span></a></li> <li><a class="minibadge minibadge-oppnet" href="https://oppnet.nih.gov"><span class="minibadge-text">OppNet</span></a></li> <li><a class="minibadge minibadge-nidb" href="https://intramural.nih.gov"><span class="minibadge-text">NIDB</span></a></li> <li><a class="minibadge minibadge-nihblueprint" href="https://neuroscienceblueprint.nih.gov"><span class="minibadge-text">NIH Blueprint for Neuroscience Research</span></a></li> </ul> </div> </div> </li>  <li class="tab-wrapper tab-wrapper-institutes tab-wrapper-right"> <div class="tab"><a href="/institutes-nih"><span>Institutes at NIH</span></a></div> <div class="megamenu"> <div class="megamenu-col megamenu-leftcol"> <ul> <li><a href="/institutes-nih/list-institutes-centers">List of Institutes and Centers</a></li> <li><a href="/institutes-nih/nih-office-director">NIH Office of the Director</a></li> <li><a href="/institutes-nih/directors-nih-institutes-centers">Directors of NIH Institutes and Centers</a></li> <li><a href="/institutes-nih/nih-institute-center-contact-information">NIH Institute and Center Contact Information</a></li> </ul> <p class="more-wrapper"><a class="button gold" href="/institutes-nih" title="More About the Institutes at NIH">More »</a></p> </div> <div class="megamenu-col megamenu-rightcol"> <p class="subhead">Quick Links</p> <ul class="ic-acronymn-list"> <li><a href="https://www.cancer.gov" title="National Cancer Institute">NCI</a></li> <li><a href="https://nei.nih.gov" title="National Eye Institute">NEI</a></li> <li><a href="https://www.nhlbi.nih.gov" title="National Heart, Lung, and Blood Institute">NHLBI</a></li> <li><a href="https://www.genome.gov" title="National Human Genome Research Institute">NHGRI</a></li> <li><a href="https://www.nia.nih.gov" title="National Institute on Aging">NIA</a></li> <li><a href="https://www.niaaa.nih.gov" title="National Institute on Alcohol Abuse and Alcoholism">NIAAA</a></li> <li><a href="https://www.niaid.nih.gov" title="National Institute of Allergy and Infectious Diseases">NIAID</a></li> <li><a href="https://www.niams.nih.gov" title="National Institute of Arthritis and Musculoskeletal and Skin Diseases">NIAMS</a></li> <li><a href="https://www.nibib.nih.gov" title="National Institute of Biomedical Imaging and Bioengineering">NIBIB</a></li> <li><a href="https://www.nichd.nih.gov" title="Eunice Kennedy Shriver National Institute of Child Health and Human Development">NICHD</a></li> <li><a href="https://www.nidcd.nih.gov" title="National Institute on Deafness and Other Communication Disorders">NIDCD</a></li> <li><a href="https://www.nidcr.nih.gov" title="National Institute of Dental and Craniofacial Research">NIDCR</a></li> <li><a href="https://www.niddk.nih.gov" title="National Institute of Diabetes and Digestive and Kidney Diseases">NIDDK</a></li> <li><a href="https://nida.nih.gov" title="National Institute on Drug Abuse">NIDA</a></li> <li><a href="https://www.niehs.nih.gov" title="National Institute of Environmental Health Sciences">NIEHS</a></li> <li><a href="https://www.nigms.nih.gov" title="National Institute of General Medical Sciences">NIGMS</a></li> <li><a href="https://www.nimh.nih.gov" title="National Institute of Mental Health">NIMH</a></li> <li><a href="https://www.nimhd.nih.gov" title="National Institute on Minority Health and Health Disparities">NIMHD</a></li> <li><a href="https://www.ninds.nih.gov" title="National Institute of Neurological Disorders and Stroke">NINDS</a></li> <li><a href="https://www.ninr.nih.gov" title="National Institute of Nursing Research">NINR</a></li> <li><a href="https://www.nlm.nih.gov" title="National Library of Medicine">NLM</a></li> <li><a href="https://www.cc.nih.gov" title="NIH Clinical Center">CC</a></li> <li><a href="https://www.cit.nih.gov" title="Center for Information Technology">CIT</a></li> <li><a href="https://www.csr.nih.gov" title="Center for Scientific Review">CSR</a></li> <li><a href="https://www.fic.nih.gov" title="Fogarty International Center">FIC</a></li> <li><a href="https://ncats.nih.gov" title="National Center for Advancing Translational Sciences">NCATS</a></li> <li><a href="https://nccih.nih.gov" title="National Center for Complementary and Integrative Health">NCCIH</a></li> </ul> </div> </div> </li>  <li class="tab-wrapper tab-wrapper-about tab-wrapper-right"> <div class="tab"><a href="/about-nih"><span>About NIH</span></a></div> <div class="megamenu"> <div class="megamenu-col megamenu-leftcol"> <ul> <li><a href="/about-nih/who-we-are">Who We Are</a></li> <li><a href="/about-nih/what-we-do">What We Do</a></li> <li><a href="https://hr.nih.gov/jobs">Jobs at NIH</a></li> <li><a href="/about-nih/visitor-information">Visitor Information</a></li> <li><a href="/about-nih/frequently-asked-questions">Frequently Asked Questions</a></li> <li><a href="/about-nih/contact-us">Contact Us</a></li>  </ul> <p class="more-wrapper"><a class="button gold" href="/about-nih" title="More About NIH">More »</a></p> </div> <div class="megamenu-col megamenu-rightcol"> <p class="subhead">Quick Links</p> <ul> <li><a href="/about-nih/who-we-are/nih-director">The NIH Director</a></li> <li><a href="/virtual-tour">Take the Virtual Tour</a></li> <li><a href="/about-nih/what-we-do/nih-turning-discovery-into-health">NIH…Turning Discovery Into Health<sup>®</sup></a></li> <li><a href="/about-nih/what-we-do/impact-nih-research">Impact of NIH Research</a></li> <li><a href="/about-nih/what-we-do/science-health-public-trust">Science, Health, and Public Trust</a></li> </ul> </div> </div> </li> </ul> <div class="tab-bkgnd"></div> </nav>  </div> <div class="l-main-wrapper"> <div class="l-main" id="page-content"> <h2 class="element-invisible">You are here</h2><div class="breadcrumb"><a href="/">Home</a> » <a href="/about-nih">About NIH</a> » <a href="/about-nih/what-we-do">What We Do</a> » <a href="/about-nih/what-we-do/science-health-public-trust">Science, Health, and Public Trust</a></div> <div class="l-region l-region--section-header"> <div id="block-ds-extras-banner-nih-initiative-ds" class="block block--ds-extras sectionheader block--ds-extras-banner-nih-initiative-ds" id="block-ds-extras-banner-nih-initiative-ds" class="block block--ds-extras sectionheader block--ds-extras-banner-nih-initiative-ds"> <div class="content" class="block__content"> <h2><a href="/about-nih/what-we-do/science-health-public-trust">Science, Health, and Public Trust</a></h2> </div> </div> </div> <aside class="l-region l-region--sidebar-first"> <nav id="block-menu-block-44" role="navigation" class="block block--menu-block subnav block--menu-block-44"> <h3 class="block__title"><a href="/about-nih/what-we-do/science-health-public-trust" title="Science, Health, and Public Trust">Science, Health, and Public Trust</a></h3> <div class="menu-block-wrapper menu-block-44 menu-name-main-menu parent-mlid-16731 menu-level-1"> <ul class="menu"><li class="first leaf menu-mlid-16741"><a href="/about-nih/what-we-do/science-health-public-trust/about-science-health-public-trust">About Science, Health, and Public Trust</a></li> <li class="leaf menu-mlid-16746"><a href="/about-nih/what-we-do/science-health-public-trust/perspectives">Perspectives</a></li> <li class="leaf menu-mlid-16751"><a href="/about-nih/what-we-do/science-health-public-trust/tools">Tools</a></li> <li class="leaf menu-mlid-16756"><a href="/about-nih/what-we-do/science-health-public-trust/resources">Resources</a></li> <li class="last leaf menu-mlid-16761"><a href="mailto:sciencehealthandpublictrust@mail.nih.gov" title="">Submit a Suggestion</a></li> </ul></div> </nav> </aside> <div class="l-content-wrapper"> <div class="l-content" role="main"> <a id="main-content"></a> <div about="/about-nih/what-we-do/science-health-public-trust/perspectives/addressing-rigor-scientific-studies" typeof="sioc:Item foaf:Document" class="ds-1col node node--nih-initiatives- featured-media-floated view-mode-full node--full node--nih-initiatives---full clearfix"> <p class="publishdate"><span property="dc:date" datatype="xsd:dateTime" content="2023-08-10T12:00:00-04:00" class="date-display-single">August 10, 2023</span></p><h1>Addressing Rigor in Scientific Studies</h1><p class="subtitle">Devon C. Crawford, Ph.D., Program Director, Office of Research Quality, NIH’s National Institute of Neurological Disorders and Stroke</p><div class="featured-media"><div class="ds-1col file file-image file-image-jpeg view-mode-featured_media clearfix"> <picture title="Photograph of Devon C. Crawford, Ph.D.">  <source srcset="https://www.nih.gov/sites/default/files/styles/featured_media_breakpoint-large-extra/public/about-nih/public-trust/20230731-crawford.jpg?itok=JvnRldUj&timestamp=1690819172 1x" media="screen and (min-width: 1100px)" /> <source srcset="https://www.nih.gov/sites/default/files/styles/featured_media_breakpoint-large/public/about-nih/public-trust/20230731-crawford.jpg?itok=dIwHd9Ts&timestamp=1690819172 1x" media="screen and (min-width: 960px)" /> <source srcset="https://www.nih.gov/sites/default/files/styles/featured_media_breakpoint-medium/public/about-nih/public-trust/20230731-crawford.jpg?itok=nUJykihW&timestamp=1690819172 1x" media="screen and (min-width: 800px)" /> <source srcset="https://www.nih.gov/sites/default/files/styles/featured_media_breakpoint-medium-small/public/about-nih/public-trust/20230731-crawford.jpg?itok=xgZAs-wc&timestamp=1690819172 1x" media="screen and (min-width: 600px)" /> <source srcset="https://www.nih.gov/sites/default/files/styles/featured_media_breakpoint-small/public/about-nih/public-trust/20230731-crawford.jpg?itok=uTgWfCu8&timestamp=1690819172 1x" media="screen and (min-width: 480px)" /> <source srcset="https://www.nih.gov/sites/default/files/styles/featured_media__breakpoint-tiny_/public/about-nih/public-trust/20230731-crawford.jpg?itok=WRMRQk66&timestamp=1690819172 1x" media="screen and (min-width: 0)" />  <img src="https://www.nih.gov/sites/default/files/styles/featured_media_breakpoint-medium/public/about-nih/public-trust/20230731-crawford.jpg?itok=nUJykihW&timestamp=1690819172" alt="Photograph of Devon C. Crawford, Ph.D." title="Photograph of Devon C. Crawford, Ph.D." /> </picture><span class="caption">Devon C. Crawford, Ph.D.</span></div> </div><p>Science communication is rapidly evolving. The <a href="https://www.nih.gov/about-nih/what-we-do/science-health-public-trust/perspectives/science-health-public-trust/science-health-public-trust/making-effective-use-preprints-tips-communicators">growing use of preprints</a> and the sheer number of published studies make it increasingly difficult to determine which findings are worthy of attention. Not all scientific studies are created equal. Communicators need to discern which are reputable in order to know what to convey to their target audience. Inaccurate or untrustworthy information can have dire consequences, so it is important to understand how to assess whether studies have robust findings and how to communicate this to audiences.</p> <p>Science communicators need to describe the major conclusions from a study, along with its implications for future research and public health practice, without overstating the results. Science is a continual process of updating knowledge that is conditional on how the results were obtained; it is not a series of discovered “facts.” All scientific conclusions are subject to interpretation, and all have some degree of uncertainty. Responsible science communicators will report important details of a study: the number of subjects, species involved, techniques used, major outcomes, and caveats. But, even this level of reporting does not provide enough information to know how much to trust the results.</p> <p>As <a href="https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4058759/">NIH has been emphasizing</a> for <a href="https://www.nature.com/articles/nature11556">more than a decade</a>, the <a href="https://grants.nih.gov/policy/reproducibility/index.htm">rigor and transparency</a> of a study are key for gleaning the robustness of its results. This includes the design, implementation, analysis, and interpretation of experiments. If a study’s validity isn’t known, the rest is moot.</p> <p>How does one know if a study is rigorous? And how can this be communicated to broad audiences? A single person can’t keep up with all of the limitations of every scientific approach, and even savvy readers of original research articles need to beware of <a href="https://pubmed.ncbi.nlm.nih.gov/28892482/">mistaking “spin” for reasonable conclusions</a> within a given study. Fortunately, there are some generally agreed-upon <a href="https://www.ninds.nih.gov/funding/preparing-your-application/preparing-research-plan/rigorous-study-design-and-transparent-reporting">principles of rigorous research</a> that apply across fields and methods.</p> <p>Transparent publications follow established guidelines to ensure that important research practices are reported. These include the <a href="https://pubmed.ncbi.nlm.nih.gov/36511921/">CONSORT statement</a> for clinical trials, <a href="https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7360023/">ARRIVE guidelines</a> for animal studies, and <a href="https://systematicreviewsjournal.biomedcentral.com/articles/10.1186/s13643-021-01626-4">PRISMA statement</a> for systematic reviews. It is difficult to assess the rigor and robustness of studies that do not fully follow these guidelines. Yet, <a href="https://pubmed.ncbi.nlm.nih.gov/33196023/">many papers do not report important practices</a>.</p> <p>Given the limited adherence to the lengthy lists contained in these guidelines, NIH’s National Institute of Neurological Disorders and Stroke (NINDS) held a workshop in 2012 to identify ways to enhance rigor-related transparency. The result was a list of <a href="https://pubmed.ncbi.nlm.nih.gov/23060188/">four important research practices</a> that should be reported for every relevant research study, from basic research to clinical trials:</p> <ul class="spaced"> <li><strong>Sample size estimation</strong><strong>:</strong> how the sample size, such as the number of participants, was chosen before the study began, ideally via statistical extrapolation from prior studies.</li> <li><strong>Blinding/Masking:</strong> how experimenters ensured that none of those involved in the study knew which samples or groups received the intervention being tested.</li> <li><strong>Randomization:</strong> how experimenters selected groups for treatment or data analysis so that all had the same chance of undergoing an intervention.</li> <li><strong>Data handing:</strong> how experimenters planned in advance to manage missing data, outliers, what data to exclude from analysis, and when to stop data collection.</li> </ul> <p>Without these practices, studies are likely to be at high risk for unconscious biases that can lead to incorrect conclusions. Depending on the goals of the study, not all these items may be relevant. However, it is still important to report whether they were relevant and used for a particular study to make it clear which measures were taken to reduce bias. Other <a href="https://www.ninds.nih.gov/funding/preparing-your-application/preparing-research-plan/rigorous-study-design-and-transparent-reporting">important experimental design and planning elements</a> that should be considered include transparent reporting of outcome measures, using appropriate control groups, defining clear measures of uncertainty, and addressing study limitations such as possible confounding factors. If the study is exploratory, if the above practices are not reported or performed, or if other relevant reporting guidelines have not been followed, the results should be interpreted as preliminary or tentative, and any conclusions should be communicated accordingly.</p> <p>Importantly, each study must also be put within the appropriate context of the <a href="https://www.nih.gov/about-nih/what-we-do/science-health-public-trust/perspectives/explaining-how-research-works">wider scientific landscape</a>. Are the results consistent with previous studies? Were those previous studies rigorously performed and transparently reported? Is there a chance that these results were only published because of the exciting result and not because of the rigor of the methods? Could there be similar or more rigorous studies out there that were not published or publicized because they did not get this exciting outcome? In other words, how surprising was this result? The more surprising, the more robust the evidence needs to be to support it.</p> <p>Science communicators can’t be expected to assess all these questions themselves. Communicators should ask the study investigators and other experts in the field for support in verifying important design elements of a particular study. If information is missing, why hasn’t it been addressed? Without this information, the finding cannot be properly interpreted.</p> <p>Communicating science accurately and responsibly is a balance between engaging the audience and providing enough important details. NIH’s NINDS explicitly emphasizes that rigor and transparency “<a href="https://www.ninds.nih.gov/current-research/trans-agency-activities/ninds-office-research-quality">are essential to enable the scientific community, as well as the community at large, to assess the value of scientific findings</a>.” They shouldn’t be left out of communications about research. Addressing a study’s rigor, transparency, and robustness not only provides important information about reliability, but it also signals to the audience that context, and by extension the ongoing process, is central to science.</p> </div> <div class="lastupdated"> </div> </div> </div> <aside class="l-region l-region--sidebar-third"> <div id="block-bean-public-trust-subscribe" class="block block--bean block--bean-public-trust-subscribe" id="block-bean-public-trust-subscribe" class="block block--bean block--bean-public-trust-subscribe"> <div class="content" class="block__content"> <div about="/block/public-trust---subscribe" class="ds-1col entity entity-bean bean-basic-block view-mode-default clearfix"> <p><a class="button-link" href="https://public.govdelivery.com/accounts/USNIH/subscriber/new?pop=t&topic_id=USNIH_101" target="_blank">Subscribe</a></p></div> </div> </div> </aside> <div class="l-connect-wrapper"> <div class="l-connect"> <h2 class="element-invisible">Connect with Us</h2> <ul> <li><a class="contactus-monochrome" href="/about-nih/contact-us"><span class="icon"></span>Contact Us</a></li> <li><a class="twitter-monochrome" href="https://twitter.com/NIH"><span class="icon"></span>X</a></li> <li><a class="facebook-monochrome" href="https://www.facebook.com/nih.gov"><span class="icon"></span>Facebook</a></li> <li><a class="instagram-monochrome" href="https://www.instagram.com/nihgov/"><span class="icon"></span>Instagram</a></li> <li><a class="youtube-monochrome" href="https://www.youtube.com/user/nihod"><span class="icon"></span>YouTube</a></li> <li><a class="flickr-monochrome" href="https://www.flickr.com/photos/nihgov/"><span class="icon"></span>Flickr</a></li> <li><a class="social-monochrome" href="/news-events/social-media-outreach"><span class="icon"></span>More Social Media from NIH</a></li> </ul> </div> </div> </div> </div> <div class="l-footer-wrapper"> <footer class="l-footer" role="contentinfo"> <h2 class="element-invisible">Footer</h2> <ul> <li><a href="/">NIH Home</a></li> <li><a href="/virtual-tour">Virtual Tour</a></li> <li><a href="https://salud.nih.gov">En Español</a></li> <li><a href="/about-nih/visitor-information">Visitor Information</a></li> <li><a href="/about-nih/frequently-asked-questions">Frequently Asked Questions</a></li> <li><a href="/privacy-policy">Privacy Policy</a></li> <li><a href="/disclaimers">Disclaimers</a></li> <li><a href="/accessibility">Accessibility</a></li> <li><a href="/nih-website-archives">NIH Website Archives</a></li> </ul> <ul> <li><a href="/nondiscrimination-notice">Nondiscrimination Notice</a></li> <li><a href="/institutes-nih/nih-office-director/office-communications-public-liaison/freedom-information-act-office">Freedom of Information Act</a></li> <li><a href="https://www.edi.nih.gov/no-fear-act">No Fear Act</a></li> <li><a href="https://www.hhs.gov/vulnerability-disclosure-policy/index.html">HHS Vulnerability Disclosure</a></li> <li><a href="https://oig.hhs.gov">Office of Inspector General</a></li> <li><a class="lastTab" href="https://www.usa.gov/">USA.gov</a></li> </ul> <p>NIH…Turning Discovery Into Health<sup>®</sup></p> <p>National Institutes of Health, 9000 Rockville Pike, Bethesda, Maryland 20892</p> <p><a href="https://www.hhs.gov">U.S. Department of Health and Human Services</a></p> </footer> </div> <div class="back-to-top"><a href="#"><span class="icon-arrow-up"></span><span class="text">Back to Top</span></a></div> </div> </div> <script src="https://www.nih.gov/sites/default/files/js/js_7Ukqb3ierdBEL0eowfOKzTkNu-Le97OPm-UqTS5NENU.js"></script> <script src="https://www.nih.gov/sites/default/files/js/js_E9yUYWK8EMydR0HMIp2PahVxPAClXyH0Cnh782A15UM.js"></script> </body> </html>