CINXE.COM

NERSC and ESnet Take Scientific Data Collaboration to the Next Level

<!doctype html> <html lang="en" class="no-js"> <head> <meta charset="utf-8"> <!--[if IE]><![endif]--> <meta http-equiv="X-UA-Compatible" content="IE=edge,chrome=1"> <base href="https://www.nersc.gov/"><!--[if lte IE 6]></base><![endif]--> <title>NERSC and ESnet Take Scientific Data Collaboration to the Next Level </title> <meta name="generator" content="Silverstripe CMS 4.13" /> <meta http-equiv="Content-Type" content="text/html; charset=utf-8" /> <meta name="description" content="When the Dark Energy Science Collaboration needed a fast, secure, and convenient way to share large data sets with scientists outside of the collaboration, they turned to Berkeley Lab Computing Sciences for help, ultimately choosing an innovative data-sharing solution: the Modern Research Data Portal." /><meta property="og:title" content="NERSC and ESnet Take Scientific Data Collaboration to the Next Level " /> <meta property="og:type" content="website" /> <meta property="og:url" content="https://www.nersc.gov/news-publications/nersc-news/science-news/2021/nersc-and-esnet-take-scientific-data-collaboration-to-the-next-level/" /> <link rel="image_src" href="/assets/Uploads/MRDP-story-DC2-sky-image__ScaleWidthWzMwMF0.jpg" type="image/jpeg" /> <meta property="og:image" content="/assets/Uploads/MRDP-story-DC2-sky-image__ScaleWidthWzMwMF0.jpg" /> <meta property="og:image:type" content="image/jpeg" /> <meta property="og:description" content="When the Dark Energy Science Collaboration needed a fast, secure, and convenient way to share large data sets with scientists outside of the collaboration, they turned to Berkeley Lab Computing Sciences for help, ultimately choosing an innovative data-sharing solution: the Modern Research Data Portal." /> <meta property="og:site_name" content="NERSC" /> <meta property="og:locale" content="en_US" /> <meta name="twitter:title" content="NERSC and ESnet Take Scientific Data Collaboration to the Next Level "> <meta name="twitter:description" content="When the Dark Energy Science Collaboration needed a fast, secure, and convenient way to share large data sets with scientists outside of the collaboration, they turned to Berkeley Lab Computing Sciences for help, ultimately choosing an innovative data-sharing solution: the Modern Research Data Portal."> <meta name="pageID" content="9382" /> <meta name="editingPage" content="https://www.nersc.gov/admin/pages/edit/show/9382" /> <link rel="shortcut icon" href="/favicon.ico" /> <!--[if lte IE 8]> <script src="/mysite/javascript/html5.js"></script> <script src="/mysite/javascript/innershiv.min.js"></script> <![endif]--> <meta name="date" content="2021-04-12 00:00:00" /> <meta name="categories" content="" /> <meta name="robots" content="index,follow" /> <!-- Google tag (gtag.js) --> <script async src="https://www.googletagmanager.com/gtag/js?id=G-2ZF39XXEHQ"></script> <script> window.dataLayer = window.dataLayer || []; function gtag(){dataLayer.push(arguments);} gtag('js', new Date()); gtag('config', 'G-2ZF39XXEHQ'); gtag('config', 'UA-19450141-1'); </script> <!-- SiteImprove head code --> <script type="text/javascript"> /*<![CDATA[*/ (function() { var sz = document.createElement('script'); sz.type = 'text/javascript'; sz.async = true; sz.src = '//siteimproveanalytics.com/js/siteanalyze_8343.js'; var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(sz, s); })(); /*]]>*/ </script> <!-- End SiteImprove --> <!-- $ --> <link rel="stylesheet" type="text/css" href="/_resources/app/javascript/dataTables-1.7/media/css/demo_table_jui.css?m=1740543092" /> <link rel="stylesheet" type="text/css" href="/_resources/themes/nersc-sass/css/screen.css?m=1740543092" media="all" /> <link rel="stylesheet" type="text/css" href="/_resources/themes/nersc-sass/css/print.css?m=1740543092" media="print" /> <link rel="stylesheet" type="text/css" href="/_resources/app/javascript/shadowbox-3.0.3/shadowbox.css?m=1740543092" /> <link rel="stylesheet" type="text/css" href="/_resources/themes/nersc-sass/javascript/jquery-ui-1.8.7.custom/css/custom-theme/jquery-ui-1.8.7.custom.css?m=1740543092" /> </head> <!--[if lt IE 7 ]> <body class="ie ie6 lte6 lte7 lte8 lte9 ArticlePage"> <![endif]--> <!--[if IE 7 ]> <body class="ie ie7 lte7 lte8 lte9 ArticlePage"> <![endif]--> <!--[if IE 8 ]> <body class="ie ie8 lte8 lte9 ArticlePage"> <![endif]--> <!--[if IE 9 ]> <body class="ie ie9 lte9 ArticlePage"> <![endif]--> <!--[if (gt IE 9)|!(IE)]><!--> <body class="ArticlePage"> <!--<![endif]--> <!--googleoff: snippet--><!--googleoff: index--> <div class="page-top"> <div class="container"> <div class="header"> <div id="branding"> <div class="SiteTitle"><a href="" title="Home" accesskey="1"><img src="_resources/themes/nersc-sass/images/logo.png" alt="NERSC" /></a><span class="SiteTagline">Powering <strong>Scientific Discovery</strong> for 50 Years</span></div> <div class="tools"> <div class="membership"> <a href="https://www.nersc.gov/Security/login?BackURL=%2Fnews-publications%2Fnersc-news%2Fscience-news%2F2021%2Fnersc-and-esnet-take-scientific-data-collaboration-to-the-next-level%2F" title="Login">Login</a> </div> <span class="tools-links"> <a href="https://www.lbl.gov/a-z-index/">A-Z Index</a> </span> <span class="social-sharing__holder"> | <a href="" title="Share this page." class="social-sharing_popup"><img src="_resources/themes/nersc-sass/images/share-icon.png" alt="share icon"> Share</a> <div class="social-sharing"> <ul> <li><a target="_blank" href="https://www.facebook.com/sharer/sharer.php?u=https://www.nersc.gov/news-publications/nersc-news/science-news/2021/nersc-and-esnet-take-scientific-data-collaboration-to-the-next-level/" title="Share this on FaceBook"><img class="share-icon" src="_resources/themes/nersc-sass/images/fb_share.png" ></a></li> <li><a target="_blank" href="https://twitter.com/intent/tweet?text=NERSC and ESnet Take Scientific Data Collaboration to the Next Level &url=https://www.nersc.gov/news-publications/nersc-news/science-news/2021/nersc-and-esnet-take-scientific-data-collaboration-to-the-next-level/&via=NERSC" title="Share this on X"><img class="share-icon" src="_resources/themes/nersc-sass/images/x_share.png" ></a></li> <li><a target="_blank" href="https://www.linkedin.com/shareArticle?mini=true&url=https://www.nersc.gov/news-publications/nersc-news/science-news/2021/nersc-and-esnet-take-scientific-data-collaboration-to-the-next-level/&title=NERSC and ESnet Take Scientific Data Collaboration to the Next Level &source=https://www.nersc.gov/news-publications/nersc-news/science-news/2021/nersc-and-esnet-take-scientific-data-collaboration-to-the-next-level/" title="share this on LinkedIn"><img class="share-icon" src="_resources/themes/nersc-sass/images/linkedin_share.png" ></a></li> <li><a target="_blank" href="https://www.reddit.com/submit?url=https://www.nersc.gov/news-publications/nersc-news/science-news/2021/nersc-and-esnet-take-scientific-data-collaboration-to-the-next-level/&title=NERSC and ESnet Take Scientific Data Collaboration to the Next Level " title="Share this on Reddit"><img class="share-icon" src="_resources/themes/nersc-sass/images/reddit_share.png" ></a></li> <li><a href="/cdn-cgi/l/email-protection#ead5999f88808f899ed7a4afb8b9a9ca8b848ecaafb9848f9ecabe8b818fcab989838f849e838c8389caae8b9e8bcaa98586868b8885988b9e838584ca9e85ca9e828fcaa48f929ecaa68f9c8f86cacc8b879ad188858e93d7bd828f84ca9e828fcaae8b9881caaf848f988d93cab989838f84898fcaa98586868b8885988b9e838584cac2aeafb9a9c3c6ca85848fca858cca9e828fca9a9885808f899e99ca8b99998589838b9e8f8eca9d839e82ca9e828fca9f9a89858783848dcaa68f8d8b8993cab99f989c8f93ca858ccab99a8b898fca8b848ecabe83878fcac2a6b9b9bec3c6ca848f8f8e8f8eca8bca8c8b999ec6ca998f899f988fc6ca8b848eca8985849c8f84838f849eca9d8b93ca9e85ca99828b988fca868b988d8fca8e8b9e086a4cca829e9e9a99d0c5c59d9d9dc4848f989989c48d859cc5848f9d99c79a9f888683898b9e83858499c5848f989989c7848f9d99c59989838f84898fc7848f9d99c5d8dad8dbc5848f989989c78b848ec78f99848f9ec79e8b818fc79989838f849e838c8389c78e8b9e8bc7898586868b8885988b9e838584c79e85c79e828fc7848f929ec7868f9c8f86c5" title="Email this to someone"><img class="share-icon" src="_resources/themes/nersc-sass/images/email.png" ></a></li> </ul> </div> </span> <span class="social-following__holder"> | <a href="" class="social-following_popup"><img src="_resources/themes/nersc-sass/images/follow-icon.png" > Follow</a> <div class="social-following"> <ul> <li><a target="_blank" href="https://www.facebook.com/LBNL.NERSC/" title="Follow us on FaceBook"><img class="share-icon" src="_resources/themes/nersc-sass/images/fb_share.png" > Facebook</a></li> <li><a target="_blank" href="https://twitter.com/nersc" title="Follow us on X"><img class="share-icon" src="_resources/themes/nersc-sass/images/x_share.png" > X</a></li> <li><a target="_blank" href="https://www.linkedin.com/company/national-energy-research-scientific-computing-center" title="Follow us on LinkedIn"><img class="share-icon" src="_resources/themes/nersc-sass/images/linkedin_share.png" > LinkedIn</a></li> </ul> </div> </span> <div class="SearchForm"> <form id="Form_SearchForm" action="https://search.usa.gov/search" accept-charset="UTF-8" method="get"> <fieldset> <div id="Search" class="field text nolabel"> <div class="middleColumn"> <input type="text" name="query" value="search…" class="text nolabel showing-search-text usagov-search-autocomplete" id="query"> </div> </div> <div class="clear"><!-- --></div> </fieldset> <input type="hidden" name="utf8" value="✓" /> <input type="hidden" name="affiliate" value="nersc" /> <div class="Actions"> <input type="submit" name="commit" value="Go" class="action" /> </div> </form> </div> </div> <div class="clear"></div> </div> </div> </div> <div class="nav"> <ul class="top-nav" id="Navigation"> <li class="link first"><a href="/" class="topnav-link">Home</a> </li> <li class="link "><a href="/about/" class="topnav-link">About</a> <ul> <li class="link first"><a href="/about/work-at-nersc/">Work at NERSC</a></li> <li class="link "><a href="/about/nersc-staff/">Staff Directory</a></li> <li class="link "><a href="/about/org-chart/">Org Chart</a></li> <li class="link "><a href="/about/nersc-10-year-strategic-plan/">Strategic Plan</a></li> <li class="link "><a href="/about/nersc-history/">History</a></li> <li class="link "><a href="/about/visitor-info/">Visits &amp; Tours</a></li> <li class="link last"><a href="/about/contact-us/">Contact us</a></li> </ul> </li> <li class="link "><a href="/science/" class="topnav-link">Science</a> <ul> <li class="link first"><a href="/science/science-news/">Science News</a></li> <li class="link "><a href="/science/science-vignettes/">Science Vignettes</a></li> <li class="link "><a href="/science/nersc-hpc-achievement-awards/">Early Career Awards</a></li> <li class="link "><a href="/science/science-highlights-presentations/">Science Highlights Presentations</a></li> <li class="link "><a href="/news-publications/publications-reports/nersc-user-publications/">NERSC User Publications</a></li> <li class="link "><a href="/science/covid-19-research/">COVID-19 Research at NERSC</a></li> <li class="link "><a href="/science/accelerator-science/">Accelerator Science</a></li> <li class="link "><a href="/science/astrophysics-and-cosmology/">Astrophysics &amp; Cosmology</a></li> <li class="link "><a href="/science/biological-sciences/">Biological Sciences</a></li> <li class="link "><a href="/science/chemistry-and-materials-science/">Chemistry &amp; Materials Science</a></li> <li class="link "><a href="/science/climate-and-earth-science/">Climate &amp; Earth Science</a></li> <li class="link "><a href="/science/energy-science/">Energy Science</a></li> <li class="link "><a href="/science/engineering-science/">Engineering Science</a></li> <li class="link "><a href="/science/environmental-science/">Environmental Science</a></li> <li class="link "><a href="/science/fusion-science/">Fusion Science</a></li> <li class="link "><a href="/science/math-and-computer-science/">Math &amp; Computer Science</a></li> <li class="link "><a href="/science/nuclear-science/">Nuclear Science</a></li> <li class="link "><a href="/science/user-submitted-research-citations/">User Submitted Research Citations</a></li> <li class="link last"><a href="/science/covid-19/">COVID-19 Support</a></li> </ul> </li> <li class="link "><a href="/systems/" class="topnav-link">Systems</a> <ul> <li class="link first"><a href="/systems/computational-systems-table/">Resource Summary</a></li> <li class="link "><a href="/systems/perlmutter/">Perlmutter </a></li> <li class="link "><a href="/systems/nersc-10/">NERSC-10</a></li> <li class="link "><a href="/systems/community-file-system/">Community File System (CFS)</a></li> <li class="link "><a href="/systems/hpss-data-archive/">HPSS Data Archive</a></li> <li class="link last"><a href="/systems/spin/">Spin</a></li> </ul> </li> <li class="link "><a href="/users/" class="topnav-link">For Users</a> <ul> <li class="link first"><a href="https://docs.nersc.gov/getting-started/">Getting Started</a></li> <li class="link "><a href="/users/accounts/">Accounts &amp; Allocations</a></li> <li class="link "><a href="/users/getting-help/">Getting Help</a></li> <li class="link "><a href="/live-status/motd/">Live Status</a></li> <li class="link "><a href="https://docs.nersc.gov">Documentation</a></li> <li class="link "><a href="/users/policies/">Policies</a></li> <li class="link "><a href="https://my.nersc.gov">My NERSC</a></li> <li class="link "><a href="/users/training/">NERSC Training</a></li> <li class="link last"><a href="/users/NUG/">NERSC Users Group</a></li> </ul> </li> <li class="section "><a href="/news-publications/nersc-news/" class="topnav-link">News</a> <ul> <li class="section first"><a href="/news-publications/nersc-news/">News</a></li> <li class="link "><a href="/news-publications/publications-reports/">Publications &amp; Reports</a></li> <li class="link "><a href="/news-publications/journal-cover-stories/">Journal Covers</a></li> <li class="link "><a href="/news-publications/galleries/">Galleries</a></li> <li class="link last"><a href="/news-publications/announcements/email-lists/">User Annnouncements</a></li> </ul> </li> <li class="link "><a href="/research-and-development/" class="topnav-link">R &amp; D</a> <ul> <li class="link first"><a href="/research-and-development/superfacility/">Superfacility</a></li> <li class="link "><a href="/research-and-development/data-analytics/">Data Analytics</a></li> <li class="link "><a href="/research-and-development/quantum-information-science/">Quantum@NERSC</a></li> <li class="link "><a href="/research-and-development/nesap/">NESAP</a></li> <li class="link "><a href="/research-and-development/advanced-technologies-research-at-nersc/">Advanced Technologies Research at NERSC</a></li> <li class="link "><a href="/research-and-development/nersc-proxy-suite/">NERSC proxy suite</a></li> <li class="link last"><a href="/research-and-development/hpc-requirements-reviews/">HPC Requirements Reviews</a></li> </ul> </li> <li class="link "><a href="/events/calendar/" class="topnav-link">Events</a> <ul> <li class="link first"><a href="/events/calendar/">NERSC Events Calendar</a></li> <li class="link "><a href="/events/nersc50-seminars/">NERSC Seminar Series</a></li> <li class="link last"><a href="/events/gradsatnersc/">Grads@NERSC Community Calls</a></li> </ul> </li> <li class="link last"><a href="/live-status/motd/" class="topnav-link">Live Status</a> <ul> <li class="link first"><a href="/live-status/motd/">Live Status</a></li> <li class="link "><a href="/live-status/scheduled-outages-2/">Scheduled Outages</a></li> <li class="link "><a href="https://my.nersc.gov/queues.php?machine=perlmutter&amp;full_name=Perlmutter">Queue Look</a></li> <li class="link last"><a href="https://my.nersc.gov">My NERSC</a></li> </ul> </li> </ul> </div> </div> <!--googleon: snippet--><!--googleon: index--> <div class="page-body"> <div class="container"> <div class="Layout columns-110 has-subnav"> <!-- $ --> <!--googleoff: snippet--><!--googleoff: index--> <div class="subnav"> <div class="vertical-menu"> <div class="label"><a class="section" href="/news-publications/nersc-news/">News</a></div> <ul class="section menu"> <li class="section"><a class="menuitem section" href="/news-publications/nersc-news/">News</a> <ul class="submenu"> <li class="section"><a class="menuitem section" href="/news-publications/nersc-news/science-news/">Science News</a> </li> <li class="link"><a class="menuitem link" href="/news-publications/nersc-news/nersc-center-news/">Center News</a> </li> <li class="link"><a class="menuitem link" href="/news-publications/nersc-news/nersc50/">NERSC@50</a> </li> </ul> </li> <li class="link"><a class="menuitem link" href="/news-publications/publications-reports/">Publications &amp; Reports</a> </li> <li class="link"><a class="menuitem link" href="/news-publications/journal-cover-stories/">Journal Covers</a> </li> <li class="link"><a class="menuitem link" href="/news-publications/galleries/">Galleries</a> </li> <li class="link"><a class="menuitem link" href="/news-publications/announcements/email-lists/">User Annnouncements</a> </li> </ul> </div> <!--googleoff: snippet--><!--googleoff: index--> <!--googleon: snippet--><!--googleon: index--> </div> <!--googleoff: snippet--><!--googleoff: index--> <div id="Breadcrumbs"> <div class="wrapper"> <div> <a href="/">Home</a> &raquo; <a href="/news-publications/nersc-news/">News</a> &raquo; <a href="/news-publications/nersc-news/">News</a> &raquo; <a href="/news-publications/nersc-news/science-news/">Science News</a> &raquo; NERSC and ESnet Take Scientific Data Collaboration to the Next Level </div> </div> </div> <!--googleon: index--><!--googleon: snippet--> <!--googleon: snippet--><!--googleon: index--> <main> <div class="main"> <div class="main-wrapper"> <div class="pre_content"> <h1 id="page-heading">NERSC and ESnet Take Scientific Data Collaboration to the Next Level </h1> <h2 class="SubTitle">New Data Sharing Tool Boosts Researchers’ Ability to Work With Ultra Large Data Sets</h2> <p class="DateAuthors"> <span class="Date">April 12, 2021</span> </p> <div class="Contact"> <div class="Contact"> <div class="Contact"> <div>By Elizabeth Ball</div> <p>Contact: <a href="&#109;&#x61;&#105;&#x6c;&#116;&#x6f;&#58;&#x63;&#115;&#x63;&#111;&#x6d;&#109;&#x73;&#64;&#x6c;&#98;&#x6c;&#46;&#x67;&#111;&#x76;">&#99;&#x73;&#99;&#x6f;&#109;&#x6d;&#115;&#x40;&#108;&#x62;&#108;&#x2e;&#103;&#x6f;&#118;</a></p> </div> </div> </div> </div> <section> <section><div class="Content"><p class="normal">When the <a href="https://lsstdesc.org/">Dark Energy Science Collaboration</a> (DESC), one of the projects associated with the upcoming <a href="https://www.lsst.org/">Legacy Survey of Space and Time</a> (LSST), needed a fast, secure, and convenient way to share large data sets with scientists around the world, they turned to the National Energy Research Scientific Computing Center (NERSC) for help. Having used NERSC resources to generate simulations and store data, it was time for DESC to enable sharing that data outside the collaboration. Working with NERSC staff they found a solution — and a leap forward for anyone sharing scientific data: the <a href="https://peerj.com/articles/cs-144/">Modern Research Data Portal (MRDP)</a>, a high-performance networking resource developed by engineers at the U.S. Department of Energy’s Energy Sciences Network (ESnet) in collaboration with computer scientists at Argonne National Laboratory and the University of Chicago.</p><div class="captionImage right" style="width: 500px;"><img src="/assets/Uploads/MRDP-story-DC2-sky-image__ResizedImageWzUwMCwzNzdd.jpg" alt="MRDP story DC2 sky image" class="right" width="500" height="377" loading="lazy"> <p class="caption right">Using supercomputing resources at NERSC, Dark Energy Science Collaboration researchers are simulating portions of the data sets they expect to receive from the Legacy Survey of Space and Time, producing realistic faux data to mimic the workings of the finished observatory and sharing these data sets with other scientists via the Modern Research Data Portal. Credit: DESC</p> </div><p class="normal">The ten-year LSST, conducted by the Vera C. Rubin Observatory under construction in northern Chile, will soon be producing vast amounts of imaging data. By conducting the largest-ever astronomical survey using the world’s largest digital camera, the science collaborations working with Rubin Observatory hope to answer foundational questions about the universe.</p><p class="normal">Observatory systems will need to be ready to receive and analyze massive amounts of data when the facility is complete; in preparation, DESC is currently simulating portions of those data sets, producing realistic faux data to mimic the workings of the finished observatory.</p><p class="normal">“LSST will observe about half the sky for ten years. We picked a 300-square-degree patch within that survey area and simulated a five-year survey,” said DESC Computing Coordinator Katrin Heitmann. “After we simulated the data, we used the Rubin Science Pipelines to process it. It was quite an elaborate process, but we wanted to test the processing pipelines in detail and generate an LSST-like data set. Now the science collaboration has started running their analysis pipelines on the simulated data and trying to get cosmology out.”</p><p class="normal">With the release of these data sets, scientists around the world, both inside and outside LSST DESC can view and download them. “Since this is a rather rich data set,” said Heitmann, “we thought it would be nice to share it [with scientists] beyond the collaboration.” That’s where the MRDP comes in.</p><h2 class="normal">A Faster Way to Transfer Data</h2><p class="normal">Many scientists still use traditional, monolithic data portals to move and share data. These portals are based on the standard HTTP protocol — but the software and hardware of the “civilian” Internet can’t always keep up with the increasing size of data sets or what it takes to move them around. Transfers may be slow, and the probability of transfer error becomes more common as the size of the data set becomes more unwieldy.</p><p class="normal">The MRDP — based on the <a href="https://fasterdata.es.net/science-dmz/">Science DMZ</a>, a high-performance network architecture developed at ESnet — is an increasingly common method for sharing data; it separates the administrative elements of data sharing, such as user authentication and request processing, from the task of data storage. DMZ is a term used in computing architecture to indicate a zone somewhere between internal and external networks; the Science DMZ allows data transfers requiring high performance, such as large data sets, to reach the outside world without competing with other traffic. The Science DMZ employs multiple data transfer nodes (<a href="https://fasterdata.es.net/science-dmz/DTN/">DTN</a>s), specialized devices dedicated to high-speed data transfer and running high-performance data transfer tools.</p><p class="normal">When a user initiates a data transfer using an MRDP portal, “the data request is outsourced to systems that are specialized to do the data transfer,” said NERSC data architect Rollin Thomas. “It’s a different model, but it’s better because you can achieve much faster rates of transfer and much higher bandwidth. It’s generally the way to transfer large scientific data sets, of which we have many more than we used to.” The separation of functions and use of a separate high-speed network, such as ESnet, allows for faster data transfer, stronger security, more convenient access for users, and ultimately more capacity for complexity and scaling up.</p><p class="normal">Part of the appeal of the MRDP design is its speed; the Science DMZ can transfer one terabyte (the equivalent of 200 DVDs’ worth of data) every 20 minutes. “Using older systems, it could have taken hours, days, or weeks to move this kind of data around,” said ESnet Science Engagement Engineer Jason Zurawski. “This faster access means we can do more science.</p><h2 class="normal">When DESC Met MRDP</h2><p class="normal">When DESC needed a platform to share their data, software developer Heather Kelly reached out to the NERSC team for consultation. Together, they agreed: an MRDP setup, the first in NERSC history, would be the best way to share and showcase their data before and after the Rubin Observatory is completed. “No matter where the processing or the simulating was happening, ultimately all the data is stored [at NERSC]. So when we wanted to serve up the data….it seemed like a natural choice,” said Kelly.</p><p class="normal">With some guidance from NERSC staff, Kelly and her team set up their MRDP instance within a month, using the NERSC service platform Spin and leveraging NERSC’s <a title="NERSC Globus documentation" href="https://docs.nersc.gov/services/globus/" target="_blank">Globus Sharing endpoint</a> for transferring data. “We’re fortunate in that we have storage space at NERSC and collaborators all over the world,” she said. “But when we want to share outside that group, we have to provide access to the data in a way that’s easy. People find this web interface simple to use, and it makes it easier for us to make the data publicly available.”</p><h2 class="normal">A Data Portal for the Future</h2><p class="normal">The MRDP used by DESC is the first of its kind at NERSC, but almost certainly not the last. As experimental science yields more and more data, more scientists may be drawn to these accessible, high-performance setups, and NERSC will be ready for them.</p><p class="normal">“I think the work this team has been doing with the modern research data portal is the first example of how we’re going to see an increasing number of scientists using NERSC,” said NERSC Data Science Engagement group lead Debbie Bard. “It’s the first step of a much bigger trend: science is producing ever-larger amounts of data from simulations and experiments and sharing that data with their community.”</p><p class="normal">In turn, this new design pattern is shaping the way NERSC prepares for future users, she added. “We can create tools and technologies that can handle [the challenge of big data transfers], and then those tools can be used by all our other users as well, so it scales out the support. So we’re not just doing this work for this science team; all our other science teams can take advantage as well.”</p><p class="normal">So where do NERSC and the MRDP go from here? After working with the DESC, NERSC staff is prepared to help users create their own modern research data portals while avoiding obstacles along the way and working more efficiently.</p><p class="normal">“We learned about where users are going to get stuck building an MRDP and how we can help them stand these things up in a much more self-serve kind of way,” said Thomas. “We’re going to have people not just able to share their data but make it available in the best way possible that the technology allows. That’s the Science DMZ model and the modern research data portal.”</p><p class="normal">ESnet and NERSC are DOE Office of Science user facilities based at Lawrence Berkeley National Laboratory.</p></div></section> <hr /> <div class="AboutBoilerplate"><p><strong>About NERSC and Berkeley Lab</strong><br> The&nbsp;National Energy Research Scientific Computing Center (NERSC) is a U.S. Department of Energy Office of Science User Facility that serves as the primary high performance computing center for scientific research sponsored by the Office of Science. Located at Lawrence Berkeley National Laboratory, NERSC serves almost 10,000 scientists at national laboratories and universities researching a wide range of problems in climate, fusion energy, materials science, physics, chemistry, computational biology, and other disciplines. <a href="http://www.lbl.gov/">Berkeley Lab</a> is a DOE national laboratory located in Berkeley, California. It conducts unclassified scientific research and is managed by the University of California for the U.S. Department of Energy. »<a href="http://cs.lbl.gov">Learn more about computing sciences</a> at Berkeley Lab.&nbsp;</p></div> </section> </div> </div> </main> </div> <!--googleoff: snippet--><!--googleoff: index--> <div class="footer"> <div class="LastEdited">Last edited: 2021-04-21 16:22:19</div> <div class="footer-wrapper"> <div class="badges"> <a href="http://www.lbl.gov/" title="Visit Lawrence Berkeley National Laboratory’s main website."><img src="/_resources/themes/nersc-sass/images/berkeley-lab-badge.png?m=1740543092" alt="Berkeley Lab logo" /></a> <a href="http://science.energy.gov/" title="Visit the U.S. Department of Energy Office of Science website."><img src="/_resources/themes/nersc-sass/images/doe-badge.png?m=1740543092" alt="U.S. Department of Energy Office of Science logo" /></a> </div> <ul class="footer-links"> <li><a href="/about/contact-us/">Contact us</a></li> <li><a href="https://www.lbl.gov/Disclaimers.html">Privacy &amp; Security</a></li> <li><a href="https://cs.lbl.gov">Computing Sciences Area</a></li> </ul> </div> </div> </div> </div> <script data-cfasync="false" src="/cdn-cgi/scripts/5c5dd728/cloudflare-static/email-decode.min.js"></script><script type="application/javascript" src="/_resources/themes/nersc-sass/javascript/jquery.min.js?m=1740543092"></script> <script type="application/javascript" src="/_resources/app/javascript/no-console.js?m=1740543092"></script> <script type="application/javascript" src="/_resources/app/javascript/jquery.beautyOfCode.js?m=1740543092"></script> <script type="application/javascript" src="/_resources/app/javascript/dataTables-1.7/media/js/jquery.dataTables.min.js?m=1740543092"></script> <script type="application/javascript" src="/_resources/app/javascript/shadowbox-3.0.3/shadowbox.js?m=1740543092"></script> <script type="application/javascript" src="/_resources/app/javascript/jquery.metadata.2.0/jquery.metadata.min.js?m=1740543092"></script> <script type="application/javascript" src="/_resources/app/javascript/jquery.media.js?m=1740543092"></script> <script type="application/javascript" src="/_resources/app/javascript/jquery.oembed.min.js?m=1740543092"></script> <script type="application/javascript" src="/_resources/app/javascript/jquery.cycle/jquery.cycle.all.min.js?m=1740543092"></script> <script type="application/javascript" src="/_resources/themes/nersc-sass/javascript/jquery-ui-1.8.7.custom/js/jquery-ui-1.8.7.custom.min.js?m=1740543092"></script> <script type="application/javascript" src="/_resources/app/javascript/jquery.expander/jquery.expander.min.js?m=1740543092"></script> <script type="application/javascript" src="/_resources/app/javascript/core.js?m=1740543092"></script> <script type="application/javascript" src="/_resources/themes/nersc-sass/javascript/theme-core.js?m=1740543092"></script> </body> </html>

Pages: 1 2 3 4 5 6 7 8 9 10