NCEP Meteorological Assimilation Data Ingest System (MADIS)

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd"> <html lang="en"> <head> <title>NCEP Meteorological Assimilation Data Ingest System (MADIS)</title> <meta http-equiv="Content-Type" content="text/html; charset=utf-8"> <meta name="author" content="MADIS Support Team"> <link href="css/header.css" rel="stylesheet" type="text/css" >   <link rel="stylesheet" type="text/css" href="css/ncepmadis.css" media="all">      <script type="text/javascript" src="css/ie.js"></script> </head>  <div id="container"> <div class="header"> <div class="header-content"> <a href="https://www.noaa.gov" class="header-noaa-icon"><img src="images/header-noaa.png" alt="NOAA"></a> <a href="https://www.weather.gov" class="header-nws-icon"><img src="images/header-nws.png" alt="NOAA"></a> <a href="https://www.commerce.gov" class="header-doc"><img src="images/header_doc.png" alt="United States Department of Commerce"></a> <div class="header-center-text"><a class="header-center-text" href="https://www.nco.ncep.noaa.gov">NCEP CENTRAL OPERATIONS</a></div> <div class="header-noaa-text"><a class="header-noaa-text" href="https://www.noaa.gov">National Oceanic and Atmospheric Administration</a></div> </div> </div> <div class="header-shadow"><div class="header-shadow-content"></div></div>    <div class="topnav"> <ul id="topnav">  <li> <div class="section-link"> <a href="#">home</a> </div> </li>  <li> <div class="section-link"> <a href="https://www.weather.gov/news">news</a> </div> <div class="drop"> <ul> <li><a href="https://www.weather.gov/news">NWS News</a></li> <li><a href="https://www.nws.noaa.gov/com/weatherreadynation/calendar.html">Events</a></li> <li><a href="https://www.weather.gov/socialmedia">Social Media</a></li> <li><a href="https://www.nws.noaa.gov/om/brochures.shtml">Pubs/Brochures/Booklets</a></li> <li><a href="https://www.noaa.gov/NOAA-Communications">NWS Media Contacts</a></li> </ul> </div> </li>  <li> <div class="section-link"> <a href="https://www.weather.gov/search">search</a> </div> <div class="drop"> <div class="site-search"> <form method="get" action="https://search.usa.gov/" style="margin-bottom: 0; margin-top: 0;"> <input type="hidden" name="v:project" value="firstgov"> <label for="query">Search For</label> <input type="text" name="query" id="query" size="12" > <input type="submit" value="Go" > <p> <input type="radio" name="affiliate" checked="checked" value="nws.noaa.gov" id="nws" > <label for="nws" class="search-scope">NWS</label> <input type="radio" name="affiliate" value="noaa.gov" id="noaa" > <label for="noaa" class="search-scope">All NOAA</label> </p> </form> </div> </div> </li>  <li> <div class="section-link"> <a href="https://www.weather.gov/search">forecast</a> </div> <div class="drop"> <div id="forecast-lookup" class="site-search"> <form name="getForecast" id="getForecast" action="https://forecast.weather.gov/zipcity.php" method="get"> <label for="inputstring">Local forecast by <br>"City, St" or ZIP code</label> <input id="inputstring" name="inputstring" type="text" size="12" value="Enter location ..." onclick="this.value=''" > <input name="btnSearch" id="btnSearch" type="submit" value="Go" > <div id="txtError"> <div id="errorNoResults" style="display:none;">Sorry, the location you searched for was not found. Please try another search.</div> <div id="errorMultipleResults" style="display:none">Multiple locations were found. Please select one of the following:</div> <div id="errorChoices" style="display:none"></div> <input id="btnCloseError" type="button" value="Close" style="display:none" > </div> <div id="txtHelp"><a style="text-decoration: underline;" href="javascript:void(window.open('https://weather.gov/ForecastSearchHelp.html','locsearchhelp','status=0,toolbar=0,location=0,menubar=0,directories=0,resizable=1,scrollbars=1,height=500,width=530').focus());">Location Help</a></div> </form> </div> </div> </li>  <li> <div class="section-link"> <a href="https://www.weather.gov/about">about</a> </div> <div class="drop"> <ul> <li><a href="https://www.weather.gov/about">About NWS</a></li> <li><a href="https://www.weather.gov/organization">Organization</a></li> <li><a href="https://www.nws.noaa.gov/sp/">Strategic Plan</a></li> <li><a href="https://sites.google.com/a/noaa.gov/nws-best-practices/">For NWS Employees</a></li> <li><a href="https://www.weather.gov/international/">International</a></li> <li><a href="https://www.weather.gov/organization/ncep">National Centers</a></li> <li><a href="https://www.nws.noaa.gov/tg/">Products and Services</a></li> <li><a href="https://www.weather.gov/contact">Contact Us</a></li> <li><a href="https://w1.weather.gov/glossary/">Glossary</a></li> </ul> </div> </li> </ul> </div> </div>  <div id="pbody">  <div id="minor_nav"> <ul id="vertnav"> <br> <li><a href="index.shtml">Distribution Services Home</a></li> </ul> <dl> <dt>News</dt> </dl> <ul id="vertnav"> <li><a href="https://madis.ncep.noaa.gov/madis_recent.shtml">What's Here - v4.0!</a></li>  </ul> <dl> <dt>Resources</dt> </dl> <ul id="vertnav"> <li><a href="https://madis.ncep.noaa.gov/data_application.shtml">Data Application Form</a></li>  <li><a href="https://madis.ncep.noaa.gov/madis_restrictions.shtml">Dataset Restrictions</a></li> <li><a href="madis_api.shtml">API</a></li> <li>Text/XML Surface Dumps   > <ul> <li><a href="public/sfcdumpguest.shtml">Meteorological Surface (guest)</a></li> <li><a href="public/sfcdump.shtml">Meteorological Surface (public)</a></li> <li><a href="public2/sfcdump.shtml">Meteorological Surface (public2)</a></li> <li><a href="research/sfcdump.shtml">Meteorological Surface (research)</a></li> <li><a href="noaa/sfcdump.shtml">Meteorological Surface (noaa)</a></li> <li><a href="public/hydrodumpguest.shtml">Hydrological Surface (guest)</a></li> <li><a href="public/hydrodump.shtml">Hydrological Surface (public)</a></li> <li><a href="public2/hydrodump.shtml">Hydrological Surface (public2)</a></li> <li><a href="research/hydrodump.shtml">Hydrological Surface (research)</a></li> <li><a href="noaa/hydrodump.shtml">Hydrological Surface (noaa)</a></li> </ul> <li>Graphical Displays   > <ul> <li> <a href="/MadisSurface/">Meteorological Surface </a></li> <li><a href="/MadisMetadata/">Metadata</a></li> <li><a href="/MadisAircraftPublic/">Aircraft (Public)</a></li> <li><a href="/MadisAircraft/">Aircraft (Restricted)</a></li> <li><a href="/cap">Multi-Agencey Profiler (CAP)</a></li> </ul> <li><a href="wmo_gdc.shtml">GDC-ABO</a></li>  </ul>  </ul> <dl> <dt>Support</dt> </dl> <ul id="vertnav"> <li><a href="mailto:madis-support@noaa.gov">Contact Us</a></li> <li><a href="https://www.noaa.gov/protecting-your-privacy">Privacy Policy</a></li> </ul> <dl> <dt>Related Links</dt> </dl> <ul id="vertnav"> <li><a href="http://gdc-abo.wmo.int/">WMO Global Data Center</a></li> <li><a href="https://madis.ncep.noaa.gov/index.shtml">MADIS</a></li> <li><a href="https://hads.ncep.noaa.gov">HADS</a></li> <li><a href="https://amdar.ncep.noaa.gov/index.shtml">AMDAR</a></li> <br> <li><a href="https://ruc.noaa.gov/amdar/">AMDAR Research</a></li> </ul> </div> <div id="content"> <a class="textvers" name="content_link"> </a> <html> <style type="text/css">  </style> <body bgcolor="#ffffff" text="#333333"> <div id="banner"> <img src="images/MADISNCEPBanner.jpg" alt="MADIS Banner Image" height="162"></div> <br>  <h1>MADIS Distribution Services</h1> <br> <H2><FONT COLOR="#3333FF">Overview</FONT></H2> <br> MADIS subscribers have access to an integrated, reliable, and easy-to-use database containing real-time and archived observational datasets plus real-time surface analyses grids. To gain access to the MADIS observational database, please fill out a data application form. This will help the MADIS team assess your needs and provide you with the correct data restriction access. <br><br> MADIS ingests <a href="https://madis.ncep.noaa.gov/madis_datasets.shtml">data files</a> from NOAA data sources and non-NOAA data providers, decodes the data and then encodes all of the observational data into a common format with uniform observation units and time stamps. <a href="https://madis.ncep.noaa.gov/madis_qc.shtml">Quality control</a> checks are conducted and the integrated data sets are stored in the MADIS database with a series of flags indicating the quality of the observation from a variety of perspectives (e.g. temporal consistency and spatial consistency), or more precisely, a series of flags indicating the results of various QC checks. MADIS users and their applications can then inspect the flags and decide whether or not to use the observation. <br><br> MADIS data is made available to the enterprise using multiple data transfer protocols via the Internet: <ul> <li><b>ftp/LDM</b></li> <ul> The datasets in the MADIS database are stored in netCDF files. This is the file format that is used when users request MADIS data using either ftp or LDM protocols. <ul><li>real-time access</li> <li>archive of saved real-time access (ftp only)</li> <li>data delivered in netCDF file format</li> </ul> <b>LDAD Mesonet Ingest (AWIPS Users Only)</b> <ul> MADIS supplies LDAD-compatible CSV files for many different mesonets. Note that you don't have to select any of the dataset buttons in the data application form, just put "<b>LDAD mesonet CSV</b>" data into the remarks section. <br><br> Once you subscribe, we will set you up with an account on our ftp server (even if you're an ldm user). Once you log onto the server, you'll find <i>README</i> instructions for each mesonet, the necessary LDAD metadata files (<i>*Station.txt, *.desc</i>), and any necessary scripts and LDAD preprocessors. The station tables are updated weekly. <br><br> Also, where applicable (such as for GPSMET integrated precipitable water vapor), the <i>README</i> file will explain how to customize AWIPS to display additional variables that aren't part of the baseline AWIPS system. <br><br> <b>Detailed Information</b> <ul> <li><a href="https://madis.ncep.noaa.gov/mesonet_providers.shtml">List of mesonets currently available</a><sup>*</sup></li> <li><a href="/MadisSurface/">Station locations and real-time data for all of the mesonets</a></li> <li><a href="madis_ldad_filter.shtml">Instructions on how to filter out stations outside of your geographic area from national-scale mesonets</a></li> <li><a href="LLL-prcp_Since.txt">Accumulated precipitation type table for "super mesonet sampling" customization</a></li> </ul> <sup>*</sup>If any mesonets that you're interested in aren't yet on our ftp server, please <a href="mailto:MADIS_Support@noaa.gov">let us know</a>. </ul> </ul> <li><b>Applications Program Interface (API)</b></li> <ul>MADIS provides an <a href="madis_api.shtml">Application Program Interface (API)</a> and related utility programs, which allow easy access to MADIS observations without having to develop a program for reading netCDF files. The API allows each user to specify station and observation types, as well as QC choices, and domain and time boundaries. Many of the implementation details that arise in data ingest programs are automatically performed. Users of the MADIS API, for example, can choose to have their wind data automatically rotated to a specified grid projection, and/or choose to have mandatory and significant levels from radiosonde data interleaved, sorted by descending pressure, and corrected for hydrostatic consistency. The API is designed so that the underlying format of the database is completely invisible to the user. </ul> <li><b>Web Services</b></li> <ul> Users wanting access to MADIS data via web services are provided with data viewers and graphical displays. <br><br> <li><b>Text/XML Viewer</b></li> <ul> This service provides access to both public and restricted text, csv, or xml formats of the data based on the selections of time, station, provider, variable and quality control. The data is categorized into the two datasets below and available according to your restriction level access. <br><br>MADIS Text/XML Viewer accounts are recommended for those who have a need for data on-demand, as contrasted with continuous real-time access. In general, users who require a continuous datafeed will get better performance by accessing the data via ftp or LDM. Also note that at this time the Text/XML viewer currently only supports the MADIS surface observation datasets. <br><br> <li>Meteorological Surface</li> <ul> <a href="public/sfcdumpguest.shtml">Guest</a>, <a href="public/sfcdump.shtml">Public</a>, <a href="public2/sfcdump.shtml">Public Approved for Additional Mesonets</a>, <a href="research/sfcdump.shtml">Government, Research, Education</a>, <a href="noaa/sfcdump.shtml">NOAA</a> </ul> <li>Hydrological Surface</li> <ul> <a href="public/hydrodumpguest.shtml">Guest</a>, <a href="public/hydrodump.shtml">Public</a>, <a href="public2/hydrodump.shtml">Public Approved for Additional Mesonets</a>, <a href="research/hydrodump.shtml">Government, Research, Education</a>, <a href="noaa/hydrodump.shtml">NOAA</a> </ul> </ul> <li><b>Graphical Displays</b></li> <ul> This service provides access to both public and restricted Meteorological Surface, Metadata, Aircraft and CAP Graphical displays. Click on a graphical display below to view the latest data based on your restriction level access. <br><br> <li><a href="/MadisSurface">Meteorological Surface</a>, <a href="/MadisMetadata">Metadata</a>, <a href="/MadisAircraftPublic">Aircraft (Public)</a>, <a href="/MadisAircraft">Aircraft (Restricted)</a>,<a href="/cap">Multi-Agency Profiler (CAP)</a> </ul> </ul> <li><b>RSAS</b></li> <ul> The surface analyses grids assimilate all of the MADIS surface datasets (including the highly-dense integrated mesonet data). The grids are produced by the Rapid Update Cycle <a href="https://ruc.noaa.gov/">(RUC)</a> Surface Assimilation System <a href="https://madis.ncep.noaa.gov/madis_RSAS.shtml">(RSAS)</a>, which incorporates a 15-km grid stretching from Alaska in the north to Central America in the south, and also covers significant oceanic areas. The RSAS grids are valid at the top of each hour, and are updated every 15 minutes. For users interested in the MADIS RSAS grids, the <a href="rsasdump.shtml">RSASDUMP</a> utility can be used to output the data in plain text. </ul> </ul> <a name="qcstats"></a> <h2>Quality Control Statistics</h2> <br> To view a current list of surface provider QC Summary Messages available in an hourly, daily, weekly, or monthly output, please click <a href="qcms_messages.html">here</a>. <br><br> <h2>Data Compatibility</h2> <br> MADIS data files are compatible with: <ul><li>AWIPS-I and AWIPS-I-like display systems</li> <li>the following analysis software:</li> <ul><li><a href="https://laps.noaa.gov/"> Local Analysis and Prediction System (LAPS)</a></li> <li><a href="https://www.mmm.ucar.edu/wrf/users/wrfda/">Weather Research and Forecasting (WRF) Model Variational Data Assimilation System</a></li> <ul>The current observation pre-processor package that's part of the <A HREF="https://www.mmm.ucar.edu/wrf/users/wrfda/">Weather Research and Forecasting (WRF) variational data assimilation package</A> includes a converter that will convert some of the MADIS datasets into the little_r text format that is then sent to the variational analysis. The MADIS datasets that are currently supported include mesonet, aircraft, ASOS and other METAR, Canadian SAO's, maritime (including ships, buoys and C-MAN stations), radiosondes, aircraft, and GOES satellite winds.</ul> <li><a href="https://www.cmascenter.org/">Atmospheric Model Evaluation Tool (AMET)</a></li> <ul>The AMET tool is provided to the air quality modeling community by the Community Modeling & Analysis System at the University of North Carolina at Chapel Hill.</ul> <li><a href="https://www.caps.ou.edu/ARPS/"> Advanced Regional Prediction System (ARPS) </a></li> <li><a href="https://www2.mmm.ucar.edu/mm5/mm5-home.html"> MM5 </a></li>  </ul> </ul> <hr> <br> <h2>Support</h2> <br> Please address any questions to the MADIS Help Desk at <a href="mailto:madis-support@noaa.gov?Subject=MADIS-Data%20Help" target="_top">madis-support@noaa.gov</a>. <br><br> <a href="https://www.noaa.gov/privacy.html">Privacy Policy</a> <p> <hr> <p align="right"> Last updated 25 July 2018</p> </body></html>  <div class="clear"> </div> </div>  </div>   <div class="footer-legal"> <div class="footer-legal-content"> <div class="footer-legal-gov"> <a href="https://www.usa.gov"><img src="images/usa_gov.png" alt="usa.gov" width="110" height="30" /></a> </div> <div class="footer-legal-column"> <p> <a href="https://www.commerce.gov">US Dept of Commerce</a><br /> <a href="https://www.noaa.gov">National Oceanic and Atmospheric Administration</a><br /> <a href="https://www.weather.gov" >National Weather Service</a><br /> 5830 University Research Ct.<br />College Park, MD 20740<br /> Web Master's Email: <a href='mailto:w-nws.webmaster@noaa.gov'>w-nws.webmaster@noaa.gov</a> </p> </div> <div class="footer-legal-column2"> <a href="https://www.weather.gov/disclaimer">Disclaimer</a><br /> <a href="https://www.cio.noaa.gov/services_programs/info_quality.html">Information Quality</a><br /> <a href="https://www.weather.gov/help">Help</a><br /> <a href="https://www.weather.gov/glossary">Glossary</a> </div> <div class="footer-legal-column3"> <a href="https://www.weather.gov/privacy">Privacy Policy</a><br /> <a href="https://www.noaa.gov/information-technology/foia">Freedom of Information Act (FOIA)</a><br /> <a href="https://www.weather.gov/about">About Us</a><br /> <a href="https://www.weather.gov/careers">Career Opportunities</a> </div> </div> </div> </div> </div> </div>  </body> </html>

CINXE.COM

NCEP Meteorological Assimilation Data Ingest System (MADIS)