NAEP Process Data

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"> <html xmlns="http://www.w3.org/1999/xhtml" lang="en"> <head id="ctl00_Head1"><title> NAEP Process Data </title><meta http-equiv="Content-Type" content="text/html; charset=utf-8" /><meta http-equiv="Expires" content="0" />  <script type="text/javascript"> (function (w, d, s, l, i) { w[l] = w[l] || []; w[l].push({ 'gtm.start': new Date().getTime(), event: 'gtm.js' }); var f = d.getElementsByTagName(s)[0], j = d.createElement(s), dl = l != 'dataLayer' ? '&l=' + l : ''; j.async = true; j.src = 'https://www.googletagmanager.com/gtm.js?id=' + i + dl; f.parentNode.insertBefore(j, f); })(window, document, 'script', 'dataLayer', 'GTM-N6K3L68'); </script>  <link href="https://fonts.googleapis.com/css?family=Open+Sans:400,600,700,800|Montserrat:400,700" rel="stylesheet" type="text/css" /><link href="https://use.fontawesome.com/releases/v5.0.6/css/all.css" rel="stylesheet" /><link rel="stylesheet" type="text/css" href="/shared/shared/Bootstrap3/css/bootstrap3.min.css" /><link rel="stylesheet" type="text/css" href="/shared/shared/Bootstrap3/css/bootstrap2-custom.min.css" /><link rel="stylesheet" type="text/css" href="/shared/shared/Bootstrap3/css/sp-responsive.css" /><link rel="stylesheet" type="text/css" href="/shared/shared/Bootstrap3/css/bootstrap3-custom.min.css" /><link rel="stylesheet" type="text/css" href="/adminonly/themes/redesign_2015/header/header.css" /><link rel="stylesheet" type="text/css" href="/adminonly/themes/redesign_2015/css/NRC_redesign_2015.css" /><link rel="stylesheet" type="text/css" href="/adminonly/themes/redesign_2015/css/NRC_redesign_2015_small.css" media="(max-width: 1023px)" /><link rel="stylesheet" type="text/css" href="/adminonly/themes/theme_main/css/NRCUserSelectableStyle.css" /><link rel="stylesheet" type="text/css" href="/adminonly/themes/redesign_2015/css/NRC_Redesign_UserSelectableStyle.css" /> <script src="/adminonly/themes/redesign_2024/jquery/jquery-3.6.0.min.js" type="text/javascript"></script> <script src="/shared/shared/Bootstrap3/js/bootstrap3.min.js" type="text/javascript"></script> <script src="/shared/shared/Bootstrap3/js/bootstrap3-custom.js" type="text/javascript"></script> <script src="/adminonly/themes/redesign_2015/scripts/jquery-accessibleMegaMenu.js" type="text/javascript"></script> <script src="/adminonly/themes/redesign_2024/header/header.js" type="text/javascript"></script> <script src="/adminonly/themes/redesign_2015/scripts/NRC_redesign_2015.js" type="text/javascript"></script> <script src="/adminonly/themes/redesign_2015/scripts/nde_dropdown_selector.js" type="text/javascript"></script> <script src="/adminonly/scripts/urlDecode.js" type="text/javascript"></script> <script type="text/javascript" src="/adminonly/themes/redesign_2024/jquery/jquery-ui-1.13.2/jquery-ui.min.js"></script> <style type="text/css"> /* override fancybox style */ .fancybox-wrap { overflow: visible !important; } </style> <script type="text/javascript"> jQuery(function () { // Call DocumentReadyCall() function // located in /adminonly/themes/redesign_2015/scripts/NRC_redesign_2015.js DocumentReadyCall(); // remove saved complex 508 table header metadata from SP. $("table .complex-table-headers-metadata").remove(); // span tag around of .BackToTop that is added via reusable content in CF. // this span tag prevents from align to right in IE7. // so remove the span. var backtotop = $(".BackToTop").parent(); //<p align="right"> tag if (backtotop.parent().is("span.ms-reusableTextView")) { backtotop.unwrap(); } }); </script> <meta name="keywords" content="NAEP, process data, student achievement" /> <meta name="description" content="NAEP process_data -" /> <meta name="DC.Description" content="NAEP process_data -" /> <meta name="DC.Title" content="NAEP Process Data" /> <style type="text/css"> .tg {border-collapse:collapse;border-spacing:0;margin:0px auto;} .tg td{font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;border-color:black;} .tg th{font-family:Arial, sans-serif;font-size:14px;font-weight:normal;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;border-color:black;} .tg .tg-5sk0{font-weight:bold;background-color:#001871;color:#ffffff;border-color:#001871;text-align:center;vertical-align:top} .tg .tg-v0mg{ border-color: #ffffff #001871 #ffffff #001871; text-align: center; vertical-align: middle } .tg .tg-udnh{font-weight:bold;background-color:#001871;color:#ffffff;border-color:#828282;text-align:center;vertical-align:top} .tg .tg-002g{background-color:#ffffff;border-color: #001871 #001871 #001871 #001871;text-align:left;vertical-align:top} </style> <script src="/WebResource.axd?d=8j_Yrh5SAZ5MY08l9UkVbXkI-Iy5w1LetjL0AUXNqFQI5VKjohx0NQIOyEzt2hu5l1wrByQs9Y9-y3FSrrr2kG51JmJNzgql-M3Qkjac0SE1&t=638240199755514788" type="text/javascript"></script> </head> <body>  <noscript> <iframe src="https://www.googletagmanager.com/ns.html?id=GTM-N6K3L68" height="0" width="0" style="display: none; visibility: hidden"></iframe> </noscript>  <div class="nrc_bodyContainer">   <a name="top"></a> <div class="header__top"> <div class="header__top__inner"> <div class="header__logo"> <a href="/"> <img alt="The Nation's Report Card" src="/adminonly/themes/redesign_2015/images/nationsreportcard-logo.png"> </a> </div> <nav> <ul class="nav-menu nav--utility"> <li class="nav-item"> <div class="nav-heading" tabindex="0" aria-label="Reports" role="button">Reports</div> <div class="sub-nav reports"> <ul class="sub-nav-group"> <li class="sub-nav-section"> <div>Current Releases</div> <ul> <li> <a href="/arts_2016/">Arts</a> </li> <li> <div>Civics</div> <ul> <li> <a href="/highlights/civics/2022/">Highlights from the 2022 Assessment</a> </li> <li> <a href="/civics/">Report Card</a> </li> </ul> </li> <li> <a href="/economics_2012/">Economics</a> </li> <li> <div>Geography</div> <ul> <li> <a href="/highlights/geography/2018/">Highlights from the 2018 Assessment</a> </li> <li> <a href="/geography/">Report Card</a> </li> </ul> </li> <li> <div>Long-Term Trend</div> <ul> <li> <a href="/highlights/ltt/2022/">Highlights from the 2022 Age 9 Assessment</a> </li> <li> <a href="/highlights/ltt/2023/">Highlights from the 2023 Age 13 Assessment</a> </li> <li> <a href="/ltt/">Long-Term Trend</a> </li> </ul> </li> <li> <div>Mathematics</div> <ul> <li> <a href="/highlights/mathematics/2022/">Highlights from the 2022 Grade 4 and 8 Assessment</a> </li> <li> <a href="/mathematics/?grade=4">Report Card for the Nation, States, and Districts (grades 4 and 8)</a> </li> <li> <a href="/highlights/mathematics/2019/g12/">Highlights from the 2019 Grade 12 Assessment</a> </li> <li> <a href="/mathematics/?grade=12">Report Card for Grade 12</a> </li> </ul> </li> <li> <div>Reading</div> <ul> <li> <a href="/highlights/reading/2022/">Highlights from the 2022 Grades 4 and 8 Assessment</a> </li> <li> <a href="/reading/?grade=4">Report Card for the Nation, States, and Districts (grades 4 and 8)</a> </li> <li> <a href="/highlights/reading/2019/g12/">Highlights from the 2019 Grade 12 Assessment</a> </li> <li> <a href="/reading/?grade=12">Report Card for Grade 12</a> </li> </ul> </li> </ul> </li> </ul> <ul class="sub-nav-group"> <li class="sub-nav-section"> <ul> <li> <div><br>Science</div> <ul> <li> <a href="/highlights/science/2019/">Highlights from the 2019 Assessment</a> </li> <li> <a href="/science/">2019 Report Card for the Nation</a> </li> <li> <a href="/science_2015/#overview/accordion-3?grade=4">2015 State Assessment</a> </li> <li> <a href="/science_2009/district_summary.aspx">2009 Trial Urban District Assessment</a> </li> </ul> </li> <li> <div>Survey Questionnaires</div> <ul> <li><a href="/stq/">School and Teacher Questionnaire Special Study</a></li> <li> <a href="/sq_students_views_2015/">Students' Views of Mathematics, Reading, and Science</a> </li> <li> <a href="/sq_computer/">Students' Computer Access and Use</a> </li> <li> <a href="/sq_classroom/">Classroom Instruction for Mathematics, Reading, and Science</a> </li> </ul> </li> <li> <div>Technology and Engineering Literacy</div> <ul> <li> <a href="/tel_2018_highlights/">Highlights from the 2018 Assessment</a> </li> <li> <a href="/tel/">Report Card</a> </li> </ul> </li> <li> <div>U.S. History</div> <ul> <li> <a href="/highlights/ushistory/2022/">Highlights from the 2022 Assessment</a> </li> <li> <a href="/ushistory/">Report Card</a> </li> </ul> </li> <li> <div>Writing</div> <ul> <li> <a href="/writing_2011/">National</a> </li> <li> <a href="/writing_2007/w0037.aspx">Trial Urban District Assessment</a> </li> </ul> </li> </ul> </li> <li class="sub-nav-section"> <br> <br> <a href="/report_archive.aspx">Report Library</a> </li> </ul> <div class="nav-vertical-devider"></div> <ul class="sub-nav-group"> <li class="sub-nav-section"> <div>Special Studies</div> <ul> <li> <a href="/process_data/">2017 Mathematics Process Data</a> </li> <li> <a href="/hstsreport/#home">High School Transcript Study</a> </li> <li> <a href="/megastates/">Mega-States</a> </li> <li> <a href="/puerto_rico_2013/#/">Mathematics Puerto Rico</a> </li> <li> <a href="/reading_2013/vocabulary/">Vocabulary</a> </li> </ul> </li> </ul> </div> </li> <li class="nav-item"> <div class="nav-heading" tabindex="0" aria-label="Dashboards" role="button">Dashboards</div> <div class="sub-nav"> <ul class="sub-nav-group"> <li> <a href="/dashboards/achievement_gaps.aspx">Achievement Gap Dashboard</a> </li> <li> <a href="/dashboards/regression/">Racial/Ethnic Score Gap Tool</a> </li> <li> <a href="/dashboards/schools_dashboard.aspx">Public, Private, and Charter Schools Dashboard</a> </li> </ul> </div> </li> <li class="nav-item"> <div class="nav-heading" tabindex="0" aria-label="Data Tools" role="button">Data Tools</div> <div class="sub-nav"> <ul class="sub-nav-group"> <li> <a href="/data_tools.aspx">Data Tools Home</a> </li> <li> <a href="/ndecore/">Data Explorer</a> </li> <li> <a href="/profiles/stateprofile/">State Profiles</a> </li> <li> <a href="/profiles/districtprofile/">District Profiles</a> </li> <li> <a href="/api_documentation.aspx">NAEP Data Service API</a> </li> <li> <a href="https://nces.ed.gov/nationsreportcard/nqt/">Questions Tool</a> </li> <li> <a href="/itemmaps/">Item Maps</a> </li> </ul> </div> </li> <li class="nav-item"> <div class="nav-heading"> <a href="/media.aspx">Newsroom</a> </div> <div class="sub-nav" style="display:none;"/> </li> <li class="nav-item"> <div class="nav-heading"> <a href="/focus_on_naep/">Focus on NAEP</a> </div> <div class="sub-nav" style="display:none;"/> </li> <li class="nav-item"> <div class="nav-heading"> <a href="/takenaepnow/">Take NAEP Now</a> </div> <div class="sub-nav" style="display:none;"/> </li> </ul> </nav> <form action="/search/"> <div class="search-box" onKeyPress="ToggleSearchButton(event)" tabIndex="0" aria-label="Search" role="button"> <img alt="Select to Open Search" id="magnifier" class="magnifying-glass"src="/adminonly/themes/redesign_2015/images/mag.png" onclick="ToggleSearchButton(0)" /> <div id="search-container-id" class="hide-search-box"> <img alt="Carrot Image" id="carrot-glass-id" class="carrot-image" src="/adminonly/themes/redesign_2015/images/carrot.png" /> <div class="search-container"> <input name="q" title="Search Box" id="searchBox" type="text" maxlength="256" class="search-textbox" onblur="HideSearch(event)"> <button title="Search Button" id="searchButton" type="submit" value="Search" class="search-button"> <span class="search-text">SEARCH</span> </button> <input name="output" type="hidden" value="xml_no_dtd" /> <input name="client" type="hidden" value="nces" /> <input name="sort" type="hidden" value="date:D:L:d1" /> <input name="entqr" type="hidden" value="3" /> <input name="site" type="hidden" value="nationsreportcard" /> </div> </div> </div> </form> </div> </div>  <form name="aspnetForm" method="post" action="default.aspx" id="aspnetForm"> <div> <input type="hidden" name="__EVENTTARGET" id="__EVENTTARGET" value="" /> <input type="hidden" name="__EVENTARGUMENT" id="__EVENTARGUMENT" value="" /> <input type="hidden" name="__VIEWSTATE" id="__VIEWSTATE" value="/wEPDwUJODk5OTg5MTY2ZGQj/9ElX0Mir6d7BZSsSVINimcOIw==" /> </div> <script type="text/javascript"> //<![CDATA[ var theForm = document.forms['aspnetForm']; if (!theForm) { theForm = document.aspnetForm; } function __doPostBack(eventTarget, eventArgument) { if (!theForm.onsubmit || (theForm.onsubmit() != false)) { theForm.__EVENTTARGET.value = eventTarget; theForm.__EVENTARGUMENT.value = eventArgument; theForm.submit(); } } //]]> </script> <script src="/WebResource.axd?d=0Io_JG82eEa9K5Z3_7IKL282XsiGAMqeYyZUCsYF8HRdJnB3iMBYRsyZ5nPdS7IAT2pNnuL5jHJ9EFjjcGWam7Ewcc01&t=638313619312541215" type="text/javascript"></script> <script src="/ScriptResource.axd?d=diH_4ctp3OSUwUX43PjybImCUHDLrD8wufnbPygE0_PbSMjk_z9oZ7pm4lVK9QblLcPUb_UQpZjVH360vwxrpOEPSSjCqGfiXeyzlLM0lGCOCHlUX2_wChys0OqyvIB5YOxW9QK-_5VJRuXl_YTYrv05b301&t=ffffffffa50b82b2" type="text/javascript"></script> <script src="/ScriptResource.axd?d=05HaAU87TTW-vICaDaqqxqvFWw-PXCu10o_EHqc28b4ol6EmwxTjhygfTm5p9vC4KvP3_IxFBTWTSr73yL_EA8Rm7CfAtzDUsj67Z-pYOqoTfkSfBpAm_SqRQGgeJDeqcN3PH3u3CCA-bV7vhl34_udDlC6yY-lVmQpnVgv9azO77_6c0&t=ffffffffa50b82b2" type="text/javascript"></script> <div> <input type="hidden" name="__VIEWSTATEGENERATOR" id="__VIEWSTATEGENERATOR" value="EA8940A7" /> </div> <script type="text/javascript"> //<![CDATA[ Sys.WebForms.PageRequestManager._initialize('ctl00$ScriptManager1', document.getElementById('aspnetForm')); Sys.WebForms.PageRequestManager.getInstance()._updateControls([], [], [], 90); //]]> </script> <div id="body" class="nrc_body"> <div id="pageContentContainer" class="nrc_pageContentContainer"> <div id="pageContent" class="nrc_pageContent"> <div id="PubContent"> <div id="contentWrapper" class="single-column"> <div class="row top-bluerow"> <div class="col-xs-9">  <h1 class="nrc-rteElement-h1" style="margin-top:20px;margin-bottom:20px;"> <span style="color:#ffffff;font-size:28px;">Process Data From the 2017 NAEP Grade 4 and Grade 8 Mathematics Assessment</span></h1>  </div> <div class="social col-xs-3">  <div class="icon-print"> <a href="javascript:window.print();"> <img src="/adminonly/themes/redesign_2015/images/spacer.gif" alt="Print" /> </a> </div>  </div> </div> <div class="row main-content"> <div class="col-xs-12">  <p>The National Center for Education Statistics continues its commitment to better understanding what the nations’ students know by releasing a new sample of process data from the math portion of the National Assessment of Educational Progress (NAEP). This website informs all prospective users who plan to work with the NAEP process data. The following sections describe what process data are, how the process data are prepared, and how the final data sets are structured.<br></p><p class="nrc-rteElement-p"> <strong style="font-size:14pt;color:#001871;">Introduction</strong><span style="font-size:14pt;"> </span></p><p class="nrc-rteElement-p">Digital technology has been increasingly utilized in classrooms over the last decade, and NAEP has made significant strides to align with these practices. In 2017, the <a href="/math_2017/about/digitally-based-assessment?grade=4">NAEP mathematics assessment</a> was operationally administered for the first time as a digitally based assessment for grades 4 and 8, providing students with an engaging assessment experience corresponding to the delivery mode of many other large-scale assessments.</p><p class="nrc-rteElement-p">This transition to digitally based assessment (DBA) has also allowed NAEP to measure content knowledge in innovative ways, employ assistive technology to provide enhanced accommodations for students with disabilities, and gather new types of data that deepen our understanding of what students know and can do, including their utilization of new technologies (e.g., equation editor) to interact with the assessment items.</p><p class="nrc-rteElement-p">The data logged within the testing system include student interactions with the assessment platform and the assessment tasks; the data consist of time-stamped records of student-initiated actions (e.g., clicking an assessment feature such as drawing tools) and automatically generated actions by the system (e.g., enter item). The information extracted from this data represents students' interactions during the assessment and processes to arrive at answers, including the time students take to complete specific tasks, the steps or strategies they use to solve problems, and the resources or tools they utilize during the assessment. These new data are called <em>process data</em>. To further enrich our understanding of what students know and can do in the digital environment, process data from the 2017 NAEP grade 8 mathematics assessment was previously made available for secondary analysis in 2021. Process data from the 2017 NAEP grade 4 mathematics will also be accessible for further examination in 2023.</p><p class="nrc-rteElement-p"> <strong style="font-size:14pt;color:#001871;">Digitally Based Assessment in NAEP</strong><span style="font-size:14pt;"> </span></p><p class="nrc-rteElement-p"> <a href="https://nces.ed.gov/nationsreportcard/dba/" target="_blank">NAEP DBAs</a> provide enhanced flexibility in meeting the needs of different students, offering a range of <a href="https://nces.ed.gov/nationsreportcard/about/accom_table.aspx#universal" target="_blank">Universal Design Elements</a>, or built-in functions, which facilitate broader participation without requiring special accommodation sessions. In the digital realm, what used to be considered accommodations in traditional paper-based testing are seamlessly integrated into the universal design framework (e.g., read aloud using text-to-speech, highlighter tool). NAEP also uses assistive technology to offer accommodations for students with disabilities (e.g., extended time). Furthermore, onscreen tools are provided to students to assist them in their problem-solving, such as an in-test equation editor. The goal is for all students to have a seamless assessment administration. </p><p class="nrc-rteElement-p">At the beginning of each assessment, NAEP offers a brief, interactive tutorial that allows students to become familiar with the system, its tools, and its functionality. The tutorials are available to the public online. Experience one of the <a href="https://enaep-public.naepims.org/2017/EN/main.html?subject=Math4" target="_blank">NAEP DBA tutorials</a> and learn more about the tools available to students.</p><p class="nrc-rteElement-p">Some Universal Design Elements and onscreen tools available for the NAEP grade 4 mathematics assessment are described below.</p><div> <img src="/subject/process_data/images/process_data_toolbar_g4.jpg" alt="Graphic of a math assessment item with two points, A and B, plotted on a number line that has the numbers 0 through 3 below it. There are hash marks on the line and arrows at both ends. The following questions are below the number line: Which of the following numbers can be represented by point A on the number line? Multiple choice: A) 0.6 B) 2.4 C) 2.6 D)3.4. There is a gray bar across the top of the image with the following icons inside it, from left to right: A question mark in a black circle. A color contrast symbol that is rectangular and has a lowercase white letter a in a black square on the left and a lowercase black letter a in a white square, which is labeled Color Contrast/Change Theme. A light gray magnifying glass with a minus sign in the middle, and a black magnifying glass with a plus sign in the middle, with the label Zooming underneath and two red arrows pointing up at the magnifying glasses. A capital letter T with a blue speech bubble next to it that has three white dots inside, with a Text-to-Speech label above that has two orange arrows pointing down at the capital letter T and the blue speech bubble with three white dots. A black square with three rings at the top and a scribble on it, with a Scratchwork/Highlighter label below it that has two green arrows pointing up at the black square. A green dotted square box above a black line with the number 2 below the same black line and the label Equation Editor above it that has a purple arrow pointing down at the box, line, and number 2. A gray icon that looks like a calculator, with a small screen at the top and three rows of buttons beneath it. A white, star-shaped icon behind the number VH098751. A gray icon that looks like a stop watch, with one button on the top, one button on the right side, a minute hand, and an hour hand. The word Progress. A gray arrow pointing left. A dark gray arrow in a light gray circle that is pointing right." style="margin:0px;width:100%;" /> <br> </div><p class="nrc-rteElement-p"> <br>Figure 1. Example item: 2017 NAEP Grade 4, illustration of Universal Design Elements.</p><div><table width="100%" cellspacing="0" cellpadding="0" border="1" style="text-align:left;font-family:"open sans", sans-serif;font-size:14px;font-weight:normal;vertical-align:top;"> <colgroup> <col width="50%" /> <col width="50%" /> </colgroup> <tbody><tr><th scope="row" width="40%"> <img src="/subject/process_data/images/tile_blue.png" alt="blue color tile" style="margin:0px;" /> Color Contrast/Change Theme</th><td width="60%">Students have a choice of three color-contrast options, including one high-contrast option.</td></tr><tr><th scope="row"> <img src="/subject/process_data/images/tile_red.png" alt="red color tile" style="margin:0px;" /> Zooming</th><td>Text size options are provided for short stand-alone items (e.g., multiple-choice and short constructed-response items) but not for scenario-based tasks. Default zoom option is 100% with a choice of three other options, or “states” (125%, 150%, and 200%).</td></tr><tr><th scope="row"> <img src="/subject/process_data/images/tile_orange.png" alt="orange color tile" style="margin:0px;" /> Text-to-Speech<br></th><td>Directions, text, figures, and tables within the assessment can be read aloud using the text-to-speech function.<br></td></tr><tr><th scope="row"> <img src="/subject/process_data/images/tile_green.png" alt="green color tile" style="margin:0px;" /> Scratchwork</th><td>A scratchwork tool is available for student use. This tool contains an embedded pencil mode and highlighter mode for annotating figures, performing computations, drawing diagrams, and highlighting portions of a question. Eraser mode is used to clear any drawings and highlights.<br></td></tr><tr><th scope="row"> <img src="/subject/process_data/images/tile_purple.png" alt="purple color tile" style="margin:0px;" /> Equation Editor<br></th><td>An onscreen equation editor is provided for specific items to enter numbers and expressions using the available mathematical symbols.<br></td></tr><tr><th scope="row"> <img src="/subject/process_data/images/tile_gold.png" alt="gold color tile" style="margin:0px;" /> Calculator<br></th><td>For specific blocks in which a calculator is allowed, an onscreen calculator is provided. Please note that calculator model differs by grade.<br></td></tr></tbody></table></div><p class="nrc-rteElement-p"> <br>Try the elements and tools by creating <a href="https://www.nationsreportcard.gov/takenaepnow/createtest" target="_blank">your own test</a> containing released questions from the 2017 NAEP grade 4 or 8 mathematics assessment. </p><div style="text-align:left;"> <strong style="font-size:18.6667px;color:#001871;">About NAEP Process Data</strong><br><br> </div><p class="nrc-rteElement-p">The NAEP digital platform collects different types of data during a student’s assessment session. In addition to students’ responses to items in the cognitive assessment and their responses to survey questions, the testing system also logs the actions or events as students interact with the digital assessment. This data can be transformed into process data variables that represent meaningful student interactions with NAEP assessments. For example, process data variables can describe the amount of time that a student spent on an item; usage of the text to speech, zooming, and highlighting tools; and the number of visits to each assessment item. In contrast to the grade 8 data release, the current release of grade 4 data will not include pre-calculated process data variables. The primary rationale behind this decision is to promote researchers' engagement with the data to gain a deeper understanding of the dataset and its nuances and encourage them to derive the variables that serve their specific research questions and needs better. </p><p class="nrc-rteElement-p">The table below provides a high-level overview of the data in the files available from the released 2017 NAEP grade 4 and grade 8 mathematics assessment dataset. Both grade 8 and grade 4 releases include the full suite of restricted-use response data product materials, process data files, released cognitive items, and survey questionnaire files.</p><table width="100%" cellspacing="0" cellpadding="0" border="1" style="text-align:left;font-family:"open sans", sans-serif;font-size:14px;font-weight:normal;vertical-align:top;"> <colgroup> <col width="40%" /> <col width="60%" /> </colgroup> <tbody><tr><th scope="row">Restricted-Use Response Data</th><td><ul><li>Respondent data</li><li>Documentation </li><li>File formats for different software packages</li></ul></td></tr><tr><th scope="row">Restricted-Use Process Data</th><td><ul><li>Process data</li><li>Documentation</li></ul></td></tr><tr><th scope="row">Restricted-Use Complementary Data</th><td><ul><li>Student demographics and accommodation information</li><li>Student raw and scored item responses</li><li>Item bridge information</li><li>Item file pdf</li><li>Student survey questionnaire pdf</li></ul></td></tr></tbody></table><p class="nrc-rteElement-p"> <br>A brief description of the data provided above is presented below.</p><ul><li>Respondent data include students’ raw response data for cognitive and survey questionnaire items, student demographics and accommodation information; and teacher and school level variables;</li><li>Process data contain process data logs from each student;</li><li>Student information provided within the complementary data include selected demographic and accommodation variables from the respondent data, and students’ raw (e.g., option A for a multiple-choice item) and scored (e.g., 1 for correct, 0 for incorrect) item responses including total raw score for each block<sup><a href="#sup1">1</a></sup>;</li><li>Item information provided within the complementary data include item ID bridge (to support appropriate merging of data files) and images of the released assessment items from the eNAEP digital test platform;</li><li>Student survey questionnaire within the complementary data asks about information on students’ demographic characteristics, opportunities to learn in and out of the classroom, and educational experiences;</li></ul><p class="nrc-rteElement-p"> <strong style="font-size:14pt;color:#001871;">About 2017 NAEP Mathematics Released Process Datasets</strong> </p><p class="nrc-rteElement-p">The 2017 NAEP grade 4 and grade 8 mathematics assessments included two 30-minute blocks of cognitive items in each digital test form, followed by a 15-minute survey questionnaire that collected information on students’ demographic characteristics, opportunities for learning in and outside of the classroom, and educational experiences. There were 10 cognitive blocks administered at each grade, where each block was paired with every other block, resulting in 50 unique digital test forms. After the assessment administration, the NAEP program released two blocks of cognitive items used in 2017 for both grades 4 and 8. One of the released cognitive blocks at each grade was adapted from the paper-based assessment, and the other released cognitive block was designed for the digitally based assessment. It is important to note that the released blocks may not contain all administered items, which creates a partially released block. The unreleased items are secured for reuse in future assessments. At both grades, one of 50 test forms includes both released blocks, while the other forms may only contain one or no released blocks.</p><p class="nrc-rteElement-p">The upcoming grade 4 release and the previous grade 8 release both include block and form level process data. However, there are some differences in the number and specific focus of the data files. Below is a table describing the main differences between the two releases.</p><table width="100%" cellspacing="0" cellpadding="0" border="1" style="text-align:left;font-family:"open sans", sans-serif;font-size:14px;font-weight:normal;vertical-align:top;"> <colgroup> <col width="50%" /> <col width="50%" /> </colgroup> <tbody><tr><th scope="col">Grade 4</th><th scope="col">Grade 8</th></tr><tr><td>Two partially released blocks (MA and MI)</td><td>One fully released block (MC)</td></tr><tr><td>Block MA: 13 out of 14 items were released<br> Block MI: 16 out of 20 items were released</td><td>Block MC: all 15 items were released</td></tr><tr><td>Block MA: ~ 29,100 students<br> Block MI: ~ 30,300 students</td><td>Block MC: ~ 28,200 students</td></tr><tr><td>Specific action focused data for select items:<br> <ul><li>Calculator data</li><li>Keypress and Equation Editor data</li></ul></td><td>Specific action focused data not included</td></tr><tr><td>One Form comprised of Block MI and Block MA</td><td>One Form comprised of Block MI (only at the form level) and Block MC</td></tr><tr><td>Form ~ 2900 students</td><td>Form ~ 2800 students</td></tr><tr><td>Features data not included</td><td>Includes features data (pre-calculated process data variables)</td></tr></tbody></table><p class="nrc-rteElement-p"> <br>Below is a table describing the main variables included in the process data files for both Grade 4 and Grade 8.</p><table width="100%" cellspacing="0" cellpadding="0" border="1" style="text-align:left;font-family:"open sans", sans-serif;font-size:14px;font-weight:normal;vertical-align:top;"> <colgroup> <col width="50%" /> <col width="50%" /> </colgroup> <tbody><tr><th scope="col">Variable Name</th><th scope="col">Definition</th></tr><tr><td>PSEUDO</td><td>Anonymized student identification number</td></tr><tr><td>BLOCKCODE</td><td>The identifier for the block.</td></tr><tr><td>ACCESSIONNUMBER</td><td>The unique identifier for the item.</td></tr><tr><td>ITEMTYPECODE</td><td>The type of item (e.g., multiple choice, fill in the blank).</td></tr><tr><td>OBSERVABLETYPE</td><td>The executed action (e.g., enter item, math keypress).</td></tr><tr><td>CUMULATIVETIME</td><td>The elapsed time, in seconds, starting from the first event, up to and including each event within a block.</td></tr><tr><td>EXTENDEDINFO</td><td>Additional information for specific OBSERVABLETYPE actions.</td></tr></tbody></table><p class="nrc-rteElement-p"> <strong style="font-size:14pt;color:#001871;"> <br>Get Familiar with NAEP Process Data</strong> </p><p class="nrc-rteElement-p">This website provides information about the upcoming release of data files from the 2017 NAEP grade 4 mathematics assessment. Researchers will have access to process data collected from two blocks of grade 4 mathematics assessment questions, along with process data from the grade 4 mathematics survey questionnaire (e.g., capturing students' demographic characteristics, learning opportunities in and outside of school, and educational experiences). Process data allows researchers to go beyond analyzing students’ responses to questions as simply right or wrong. Instead, process data enables researchers to delve deeper into research by exploring topics such as the time students spend on questions, pathways through assessment sections, and tools used while solving problems.</p><p class="nrc-rteElement-p">To effectively utilize process data, researchers must possess a solid understanding of the data and its characteristics. Before deriving insights from process data, researchers must create meaningful variables. Researchers interested in using the data from the grade 4 release will need to extract their own process data variables, as the dataset of pre-constructed process data variables will not be available (only available in the grade 8 release). The intention is to encourage researchers to gain a more comprehensive understanding of Process Data and to foster the generation of a diverse range of derived variables from NAEP data that can effectively address their research questions. We encourage interested researchers to watch the webinars, <a href="https://www.youtube.com/watch?v=12P9074vvEM&t=851s" target="_blank"> <em>Introduction to NAEP Process Data</em></a> and <a href="https://youtu.be/THXTfKNfEn4" target="_blank"> <em>2017 NAEP Grade 4 Mathematics Process Data</em></a>. These webinars will assist researchers in generating research questions and adopting best practices for data utilization. The webinars cover topics such as the preparation of process data for analysis, challenges associated with extracting meaningful insights from this type of data, as well as research examples and analyses that have utilized NAEP process data. The second webinar also provides information specific to the current grade 4 process data release and introduces the new item-level process data. </p><p class="nrc-rteElement-p"> <strong style="font-size:14pt;color:#001871;">Access This Dataset</strong> </p><p class="nrc-rteElement-p">To explore this process dataset, interested researchers should <a href="https://nces.ed.gov/pubsearch/licenses.asp#license" target="_blank">apply for a restricted-use license</a> and request access to the files through the National Center for Education Statistics (NCES) website.</p><p class="nrc-rteElement-p"> <strong style="font-size:14pt;color:#001871;">Protecting Personally Identifiable Information of Students, Teachers, and Schools in NAEP Data</strong> </p><p class="nrc-rteElement-p">Under the National Assessment of Educational Progress Authorization Act (Public Law 107-279 III, section 303), the Commissioner of the National Center for Education Statistics (NCES) is charged with ensuring that NAEP tests do not question test takers about personal or family beliefs or make information about their personal identity publicly available. By design, Personally Identifiable Information (PII) is not available at the individual student or school levels. By law, NAEP is not allowed to report results at the individual or school levels. </p><p class="nrc-rteElement-p">Direct identifiers (i.e., names of students and schools) are not available in the restricted-use process dataset. Each student is represented with a unique pseudo ID that cannot be directly linked to PII. Although every care is taken during the preparation of data sets, the restricted-use process data files may contain PII. However, if this occurs, researchers must remember that they have promised under penalty of law to keep these identities confidential. Any re-identification of students or schools through the use of this dataset is prohibited by law. Violations are subject to prosecution as a Class E Felony, with penalties of up to five years imprisonment and/or a $250,000 fine.</p><p class="nrc-rteElement-p">Frequently Asked Questions: FAQs</p><ol><li style="font-weight:bold;"> <strong>What are the benefits of using process data?</strong></li><ol type="a"><li>Process data provide a wealth of information on how students interact with assessment and the digital platforms that were previously not available in paper and pencil assessments. Coupling process data with the NAEP background data can provide additional insights to better understand student behavior and performance. </li></ol><li style="font-weight:bold;"> <strong>What security measures do I/my organization need to use this data?</strong></li><ol type="a"><li>Please refer to sections 2.4 and 2.5 of the <a href="https://nces.ed.gov/statprog/rudman/toc.asp" target="_blank">restricted-use license manual</a>.</li></ol><li style="font-weight:bold;"> <strong>What software is needed to analyze process data?</strong></li><ol type="a"><li>The process data are provided in the text file format that can be analyzed using statistical software such as R, Stata, SPSS, or Python.</li></ol><li style="font-weight:bold;"> <strong>Is there a way for me to view the original exam questions and interact with the digitally based assessment?</strong></li><ol type="a"><li>Exam questions can be viewed using the <a href="/nqt/" target="_blank">NAEP Questions Tool</a>.</li></ol><li style="font-weight:bold;"> <strong>What is the relationship between a form and a block?</strong></li><ol type="a"><li>For each form, NAEP’s assessment design pairs each block with every other block; therefore, the same block is included across multiple forms. For example, at grade 8, ten of the 50 forms included the released block. At grade 4, ten out of 50 forms included the partially released blocks. For more information please visit <a href="https://nces.ed.gov/nationsreportcard/tdw/instruments/cog_blockdesign.aspx" target="_blank">https://nces.ed.gov/nationsreportcard/tdw/instruments/cog_blockdesign.aspx</a></li></ol><li style="font-weight:bold;">On the NAEP website, there are some student questionnaire items with an asterisk and a note that states, “*This question was intended to be administered in the student questionnaire, but it was not administered to a sufficient number of respondents to support reporting.” <strong>Will data for those items be in the restricted use data license?</strong></li><ol type="a"><li>No, the responses to those survey items are not included in the release product. </li></ol><li style="font-weight:bold;"> <strong>What are some additional resources to learn about NAEP and NAEP assessments?</strong></li><ol type="a"><li>NAEP Sampling Design – <a href="https://nces.ed.gov/nationsreportcard/assessment_process/selection.aspx" target="_blank">https://nces.ed.gov/nationsreportcard/assessment_process/selection.aspx</a></li><li> NAEP Methods and Procedures – <a href="https://nces.ed.gov/nationsreportcard/tdw/" target="_blank">https://nces.ed.gov/nationsreportcard/tdw/</a> </li><li>NAEP Questions Tool – <a href="/nqt/" target="_blank">https://www.nationsreportcard.gov/nqt/</a></li><li>NAEP Digitally Based Assessments – <a href="https://nces.ed.gov/nationsreportcard/dba/" target="_blank">https://nces.ed.gov/nationsreportcard/dba/</a></li><li>NAEP Resources for Researchers – <a href="https://nces.ed.gov/nationsreportcard/researchcenter/" target="_blank">https://nces.ed.gov/nationsreportcard/researchcenter/</a></li></ol></ol><p class="nrc-rteElement-p"> <a id="sup1" name="sup1"></a><sup>1</sup>Please note that plausible values are not included in the grade 4 and grade 8 release data files. Student proficiency levels (i.e., basic, proficient, advanced) are not included in the grade 4 release data files.</p> <br> <br>  </div> </div> </div> </div> </div> </div> </div>  <div id="footer" class="nrc_footer"> <div class="footer-container"> <div class="tabs"> <ul class="tab-links"> <li class="tab1 active"><button type="button" name="tab1">Download Data Tables</button></li> <li class="tab2"><button type="button" name="tab2">Customize Data Tables</button></li> </ul> <div class="tab-content"> <div id="tab1" class="tab tab1 active"> <div class="nde-data-filter-content"> <div class="nde-data-filter-wrapper"> Download data summarizing:<br/> <ul> <li>participation rates</li> <li>inclusion rates and other information about students with disabilities (SD) and English learners (EL)</li> </ul> <div class="nde-data-filter subject"> <label for="subject" class="screen-reader-only">Subject</label> <span class="select-default" title="MATHNS"></span> <select title="Subject selection menu for download data" id="downloadsubject" name="subject" class="dropdown" onchange="subjectOnChange()"> <option value="ART">Arts</option> <option value="CIV">Civics</option> <option value="GEO">Geography</option> <option value="LTT9">Long-Term Trend Reading & Mathematics at Age 9</option> <option value="LTT13">Long-Term Trend Reading & Mathematics at Age 13</option> <option value="MATHNS" class="default-text">Mathematics (National, State & District)</option>  <option value="MATH12">Mathematics (Grade 12)</option> <option value="READNS">Reading (National, State & District)</option>  <option value="READ12">Reading (Grade 12)</option> <option value="SCI">Science</option> <option value="TEL">Technology & Engineering Literacy</option> <option value="HIS">U.S. History</option> <option value="VOC">Vocabulary</option> </select> </div> <div class="btn-download-table"> <a href="#" class="button" onclick="downloadData('xls')">excel file</a><a id="downloadPdfButton" href="#" class="button" onclick="downloadData('pdf')">pdf file</a> </div> </div> </div> </div> <div id="tab2" class="tab tab2"> <div class="nde-data-filter-content"> <div class="nde-data-filter-wrapper"> Generate custom tables by selecting criteria below:<br/> <div class="nde-data-filter jurisdiction"> <label for="jurisdiction" class="screen-reader-only">Jurisdiction</label> <span title="National" class="select-default"></span> <select title="Jurisdiction selection menu" id="jurdid" name="jurisdiction" class="dropdown"> <option value="0">Select a Jurisdiction</option> <option value="National" class="default-text">National</option> <option value="States">States</option> <option value="Districts">Districts</option> </select> </div> <div class="nde-data-filter grade"> <label for="grade" class="screen-reader-only">Grade</label> <span title="1" class="select-default"></span> <select title="Grade selection menu" id="gradeid" name="grade" class="dropdown"> <option value="0">Select a Grade</option> <option value="1" class="default-text">Grade 4</option> <option value="2">Grade 8</option> <option value="3">Grade 12</option> </select> </div> <div class="nde-data-filter subject"> <label for="subject" class="screen-reader-only">Subject</label> <span title="4" class="select-default"></span> <select title="Subject selection menu" name="subject" id="subjectid" class="dropdown"> <option value="0">Select a Subject</option> <option value="1">Civics</option> <option value="2">Economics</option> <option value="3">Geography</option> <option value="4" class="default-text">Mathematics</option> <option value="5">Music</option> <option value="6">Reading</option> <option value="7">Science</option> <option value="8">Technology & Engineering Literacy</option> <option value="9">U.S. History</option> <option value="10">Visual Arts</option> <option value="11">Vocabulary</option> <option value="12">Writing</option> </select> </div> <div class="nde-data-filter statistic"> <label for="statistic" class="screen-reader-only">Statistic</label> <span title="2" class="select-default"></span> <select title="Statistics selection menu" id="statid" name="statistic" class="dropdown"> <option value="1" class="default-text">Average Score</option> <option value="2">Percentages</option> <option value="3">Achievement Level</option> <option value="4">Percentiles</option> </select> </div> <div class="nde-data-filter variable"> <label for="variable" class="screen-reader-only">Variable</label> <span title="1" class="select-default">Overall</span> <select title="Variable selection menu" id="variid" name="variable" class="dropdown"> <option value="1" class="default-text">Overall</option> <option value="2">Gender</option> <option value="3">Race/Ethnicity</option> <option value="4">Region</option> <option value="5">Type of School</option> <option value="6">Charter Schools</option> <option value="7">NSLP Eligibility</option> <option value="8">Parental Education</option> <option value="9">School Location</option> <option value="9">Students with Disabilities</option> <option value="10">English Learners</option> </select> </div> <div class="btn-customize-table"> <a href="#" class="button" onclick="return showtable()">create table</a> </div> </div> </div> </div> <div class="nde-logo"> <img src="/adminonly/themes/redesign_2015/images/footer-ndequick.png" alt="NDE Quick View"/></div> </div> </div> <div class="fixed-tab"> <div class="tab-link">Related Links</div> <div class="tab-content tab"> <ul> <li><a href="/about.aspx">About The Nation's Report Card</a></li> <li><a href="http://nces.ed.gov/nationsreportcard/about/">About National Assessment of Educational Progress (NAEP)</a></li> <li><a href="/contacts.aspx">Contacts</a></li> <li><a href="/faq.aspx">FAQs</a></li> <li><a href="/glossary.aspx?ispopup=false">Glossary</a></li> </ul> <div> FOLLOW US <a class="icon_footer_twitter" title="Follow us on Twitter" aria-label="Follow us on Twitter" href="https://twitter.com/naep_nces" data-twitter=""><img alt="twitter" src="/adminonly/themes/redesign_2015/images/icon-twitter.png"/></a> <a class="icon_footer_facebook" title="Follow us on Facebook" aria-label="Follow us on Facebook" href="https://www.facebook.com/NationalAssessmentofEducationalProgress" data-facebook=""> <img alt="facebook" src="/adminonly/themes/redesign_2015/images/icon-facebook-inv.png"/></a> </div> </div> </div> </div> <div class="container-footer-bg-left" aria-hidden="true"></div> <div class="container-footer-bg-right" aria-hidden="true"></div> </div>  <input type="text" name="__spDummyText1" style="display: none;" size="1" /> <input type="text" name="__spDummyText2" style="display: none;" size="1" /> <script type="text/javascript"> //<![CDATA[ Sys.Application.initialize(); //]]> </script> </form> </div> <script type="text/javascript" src="/js/federated-analytics.all.min.js"></script> </body> </html>

CINXE.COM

NAEP Process Data