Network Error Logging

<!DOCTYPE html> <html> <head> <title>Network Error Logging</title> <meta charset='utf-8'> <meta name="color-scheme" content="light dark"> <script src='https://www.w3.org/Tools/respec/respec-w3c' defer class='remove'></script> <script class='remove'> var respecConfig = { group: "webperf", github: "https://github.com/w3c/network-error-logging/", shortname: "network-error-logging", specStatus: "ED", editors: [{ name: "Douglas Creager", url: "https://dcreager.net/", company: "GitHub", w3cid: "103120", }, { name: "Ian Clelland", company: "Google", w3cid: "76841", }], formerEditors: [{ name: "Ilya Grigorik", url: "https://www.igvita.com/", company: "Google", retiredDate: "2019-08-01", }, { name: "Julia Tuttle", company: "Google", retiredDate: "2017-09-01", }, { name: "Arvind Jain", company: "Google", retiredDate: "2015-01-01", }, { name: "Alois Reitbauer", company: "Compuware Corp.", retiredDate: "2014-10-01", }, { name: "Jatinder Mann", company: "Microsoft", retiredDate: "2014-02-01", }], localBiblio: { "NETWORK-REPORTING": { title: "Network Reporting API", href: "https://w3c.github.io/reporting/network-reporting.html", status: "ED", publisher: "W3C", }, }, }; </script> <style> .reportTypeGroup { margin-top: 1em; padding-top: 1ex; border-top: 1px solid black; } </style> </head> <body> <section id='abstract'> <p>This document defines a mechanism that enables developers to declare a network error reporting policy for a web application. A user agent can use this policy to report encountered network errors that prevented it from successfully fetching requested resources.</p> </section> <section id='sotd'> </section> <section> <h2>Introduction</h2> <p>Accurately measuring performance characteristics of web applications is an important aspect in helping site developers understand how to improve their web applications. The worst case scenario is the failure to load the application, or a particular resource, due to a network error, and to address such failures the developer requires assistance from the user agent to identify when, where, and why such failures are occurring.</p> <p>Today, application developers do not have real-time web application availability data from their end users. For example, if the user fails to load the page due to a network error, such as a failed DNS lookup, a connection timeout, a reset connection, or other reasons, the site developer is unable to detect and address this issue. Note that these kinds of network errors cannot be detected purely server-side, since by definition the client might not have been able to successfully establish a connection with the server.</p> <p>Existing methods (such as synthetic monitoring) provide a partial solution by placing monitoring nodes in predetermined geographic locations, but require additional infrastructure investments, and cannot provide truly global and near real-time availability data for real end users.</p> <p>Network Error Logging (NEL) addresses this need by defining a mechanism enabling web applications to declare a reporting policy that can be used by the user agent to report network errors for a given origin. A web application opts into using NEL by supplying a <a>NEL</a> HTTP response header field that describes the desired <a>NEL policy</a>. This policy instructs the user agent to log information about requests to that origin, and to attempt to deliver that information to a group of endpoints previously configured using the [[[REPORTING]]. As the name implies, NEL reports are primarily used to describe <em>errors</em>. However, in order to determine <em>rates</em> of errors across different client populations, we must also know how many <em>successful</em> requests are occurring; these successful requests can also be reported via the NEL mechanism.</p> <p>For example, if the user agent fails to fetch a resource from <code>https://www.example.com</code> due to an aborted TCP connection, the user agent would queue the following report via the Reporting API:</p> <dl> <dt>type</dt> <dd><code>"network-error"</code></dd> <dt>endpoint group</dt> <dd>the endpoint group configured by the <a>report_to</a> field</dd> <dt>settings</dt> <dd>TODO</dd> <dt>data</dt> <dd><pre class="highlight"> { "referrer": "https://referrer.com/", "sampling_fraction": 1.0, "server_ip": "192.0.2.42", "protocol": "http/1.1", "elapsed_time": 321, "phase": "connection", "type": "tcp.aborted" } </pre></dd> </dl> <p> See <a href="#generate-a-network-error-report"></a> for an explanation of the communicated fields and format of the report, and <a href="#examples"></a> for more hands-on examples of NEL registration and reporting process. </p> </section> <section id="conformance"> <p>Requirements phrased in the imperative as part of algorithms (such as "strip any leading space characters" or "return false and abort these steps") are to be interpreted with the meaning of the key word ("must", "should", "may", etc) used in introducing the algorithm.</p> <p>Some conformance requirements are phrased as requirements on attributes, methods or objects. Such requirements are to be interpreted as requirements on the user agent.</p> <p>Conformance requirements phrased as algorithms or specific steps may be implemented in any manner, so long as the end result is equivalent. (In particular, the algorithms defined in this specification are intended to be easy to follow, and not intended to be performant.)</p> <section> <h2>Dependencies</h2> <dl> <dt>DNS</dt> <dd> <p>The following terms are defined in the DNS specification: [[RFC1034]]</p> <ul> <li><dfn data-cite="!RFC1034#section-3.1">domain name</dfn></li> <li><dfn data-cite="!RFC1034#section-3.1">domain namespace tree</dfn></li> <li><dfn data-cite="!RFC1034#section-5">resolver</dfn></li> </ul> </dd> <dt>Fetch</dt> <dd> <p>The following terms are defined in the Fetch specification: [[FETCH]]</p> <ul> <li><dfn data-cite="!FETCH#concept-request-client">client</dfn></li> <li><dfn data-cite="!FETCH#cors-preflight-request">CORS-preflight request</dfn></li> <li><dfn data-cite="!FETCH#determine-the-network-partition-key">determine the network partition key</dfn></li> <li><dfn data-cite="!FETCH#extract-header-list-values">extract header list values</dfn></li> <li><dfn data-cite="!FETCH#header-list-contains">header list contains</dfn></li> <li><dfn data-cite="!FETCH#concept-header-name" data-lt="header names">header name</dfn></li> <li><dfn data-cite="!FETCH#concept-header-value">header value</dfn></li> <li><dfn data-cite="!FETCH#http-network-fetch">HTTP-network fetch</dfn></li> <li><dfn data-cite="!FETCH#http-network-or-cache-fetch">HTTP-network-or-cache fetch</dfn></li> <li><dfn data-cite="!FETCH#network-partition-key">network partition key</dfn></li> <li><dfn data-cite="!FETCH#redirect-status" data-lt="redirects">redirect status</dfn></li> <li><dfn data-cite="!FETCH#concept-request-header-list">request header list</dfn></li> <li><dfn data-cite="!FETCH#concept-response" data-lt="responses">response</dfn></li> <li><dfn data-cite="!FETCH#concept-response-header-list">response header list</dfn></li> </ul> </dd> <dt>High Resolution Time</dt> <dd> <p>The following terms are defined in the High Resolution Time specification: [[HR-TIME]]</p> <ul> <li><dfn data-cite="!HR-TIME#dfn-current-wall-time">current wall time</dfn></li> <li><dfn data-cite="!HR-TIME#dfn-duration-from">duration from</dfn></li> </ul> </dd> <dt>HSTS</dt> <dd> <p>The following terms are defined in the HSTS specification: [[RFC6797]]</p> <ul> <li><dfn data-cite="!RFC6797#section-8.2">superdomain match</dfn></li> </ul> </dd> <dt>HTML</dt> <dd> <p>The following terms are defined in the HTML specification: [[HTML]]</p> <ul> <li><dfn data-cite="!HTML#navigate" data-lt="navigation">navigate</dfn></li> <li><dfn data-cite="!HTML#navigator.online"><code>navigator.onLine</code></dfn></li> <li>resource <dfn data-cite="!HTML#origin" data-lt="origins">origin</dfn></li> </ul> </dd> <dt>HTTP</dt> <dd> <p>The following terms are defined in the HTTP specification: [[RFC7230]], [[RFC7231]], [[RFC7232]], [[RFC7234]]</p> <ul> <li><dfn data-cite="!RFC7231#section-6.3.1" data-lt="200 response">200 status code</dfn></li> <li><dfn data-cite="!RFC7231#section-6.5" data-lt="4xx">4xx status code</dfn></li> <li><dfn data-cite="!RFC7231#section-6.6" data-lt="5xx">5xx status code</dfn></li> <li><dfn data-cite="RFC7232#section-2.3"><code>ETag</code></dfn></li> <li><dfn data-cite="RFC7232#section-3.2"><code>If-None-Match</code></dfn></li> <li><dfn data-cite="!RFC7230#section-6.3">persistent connections</dfn></li> <li><dfn data-cite="!RFC7230#section-2.1" data-lt="requests">request</dfn></li> <li><dfn data-cite="!RFC7231#section-4">request method</dfn></li> <li><dfn data-cite="!RFC7231#section-3">resource representation</dfn></li> <li><dfn data-cite="!RFC7231#section-7" data-lt="response header">response headers</dfn></li> <li><dfn data-cite="!RFC7230#section-2.1" data-lt="servers">server</dfn></li> <li><dfn data-cite="!RFC7231#section-6">status code</dfn></li> </ul> </dd> <dt>HTTP JSON field values</dt> <dd> <p>The following terms are defined in the HTTP-JFV specification: [[HTTP-JFV]]</p> <ul> <li><dfn data-cite="!HTTP-JFV#json-field-value">json-field-value</dfn></li> </ul> </dd> <dt>JSON</dt> <dd> <p>The following terms are defined in the JSON specification: [[RFC7159]]</p> <ul> <li><dfn data-cite="!RFC7159#section-4">JSON object</dfn></li> </ul> </dd> <dt>Network Reporting API</dt> <dd> <p>The following terms are defined in the Network Reporting API specification: [[NETWORK-REPORTING]]</p> <ul> <li><dfn data-cite="!NETWORK-REPORTING#endpoint-group">endpoint group</dfn></li> <li><dfn data-cite="!NETWORK-REPORTING#generate-a-network-report">Generate a network report</dfn></li> </ul> </dd> <dt>Referrer Policy</dt> <dd> <p>The following terms are defined in the Referrer Policy specification: [[REFERRER-POLICY]]</p> <ul> <li><dfn data-cite="!REFERRER-POLICY#referrer-policy">referrer policy</dfn></li> </ul> </dd> <dt>Reporting API</dt> <dd> <p>The following terms are defined in the Reporting API specification: [[REPORTING]]</p> <ul> <li><dfn data-cite="!REPORTING#report" data-lt="reports">report</dfn></li> <li><dfn data-cite="!REPORTING#report-body">report body</dfn></li> <li><dfn data-cite="!REPORTING#report-type">report type</dfn></li> <li><dfn data-cite="!REPORTING#visible-to-reportingobservers">visible to <code>ReportingObserver</code>s</dfn></li> </ul> </dd> <dt>Resource Timing</dt> <dd> <p>The following terms are defined in the Resource Timing specification: [[RESOURCE-TIMING-2]]</p> <ul> <li><dfn data-cite="!RESOURCE-TIMING-2#dom-performanceresourcetiming-nexthopprotocol">network protocol</dfn></li> </ul> </dd> <dt>Secure Contexts</dt> <dd> <p>The following terms are defined in the Secure Contexts specification: [[SECURE-CONTEXTS]]</p> <ul> <li>the <dfn data-cite="!SECURE-CONTEXTS#is-origin-trustworthy">"Is origin potentially trustworthy?"</dfn> algorithm</li> <li><dfn data-cite="!SECURE-CONTEXTS#potentially-trustworthy-origin" data-lt="potentially trustworthy origins">potentially trustworthy origin</dfn></li> </ul> </dd> <dt>URL</dt> <dd> <p>The following terms are defined in the URL specification: [[URL]]</p> <ul> <li><dfn data-cite="!URL#concept-url-fragment">fragment</dfn></li> <li><dfn data-cite="!URL#concept-url-path">path</dfn></li> <li><dfn data-cite="!URL#concept-url-query">query</dfn></li> <li><dfn data-cite="!URL#concept-url">URL</dfn></li> <li><dfn data-cite="!URL#concept-url-serializer">URL serializer</dfn></li> </ul> </dd> </dl> </section> </section> <section> <h2>Concepts</h2> <section> <h2>Network requests</h2> <p> A <dfn data-lt="network requests">network request</dfn> occurs when the user agent must use the network to service a single <a>request</a>. </p> <p> If the user agent can service a <a>request</a> out of a local cache, that <a>request</a> MUST NOT result in a <a>network request</a>. </p> <p> If the user agent follows <a>redirects</a> as part of a <a>navigation</a>, there MUST be separate <a>network requests</a> for each <a>request</a> in the redirect chain. </p> <p> A <a>request</a> MUST NOT result in a <a>network request</a> if the user agent is known to be offline (i.e., when <a>navigator.onLine</a> returns <code>false</code>). </p> <p> A <a>request</a> MUST NOT result in a <a>network request</a> if it is blocked due to mixed content or CORS failures. Any <a>CORS-preflight request</a> MUST result in its own <a>network request</a>. </p> <p class="note"> For user agents that service <a>requests</a> according to the [[FETCH]] standard, a <a>network request</a> corresponds to one execution of the <a>HTTP-network fetch</a> algorithm. </p> <p> Regardless of which fetch algorithm and which underlying application and transport protocols are used, servicing a <a>network request</a> consists of the following <dfn data-lt="phase">phases</dfn>: </p> <ol> <li> <dfn>DNS resolution</dfn>: The user agent uses the Domain Name System [[RFC1034]] to resolve a domain name into an IP address of a <a>server</a> can that service HTTP requests to that domain. </li> <li> <dfn>Secure connection establishment</dfn>: The user agent opens a connection to the <a>server</a>, and establishes a secure channel over this connection. </li> <li> <dfn>Transmission of request and response</dfn>: Once the secure channel is established, the user agent can transmit the HTTP request, and receive the response from the <a>server</a>. </li> </ol> <p> The only mandatory <a>phase</a> is the <a>transmission of request and response</a>; the other <a>phases</a> might not be needed for every <a>network request</a>. For instance, DNS results can be cached locally in the user agent, eliminating <a>DNS resolution</a> for future requests to the same domain. Similarly, HTTP <a>persistent connections</a> allow open connections to be shared for multiple requests to the same <a>origin</a>. However, if multiple <a>phases</a> occur, they will occur in the above order. </p> <p class="ednote"> We would like to move the definition of these <a>phases</a> into [[FETCH]] so that they are more reusable. </p> <p> A <a>network request</a> is <dfn data-lt="succeed|succeeded">successful</dfn> if the user agent is able to receive a valid HTTP response from the server, and that response does not have a <a>4xx</a> or <a>5xx</a> status code. </p> <p> A <a>network request</a> is <dfn data-lt="fail|failures">failed</dfn> if it is not <a>successful</a>. </p> <p class="note"> Note that HTTP error responses (i.e., those with a <a>4xx</a> or <a>5xx</a> status code) are considered <a>failures</a>, so that they are subject to a <a>NEL policy</a>'s <a>failure sampling rate</a> instead of its <a>successful sampling rate</a>. </p> </section> <section> <h2>Network errors</h2> <p> A <dfn data-lt="network errors">network error</dfn> is the error condition that caused a <a>network request</a> to <a>fail</a>. </p> <p> Each <a>network error</a> has a <dfn data-lt="types">type</dfn>, which is a string. </p> <p> Each <a>network error</a> has a <dfn data-lt-nodefault data-lt="type-phase">phase</dfn>, which describes which <a>phase</a> the error occurred in: </p> <dl> <dt><code>dns</code></dt> <dd>the error occurred during <a>DNS resolution</a></dd> <dt><code>connection</code></dt> <dd> the error occurred during <a>secure connection establishment</a> </dd> <dt><code>application</code></dt> <dd> the error occurred during the <a>transmission of request and response</a> </dd> </dl> <p> There are several predefined <a>network error</a> <a>types</a> defined in <a href="#predefined-network-error-types"></a>. </p> </section> <section> <h2>Network error reports</h2> <p> A <dfn data-lt="network error reports">network error report</dfn> is a [[[REPORTING]]] <a>report</a> that describes a <a>network error</a>. </p> <p> <a>Network error reports</a> have a <a>report type</a> of <code>network-error</code>. </p> <p> <a>Network error reports</a> are <strong>NOT</strong> <a>visible to <code>ReportingObserver</code>s</a>. </p> <p class="note"> <a>Network error reports</a> are not <a>visible to <code>ReportingObserver</code>s</a> because they are only intended to be visible to the administrator or owner of the server <em>receiving</em> the requests. If they were <a>visible to <code>ReportingObserver</code>s</a>, then the reports would also be visible to the <em>originator</em> of the request. For cross-origin requests, this could leak information about the server's network configuration to parties outside of its control. </p> </section> <section> <h2>NEL policies</h2> <p> A <dfn data-lt="NEL policies|NEL policy" data-export id="dfn-nel-policies">NEL policy</dfn> instructs a user agent whether to collect reports about <a>network requests</a> to an <a>origin</a>, and if so, where to send them. <a>NEL policies</a> are delivered to the user agent via HTTP <a>response headers</a>. </p> <p> Each <a>NEL policy</a> has a <dfn>received IP address</dfn>, which is the IP address of the <a>server</a> that the user agent received this <a>NEL policy</a> from. </p> <p> Each <a>NEL policy</a> has an <dfn data-lt-nodefault data-lt="policy origin">origin</dfn>. </p> <p> Each <a>NEL policy</a> has a <dfn>subdomains</dfn> flag, which is either <code>include</code> or <code>exclude</code>. </p> <p> <p> Each <a>NEL policy</a> has a list of <dfn data-lt-noDefault data-lt="policy request headers">request headers</dfn> and a list of <dfn data-lt-noDefault data-lt="policy response headers">response headers</dfn>, each of which is a list of <a>header names</a>. </p> <p> Each <a>NEL policy</a> has a <dfn>reporting group</dfn>, which is the name of the Reporting <a>endpoint group</a> that reports for this policy will be sent to. </p> <p> Each <a>NEL policy</a> has a <dfn>ttl</dfn> representing the number of seconds the policy remains valid. </p> <p> Each <a>NEL policy</a> has a <dfn>creation</dfn> which is the timestamp when the user agent received the policy. </p> <p> A <a>NEL policy</a> is <dfn>stale</dfn> if the <a>duration from</a> its <a>creation</a> to the <a>current wall time</a> is greater than 172800 seconds (48 hours). </p> <p> A <a>NEL policy</a> is <dfn>expired</dfn> if the <a>duration from</a> its <a>creation</a> to the <a>current wall time</a> is greater than its <a>ttl</a> (in seconds). </p> </section> <section> <h2>Sampling rates</h2> <p> An <a>origin</a> that expects to serve a large volume of traffic might not be equipped to ingest NEL reports for every <a>network request</a> made to the origin. The origin can define <dfn data-lt="sampling rate">sampling rates</dfn> to limit the number of NEL reports that each user agent submits. Since <a>successful</a> requests should typically greatly outnumber <a>failed</a> requests, the origin can specify different sampling rates for each. </p> <p> Each <a>NEL policy</a> has a <dfn>successful sampling rate</dfn>, which is a number between 0.0 and 1.0 inclusive. </p> <p> Each <a>NEL policy</a> has a <dfn>failure sampling rate</dfn>, which is a number between 0.0 and 1.0 inclusive. </p> </section> <section> <h2>Policy cache</h2> <p> A conformant user agent MUST provide a <dfn>policy cache</dfn>, which is a storage mechanism that maintains a set of <a>NEL policies</a>, keyed by (<a>network partition key</a>, <a>origin</a>) tuples. </p> <p> This storage mechanism is opaque, vendor-specific, and not exposed to the web, but it MUST provide the following methods which will be used in the algorithms this document defines: </p> <ul> <li>Insert, update, and delete <a>NEL policies</a>.</li> <li>Retrieve the <a>NEL policy</a>, if any, for a given <a>origin</a> and <a>network partition key</a>.</li> <li>Clear the cache.</li> </ul> </section> </section> <section> <h2>Policy delivery</h2> <p> A <a>server</a> MAY define a <a>NEL policy</a> for an origin it controls via the <a>NEL</a> HTTP <a>response header</a>. </p> <section> <h2><code>NEL</code> response header</h2> <p> The <dfn><code>NEL</code></dfn> <a>response header</a> is used to communicate an <a>origin</a>'s <a>NEL policy</a> to the user agent. The ABNF (Augmented Backus-Naur Form) syntax for the <a>NEL</a> header is as follows: </p> <pre>NEL = json-field-value</pre> <p> The header's value is interpreted as an array of JSON objects, as defined by <a>json-field-value</a>. Each object in the array defines an <a>NEL policy</a> for the origin. The user agent MUST process the first valid policy in the array and ignore any additional policies in the array. </p> <p>User agents MUST ignore any unknown or invalid field(s) or value(s) that do not conform to the syntax defined in this specification. A valid <a>NEL</a> header field MUST, at a minimum, contain one object with all of the "REQUIRED" fields defined in this specification.</p> <p>The user agent MUST ignore the <a>NEL</a> header specified via a <code>meta</code> element to mitigate hijacking of error reporting via scripting attacks. The <a>NEL policy</a> MUST be delivered via the <a>NEL</a> <a>response header</a>.</p> <p class="note">The restriction on <code>meta</code> element is consistent with the [[CSP]] specification, which restricts reporting registration to HTTP header fields only for the same reasons.</p> <section> <h2>The <code>report_to</code> member</h2> <p> The <dfn><code>report_to</code></dfn> member specifies the <a>endpoint group</a> that reports for this <a>NEL policy</a> will be sent to. The <a>report_to</a> member is REQUIRED to register a <a>NEL policy</a>, and OPTIONAL if the intent is to remove a previous registration – see <a>max_age</a>. If present, its value MUST be a string; any other type will result in a parse error. </p> <p class="note"> To improve delivery of NEL reports, the <a>server</a> should set <code>report_to</code> to an <a>endpoint group</a> containing at least one endpoint in an alternative origin whose infrastructure is not coupled with the origin from which the resource is being fetched — otherwise network errors cannot be reported until the problem is solved, if ever — and provide multiple endpoints to provide alternatives if some endpoints are unreachable. </p> </section> <section> <h2>The <code>max_age</code> member</h2> <p> The REQUIRED <dfn><code>max_age</code></dfn> member specifies the lifetime of this <a>NEL policy</a>, as a non-negative integer number of seconds. Its value MUST be an non-negative integer; any other type will result in a parse error. </p> <p> A value of <code>0</code> will cause any <a>NEL policy</a> for this <a>origin</a> to be removed from the <a>policy cache</a>. </p> <p class="note"> To ensure delivery of NEL reports, the <a>server</a> should ensure that the Reporting <a>endpoint group</a> is also configured with a sufficiently high <code>max_age</code>. If the Reporting policy expires, NEL reports will not be delivered, even if the NEL policy has not expired. </p> </section> <section> <h2>The <code>include_subdomains</code> member</h2> <p> The OPTIONAL <dfn><code>include_subdomains</code></dfn> member is a boolean that enables this <a>NEL policy</a> for all subdomains of this origin (to an unlimited subdomain depth). If no member named <code>include_subdomains</code> is present in the object, or its value is not <code>true</code>, the <a>NEL policy</a> will not be enabled for subdomains. </p> <p class="note"> To ensure delivery of NEL reports for subdomains, the application should ensure that the Reporting <a>endpoint group</a> is also configured with <code>include_subdomains</code> enabled. If the Reporting policy is not, and there is not a separate Reporting policy for a given subdomain, NEL reports for that subdomain will not be delivered, even if the NEL policy includes the subdomain. </p> </section> <section> <h2>The <code>success_fraction</code> member</h2> <p> The OPTIONAL <dfn><code>success_fraction</code></dfn> member defines the <a>sampling rate</a> that should be applied to reports about <a>successful</a> <a>network requests</a> for this origin. If present, its value MUST be a number between <code>0.0</code> and <code>1.0</code>, inclusive; any other value will result in a parse error. If this member is not present, the user agent will <em>not</em> collect NEL reports about <a>successful</a> <a>network requests</a> for this origin. </p> </section> <section> <h2>The <code>failure_fraction</code> member</h2> <p> The OPTIONAL <dfn><code>failure_fraction</code></dfn> member defines the <a>sampling rate</a> that should be applied to reports about <a>failed</a> <a>network requests</a> for this origin. If present, its value MUST be a number between <code>0.0</code> and <code>1.0</code>, inclusive; any other value will result in a parse error. If this member is not present, the user agent will collect NEL reports about <em>all</em> <a>failed</a> <a>network requests</a> for this origin. </p> </section> <section> <h2>The <code>request_headers</code> member</h2> <p> The OPTIONAL <dfn><code>request_headers</code></dfn> member defines the list of <a data-lt="policy request headers">request headers</a> whose <a data-lt="header name">names</a> and <a data-lt="header value">values</a> will be included in <a>network error reports</a> about this <a>origin</a>. If present, its value MUST be a list of strings. </p> </section> <section> <h2>The <code>response_headers</code> member</h2> <p> The OPTIONAL <dfn><code>response_headers</code></dfn> member defines the list of <a data-lt="policy response headers">response headers</a> whose <a data-lt="header name">names</a> and <a data-lt="header value">values</a> will be included in <a>network error reports</a> about this <a>origin</a>. If present, its value MUST be a list of strings. </p> </section> </section> <section> <h2>Process policy headers</h2> <p> Given a <a>network request</a> (<var>request</var>) and its corresponding <a>response</a> (<var>response</var>), this algorithm extracts a <a>NEL policy</a> for <var>request</var>'s <a>origin</a>, and updates the <a>policy cache</a> accordingly. </p> <ol class="algorithm"> <li> Abort these steps if any of the following conditions are true: <ul> <li> The result of executing the <a>"Is origin potentially trustworthy?"</a> algorithm on <var>request</var>'s <a>origin</a> is <strong>not</strong> <code>Potentially Trustworthy</code>. </li> <li> <var>response</var> does not contain a <a>response header</a> whose name is <code>NEL</code>. </li> </ul> </li> <li> Let <var>origin</var> be <var>request</var>'s <a>origin</a>. </li> <li> Let <var>key</var> be the result of calling <a>determine the network partition key</a>, given <var>request</var>. </li> <li> Let <var>header</var> be the value of the <a>response header</a> whose name is <code>NEL</code>. </li> <li> Let <var>list</var> be the result of executing the algorithm defined in Section 4 of [[HTTP-JFV]] on <var>header</var>. If that algorithm results in an error, or if <var>list</var> is empty, abort these steps. </li> <li> Let <var>item</var> be the first element of <var>list</var>. </li> <li> If <var>item</var> has no member named <a>max_age</a>, or that member's value is not a number, abort these steps. </li> <li> If the value of <var>item</var>'s <a>max_age</a> member is <code>0</code>, then remove any <a>NEL policy</a> from the <a>policy cache</a> whose <a data-lt="policy origin">origin</a> is <var>origin</var>, and skip the remaining steps. </li> <li> If <var>item</var> has no member named <a>report_to</a>, or that member's value is not a string, abort these steps. </li> <li> If <var>item</var> has a member named <a>success_fraction</a>, whose value is not a number in the range 0.0 to 1.0, inclusive, abort these steps. </li> <li> If <var>item</var> has a member named <a>failure_fraction</a>, whose value is not a number in the range 0.0 to 1.0, inclusive, abort these steps. </li> <li> If <var>item</var> has a member named <a>request_headers</a>, whose value is not a list, or if any element of that list is not a string, abort these steps. </li> <li> If <var>item</var> has a member named <a>response_headers</a>, whose value is not a list, or if any element of that list is not a string, abort these steps. </li> <li> <p> Let <var>policy</var> be a new <a>NEL policy</a> whose properties are set as follows: </p> <dl> <dt><a>received IP address</a></dt> <dd> the IP address of the <a>server</a> that the user agent received <var>response</var> from <p class="ednote"> Plumb this through more explicitly in [[FETCH]]. </p> </dd> <dt><a>origin</a></dt> <dd><var>origin</var></dd> <dt><a>subdomains</a> flag</dt> <dd> <code>include</code> if <var>item</var> has a member named <a>include_subdomains</a> whose value is <code>true</code>, <code>exclude</code> otherwise </dd> <dt><a data-lt="policy request headers">request headers</a></dt> <dd> the value of <var>item</var>'s <a>request_headers</a> member </dd> <dt><a data-lt="policy response headers">response headers</a></dt> <dd> the value of <var>item</var>'s <a>response_headers</a> member </dd> <dt><a>reporting group</a></dt> <dd>the value of <var>item</var>'s <a>report_to</a> member</dd> <dt><a>ttl</a></dt> <dd>the value of <var>item</var>'s <a>max_age</a> member</dd> <dt><a>creation</a></dt> <dd>the <a>current wall time</a></dd> <dt><a>successful sampling rate</a></dt> <dd> the value of <var>item</var>'s <a>success_fraction</a> member, if present; <code>0.0</code> otherwise </dd> <dt><a>failure sampling rate</a></dt> <dd> the value of <var>item</var>'s <a>failure_fraction</a> member, if present; <code>1.0</code> otherwise </dd> </dl> </li> <li> If there is already an entry in the <a>policy cache</a> for (<var>key</var>, <var>origin</var>), replace it with <var>policy</var>; otherwise, insert <var>policy</var> into the <a>policy cache</a> for (<var>key</var>, <var>origin</var>). </li> </ol> </section> </section> <section> <h2>Report delivery</h2> <section> <h2>Choose a policy for a request</h2> <p> Given a <a>network request</a> (<var>request</var>), this algorithm determines which <a>NEL policy</a> in the <a>policy cache</a> should be used to generate reports for that <a>network request</a>. </p> <ol class="algorithm"> <li> Let <var>origin</var> be <var>request</var>'s <a>origin</a>. </li> <li> Let <var>key</var> be the result of calling <a>determine the network partition key</a>, given <var>request</var>. </li> <li> If there is an entry in the <a>policy cache</a> for (<var>key</var>, <var>origin</var>): <ol> <li>Let <var>policy</var> be that entry.</li> <li>If <var>policy</var> is not <a>expired</a>, return it.</li> </ol> </li> <li> For each <var>parent origin</var> that is a <a>superdomain match</a> of <var>origin</var>: <ol> <li> If there is an entry in the <a>policy cache</a> for (<var>key</var>, <var>parent origin</var>): <ol> <li>Let <var>policy</var> be that entry.</li> <li> If <var>policy</var> is not <a>expired</a>, and its <a>subdomains</a> flag is <code>include</code>, return it. </li> </ol> </li> </ol> </li> <li> Return <code>no policy</code>. </li> </ol> </section> <section> <h2>Extract request headers</h2> <p> Given a <a>network request</a> (<var>request</var>) and a <a>NEL policy</a> (<var>policy</var>), this algorithm extracts header values from the request as instructed by the policy. </p> <ol class="algorithm"> <li> Let <var>headers</var> be a new empty ECMAScript object. </li> <li> For each <var>header name</var> in <var>policy</var>'s <a data-lt="policy request headers">request headers</a> list: <ol> <li> If <var>request</var>'s <a data-lt="request header list">header list</a> does not <a data-lt="header list contains">contain</a> <var>header name</var>, skip to the next <var>header name</var> in the list. </li> <li> Let <var>values</var> be an empty ECMAScript list. </li> <li> For each <var>header</var> in <var>request</var>'s <a data-lt="request header list">header list</a> whose <a data-lt="header name">name</a> is <var>header name</var>, append <var>header</var>'s <a data-lt="header value">value</a> to <var>values</var>. </li> <li> Add a new property to <var>headers</var> whose name is <var>header name</var> and whose value is <var>values</var>. </li> </ol> </li> <li> Return <var>headers</var>. </li> </ol> </section> <section> <h2>Extract response headers</h2> <p> Given a <a>response</a> (<var>response</var>) and a <a>NEL policy</a> (<var>policy</var>), this algorithm extracts header values from the response as instructed by the policy. </p> <ol class="algorithm"> <li> Let <var>headers</var> be a new empty ECMAScript object. </li> <li> For each <var>header name</var> in <var>policy</var>'s <a data-lt="policy response headers">response headers</a> list: <ol> <li> If <var>response</var>'s <a data-lt="response header list">header list</a> does not <a data-lt="header list contains">contain</a> <var>header name</var>, skip to the next <var>header name</var> in the list. </li> <li> Let <var>values</var> be an empty ECMAScript list. </li> <li> For each <var>header</var> in <var>response</var>'s <a data-lt="response header list">header list</a> whose <a data-lt="header name">name</a> is <var>header name</var>, append <var>header</var>'s <a data-lt="header value">value</a> to <var>values</var>. </li> <li> Add a new property to <var>headers</var> whose name is <var>header name</var> and whose value is <var>values</var>. </li> </ol> </li> <li> Return <var>headers</var>. </li> </ol> </section> <section> <h2 id="generate-a-network-error-report" data-dfn-type="dfn" data-export>Generate a network error report</h2> <p> Given a <a>network request</a> (<var>request</var>) and its corresponding <a>response</a> (<var>response</var>), this algorithm generates a report about <var>request</var> if instructed to by any matching <a>NEL policy</a>, and returns the report and the NEL policy. Otherwise this algorithm returns null. </p> <ol class="algorithm"> <li> If the result of executing the <a>"Is origin potentially trustworthy?"</a> algorithm on <var>request</var>'s <a>origin</a> is <strong>not</strong> <code>Potentially Trustworthy</code>, return null. </li> <li> Let <var>origin</var> be <var>request</var>'s <a>origin</a>. </li> <li> Let <var>policy</var> be the result of executing <a href="#choose-a-policy-for-a-request"></a> on <var>request</var>. If <var>policy</var> is <code>no policy</code>, return null. </li> <li> Determine the active sampling rate for this request: <ul> <li> If <var>request</var> <a>succeeded</a>, let <var>sampling rate</var> be <var>policy</var>'s <a>successful sampling rate</a>. </li> <li> If <var>request</var> <a>failed</a>, let <var>sampling rate</var> be <var>policy</var>'s <a>failure sampling rate</a>. </li> </ul> </li> <li> Decide whether or not to report on this request. Let <var>roll</var> be a random number between 0.0 and 1.0, inclusive. If <var>roll</var> ≥ <var>sampling rate</var>, return null. </li> <li> Let <var>report body</var> be a new ECMAScript object with the following properties: [[ECMA-262]] <dl> <dt><code>sampling_fraction</code></dt> <dd><var>sampling rate</var></dd> <dt><code>elapsed_time</code></dt> <dd> The elapsed number of milliseconds between the start of the resource fetch and when it was completed or aborted by the user agent. </dd> <dt><code>phase</code></dt> <dd> If <var>request</var> <a>failed</a>, the <a data-lt="type-phase">phase</a> of its <a>network error</a>. If <var>request</var> <a>succeeded</a>, <code>"application"</code>. </dd> <dt><code>type</code></dt> <dd> If <var>request</var> <a>failed</a>, the <a>type</a> of its <a>network error</a>. If <var>request</var> <a>succeeded</a>, <code>"ok"</code>. </dd> </dl> </li> <li> If <var>report body</var>'s <code>phase</code> property is not <code>dns</code>, append the following properties to <var>report body</var>: <dl> <dt><code>server_ip</code></dt> <dd>The IP address of the <a>server</a> to which the user agent sent the request, if available. Otherwise, an empty string. <ul> <li>A host identified by an IPv4 address is represented in dotted-decimal notation (a sequence of four decimal numbers in the range 0 to 255, separated by "."). [[RFC1123]]</li> <li>A host identified by an IPv6 address is represented as an ordered list of eight 16-bit pieces (a sequence of `x:x:x:x:x:x:x:x`, where the 'x's are one to four hexadecimal digits of the eight 16-bit pieces of the address). [[RFC4291]] </li> </ul> </dd> <dt><code>protocol</code></dt> <dd> The <a>network protocol</a> used to fetch the resource as identified by the ALPN Protocol ID, if available. Otherwise, <code>""</code>. </dd> </dl> </li> <li> If <var>report body</var>'s <code>phase</code> property is not <code>dns</code> or <code>connection</code>, append the following properties to <var>report body</var>: <dl> <dt><code>referrer</code></dt> <dd> <var>request</var>'s referrer, as determined by the <a>referrer policy</a> associated with its <a>client</a>. </dd> <dt><code>method</code></dt> <dd><var>request</var>'s <a>request method</a>.</dd> <dt><code>request_headers</code></dt> <dd> The result of executing <a href="#extract-request-headers"></a> on <var>request</var> and <var>policy</var>. </dd> <dt><code>response_headers</code></dt> <dd> The result of executing <a href="#extract-response-headers"></a> on <var>response</var> and <var>policy</var>. </dd> <dt><code>status_code</code></dt> <dd> The <a>status code</a> of the HTTP response, if available. Otherwise, <code>0</code>. </dd> </dl> </li> <li> If <var>origin</var> is not equal to <var>policy</var>'s <a data-lt="policy origin">origin</a>, <var>policy</var>'s <a>subdomains</a> flag is <code>include</code>, and <var>report body</var>'s <code>phase</code> property is not <code>dns</code>, return null. <p class="note"> This step ensures that <a data-lt="subdomains">subdomain</a> <a>NEL policies</a> can only be used to generate reports about subdomains of the <a>policy origin</a> during the <a>DNS resolution</a> <a>phase</a> of a <a>request</a>. See <a href="#privacy-considerations"></a> for more details. </p> </li> <li> If <var>report body</var>'s <code>phase</code> property is not <code>dns</code>, and <var>report body</var>'s <code>server_ip</code> property is non-empty and not equal to <var>policy</var>'s <a>received IP address</a>: <ol> <li> Set <var>report body</var>'s <code>phase</code> to <code>dns</code>. </li> <li> Set <var>report body</var>'s <code>type</code> to <code>dns.address_changed</code>. </li> <li> Clear <var>report body</var>'s <code>request_headers</code>, <code>response_headers</code>, <code>status_code</code>, and <code>elapsed_time</code> properties. </li> <li> Assert: All fields in <var>report body</var> that are derived from information not available during <a>DNS resolution</a> have been cleared. </li> </ol> <p class="note"> This step "downgrades" a NEL report if the IP addresses of the <a>server</a> and the <a data-lt="NEL policy">policy</a> don't match. This is a privacy protection, ensuring that NEL reports are only sent to the owner of the service that the report describes. If the IP addresses don't match, then the user agent can only verify that the <a>NEL policy</a> was sent by the owner of the <a>origin</a>'s <a>domain name</a>; it cannot verify that the policy was sent by the owner of the <a>server</a> this <a>domain name</a> resolves to. We therefore downgrade the report to only contain information about <a>DNS resolution</a>. See <a href="#privacy-considerations"></a> and <a href="#origins-with-multiple-ip-addresses"></a> for more details. </p> </li> <li> If <var>policy</var> is <a>stale</a>, then delete <var>policy</var> from the <a>policy cache</a>. </li> <li> Return <var>report body</var> and <var>policy</var>. </li> </ol> </section> <section> <h2 id="deliver-a-network-report" data-dfn-type="dfn" data-export>Deliver a network report</h2> <p> Given a ECMAScript object (<var>report body</var>, usually returned from <a href="#generate-a-network-error-report">Generate a network error report</a> and then augmented by the calling specification) and its matching <a>NEL policy</a> (<var>policy</var>) and <a>network request</a> (<var>request</var>), this algorithm queues the report for delivery. </p> <ol class="algorithm"> <li> <p>Let <var>url</var> be <var>request</var>'s URL.</p> </li> <li> <p>Clear <var>url</var>'s <a>fragment</a>.</p> </li> <li> <p> If <var>report body</var>'s <code>phase</code> property is <code>dns</code> or <code>connection</code>:</p> <ol> <li> <p>Clear <var>url</var>'s <a>path</a> and <a>query</a>.</p> </li> </ol> </li> <li> <p><a>Generate a network report</a> given these parameters:</p> <dl> <dt>type</dt> <dd><code>network-error</code></dd> <dt>data</dt> <dd><var>report body</var></dd> <dt>endpoint group</dt> <dd><var>policy</var>'s <a>reporting group</a></dd> <dt>url</dt> <dd>The result of running the <a>URL serializer</a> on <var>url</var>. </dd> </dl> </li> </ol> </section> </section> <section> <h2>Predefined network error types</h2> <p> There are several predefined <a>network error</a> <a>types</a>. </p> <p> The user agent MAY extend this list with custom <a>network error</a> <a>types</a> — e.g. to accommodate new protocols, or more detailed error descriptions of existing ones. When doing so, the user agent SHOULD follow the dot-delimited pattern (<code>[group].[optional-subgroup].[error-name]</code>) for the <a>type</a> names to facilitate simple and consistent processing of the error reports — e.g. the collector may provide aggregation by category and/or one or multiple subgroups. </p> <section> <h2>DNS resolution errors</h2> <p> All of the <a>network errors</a> in this section occur during <a>DNS resolution</a>, and therefore have a <a data-lt="type-phase">phase</a> of <code>dns</code>. </p> <dl> <dt><code>dns.unreachable</code></dt> <dd>DNS server is unreachable</dd> <dt><code>dns.name_not_resolved</code></dt> <dd>DNS server responded but is unable to resolve the address</dd> <dt><code>dns.failed</code></dt> <dd>Request to the DNS server failed due to reasons not covered by previous errors</dd> <dt><code>dns.address_changed</code></dt> <dd> Indicates that the resolved IP address for a request's <a>origin</a> has changed since the corresponding <a>NEL policy</a> was received </dd> </dl> </section> <section> <h2>Secure connection establishment errors</h2> <p> All of the <a>network errors</a> in this section occur during <a>secure connection establishment</a>, and therefore have a <a data-lt="type-phase">phase</a> of <code>connection</code>. </p> <dl> <dt><code>tcp.timed_out</code></dt> <dd>TCP connection to the server timed out</dd> <dt><code>tcp.closed</code></dt> <dd>The TCP connection was closed by the server</dd> <dt><code>tcp.reset</code></dt> <dd>The TCP connection was reset</dd> <dt><code>tcp.refused</code></dt> <dd>The TCP connection was refused by the server</dd> <dt><code>tcp.aborted</code></dt> <dd>The TCP connection was aborted</dd> <dt><code>tcp.address_invalid</code></dt> <dd>The IP address is invalid</dd> <dt><code>tcp.address_unreachable</code></dt> <dd>The IP address is unreachable</dd> <dt><code>tcp.failed</code></dt> <dd>The TCP connection failed due to reasons not covered by previous errors</dd> </dl> <dl> <dt><code>tls.version_or_cipher_mismatch</code></dt> <dd>The TLS connection was aborted due to version or cipher mismatch</dd> <dt><code>tls.bad_client_auth_cert</code></dt> <dd>The TLS connection was aborted due to invalid client certificate</dd> <dt><code>tls.cert.name_invalid</code></dt> <dd>The TLS connection was aborted due to invalid name</dd> <dt><code>tls.cert.date_invalid</code></dt> <dd>The TLS connection was aborted due to invalid certificate date</dd> <dt><code>tls.cert.authority_invalid</code></dt> <dd>The TLS connection was aborted due to invalid issuing authority</dd> <dt><code>tls.cert.invalid</code></dt> <dd>The TLS connection was aborted due to invalid certificate</dd> <dt><code>tls.cert.revoked</code></dt> <dd>The TLS connection was aborted due to revoked server certificate</dd> <dt><code>tls.cert.pinned_key_not_in_cert_chain</code></dt> <dd>The TLS connection was aborted due to a key pinning error</dd> <dt><code>tls.protocol.error</code></dt> <dd>The TLS connection was aborted due to a TLS protocol error</dd> <dt><code>tls.failed</code></dt> <dd>The TLS connection failed due to reasons not covered by previous errors</dd> </dl> </section> <section> <h2>Transmission of request and response errors</h2> <p> All of the <a>network errors</a> in this section occur during the <a>transmission of request and response</a>, and therefore have a <a data-lt="type-phase">phase</a> of <code>application</code>. </p> <dl> <dt><code>http.error</code></dt> <dd> The user agent successfully received a response, but it had a <a>4xx</a> or <a>5xx</a> status code </dd> <dt><code>http.protocol.error</code></dt> <dd>The connection was aborted due to an HTTP protocol error</dd> <dt><code>http.response.invalid</code></dt> <dd>Response is empty, has a content-length mismatch, has improper encoding, and/or other conditions that prevent user agent from processing the response</dd> <dt><code>http.response.redirect_loop</code></dt> <dd>The request was aborted due to a detected redirect loop</dd> <dt><code>http.failed</code></dt> <dd>The connection failed due to errors in HTTP protocol not covered by previous errors</dd> </dl> <dl> <dt><code>abandoned</code></dt> <dd>User aborted the resource fetch before it is complete</dd> <dt><code>unknown</code></dt> <dd>error type is unknown</dd> </dl> </section> </section> <section> <h2>Examples</h2> <section> <h2>Sample Policy Definitions</h2> <pre class="example"> > GET / HTTP/1.1 > Host: example.com < HTTP/1.1 200 OK < ... < Report-To: {"group": "network-errors", "max_age": 2592000, "endpoints": [{"url": "https://example.com/upload-reports"}]} < NEL: {"report_to": "network-errors", "max_age": 2592000} </pre> <p> This <code>NEL</code> header defines a <a>NEL policy</a>, instructing the user agent to report network errors about <code>example.com</code> to the <a>endpoint group</a> named <code>network-errors</code>. The policy applies for 2592000 seconds (30 days). </p> <p>Note that above registration will only succeed if the response is communicated from a <a>potentially trustworthy origin</a>.</p> <pre class="example"> > GET / HTTP/1.1 > Host: example.com < HTTP/1.1 200 OK < ... < NEL: {"max_age": 0} </pre> <p> This <code>NEL</code> header instructs the user agent to remove any existing <a>NEL policy</a> for <code>example.com</code>. </p> </section> <section> <h2>Sample Network Error Reports</h2> <p> This section contains example <a>network error</a> <a>reports</a> the user agent might queue when a network error is encountered for an <a>origin</a> with a registered <a>NEL policy</a>. We show the full report payload that would be created by the [[REPORTING]] API when uploading the report; the payload's <code>body</code> field contains the <a>network error</a> <a>report body</a>. </p> <pre class="example"> { "age": 0, "type": "network-error", "url": "https://www.example.com/", "body": { "sampling_fraction": 0.5, "referrer": "http://example.com/", "server_ip": "2001:DB8:0:0:0:0:0:42", "protocol": "h2", "method": "GET", "request_headers": {}, "response_headers": {}, "status_code": 200, "elapsed_time": 823, "phase": "application", "type": "http.protocol.error" } } </pre> <p> This report indicates that the user agent attempted to navigate from <code>example.com</code> to <code>www.example.com</code>, which successfully resolved to <code>2001:DB8::42</code>. However, while the user agent received a <a>200 response</a> from the server via the HTTP/2 (<code>h2</code>) protocol, it encountered a protocol error in the exchange and was forced to abandon the navigation. The user agent aborted the navigation 823 milliseconds after it started. Finally, the user agent sent this report immediately after the network error was encountered – i.e. the report age is 0. </p> <pre class="example"> { "age": 0, "type": "network-error", "url": "https://widget.com/thing.js", "body": { "sampling_fraction": 1.0, "referrer": "https://www.example.com/", "server_ip": "", "protocol": "", "method": "GET", "request_headers": {}, "response_headers": {}, "status_code": 0, "elapsed_time": 143, "phase": "dns", "type": "dns.name_not_resolved" } } </pre> <p> The above report indicates that the user agent attempted to fetch <code>https://widget.com/thing.js</code> from <code>https://www.example.com/</code>. However, the user agent was unable to resolve the DNS name (<code>widget.com</code>) and the request was aborted by the user agent after 143 milliseconds. Because a previous request to <code>widget.com</code> delivered a valid <a>NEL policy</a>, the user agent generates a <a>network error</a> <a>report</a> for this request. The report was uploaded immediately after the <a>network error</a> was encountered – i.e. the report age is 0. </p> </section> <section> <h2>DNS misconfiguration</h2> <pre class="example"> > GET / HTTP/1.1 > Host: example.com < HTTP/1.1 200 OK < ... < Report-To: {"group": "network-errors", "max_age": 2592000, "endpoints": [{"url": "https://example.com/upload-reports"}]} < NEL: {"report_to": "network-errors", "max_age": 2592000, "include_subdomains": true} </pre> <p> This <code>NEL</code> header allows the owner of <code>example.com</code> to detect when they have misconfigured their DNS servers — for instance, when they have forgotten to add a new resource record resolving <code>new-subdomain.example.com</code> to an IP address. If a user agent tries to make a request to <code>new-subdomain.example.com</code>, it might generate the following report: </p> <pre class="example"> { "age": 0, "type": "network-error", "url": "https://new-subdomain.example.com/", "body": { "sampling_fraction": 1.0, "server_ip": "", "protocol": "http/1.1", "method": "GET", "request_headers": {}, "response_headers": {}, "status_code": 0, "elapsed_time": 48, "phase": "dns", "type": "dns.name_not_resolved" } } </pre> </section> <section> <h2>Monitoring cache validation</h2> <pre class="example"> > GET / HTTP/1.1 > Host: example.com < HTTP/1.1 200 OK < ... < Report-To: {"group": "network-errors", "max_age": 2592000, "endpoints": [{"url": "https://example.com/upload-reports"}]} < NEL: {"report_to": "network-errors", "max_age": 2592000, "success_fraction": 1.0, "request_headers": ["If-None-Match"], "response_headers": ["ETag"]} < ETag: 01234abcd </pre> <p> In this example, the owner of <code>example.com</code> uses <a><code>ETag</code></a> response headers to identify different versions of the resources hosted on the server. User agents can then use <a><code>If-None-Match</code></a> request headers to inform the server which version of a resource is presently cached client-side, allowing the server to avoid generating and sending the content of the resource if the client's existing copy is up to date. </p> <p> By including <a>request_headers</a> and <a>response_headers</a> fields in the <code>NEL</code> header for this domain, the browser will include a copy of the <a><code>If-None-Match</code></a> request header and <a><code>ETag</code></a> response header in any NEL report that it creates for that request, allowing the site owner to track the effectiveness of their caching policies. </p> <p> Given the above, consider the following sequence of events: </p> <ol> <li> <p> The user agent sends a <a>request</a> to <code>example.com</code>, and receives a successful <a>response</a> from the <a>server</a>, with an <a><code>ETag</code></a> header indicating the version of the resource. The user agent will generate the following NEL report: </p> <pre class="example"> { "age": 0, "type": "network-error", "url": "https://example.com/", "body": { "sampling_fraction": 1.0, "server_ip": "192.0.2.1", "protocol": "http/1.1", "method": "GET", "request_headers": {}, "response_headers": { "ETag": ["01234abcd"] }, "status_code": 200, "elapsed_time": 1392, "phase": "application", "type": "ok" } } </pre> </li> <li> <p> Some time later, the user agent sends another <a>request</a> to <code>example.com</code>. The user agent still has a copy of the original resource in its local cache, and includes its version in a <a><code>If-None-Match</code></a> request header. The server checks this version, notices that it is still current, and sends a `304` response informing the user agent that its cached copy of the resource is still valid. The user agent will generate the following report: </p> <pre class="example"> { "age": 0, "type": "network-error", "url": "https://example.com/", "body": { "sampling_fraction": 1.0, "server_ip": "192.0.2.1", "protocol": "http/1.1", "method": "GET", "request_headers": { "If-None-Match": ["01234abcd"] }, "response_headers": { "ETag": ["01234abcd"] }, "status_code": 304, "elapsed_time": 45, "phase": "application", "type": "ok" } } </pre> </li> <li> <p> Even later, the user agent sends yet another <a>request</a> to <code>example.com</code>. The user agent still has the same copy of the resource in its local cache, and includes its version in a <a><code>If-None-Match</code></a> request header, as in the previous example. However, this time the server notices that there is a new version of the resource available. It generates the content of this resource, and sends it to the client, with the new version encoded in a new <a><code>ETag</code></a> response header value. The user agent will generate the following report: </p> <pre class="example"> { "age": 0, "type": "network-error", "url": "https://example.com/", "body": { "sampling_fraction": 1.0, "server_ip": "192.0.2.1", "protocol": "http/1.1", "method": "GET", "request_headers": { "If-None-Match": ["01234abcd"] }, "response_headers": { "ETag": ["56789ef01"] }, "status_code": 200, "elapsed_time": 935, "phase": "application", "type": "ok" } } </pre> </li> </ol> </section> <section> <h2>Origins with multiple IP addresses</h2> <p> For <a>origins</a> whose <a>domain name</a> resolves to multiple IP addresses, NEL will sometimes "downgrade" an error report, providing less information about the cause of the error, since it cannot verify that the owner of the <a>origin</a> is the same as the owner of the <a>server</a> handling the <a>request</a>. </p> <p> As an example, assume that <code>example.com</code> is handled by three <a>servers</a>, each with a different IP address. The owner of the service configures DNS to resolve <code>example.com</code> to <code>192.0.2.1</code>, <code>192.0.2.2</code>, and <code>192.0.2.3</code>, and relies on user agents to balance their requests across these three IP addresses. The service owner delivers the following <a>NEL policy</a>: </p> <pre class="example"> > GET / HTTP/1.1 > Host: example.com < HTTP/1.1 200 OK < ... < Report-To: {"group": "network-errors", "max_age": 2592000, "endpoints": [{"url": "https://example.com/upload-reports"}]} < NEL: {"report_to": "network-errors", "max_age": 2592000, "success_fraction": 1.0, "failure_fraction": 1.0} </pre> <p> Given the above, consider the following sequence of events: </p> <ol> <li> <p> The user agent sends a <a>request</a> to <code>192.0.2.1</code>, and receives a successful <a>response</a> from the <a>server</a>. This response includes the above <a>NEL policy</a>, and the user agent sets the policy's <a>received IP address</a> to <code>192.0.2.1</code>. Since the <a>received IP address</a> matches the <a>server</a>'s IP address (which it must for any successful request), it generates the following NEL report: </p> <pre class="example"> { "age": 0, "type": "network-error", "url": "https://example.com/", "body": { "sampling_fraction": 1.0, "server_ip": "192.0.2.1", "protocol": "http/1.1", "method": "GET", "request_headers": {}, "response_headers": {}, "status_code": 200, "elapsed_time": 57, "phase": "application", "type": "ok" } } </pre> </li> <li> <p> The user agent sends a new <a>request</a> to <code>192.0.2.2</code>, and receives another successful <a>response</a>. This response also includes the <a>NEL policy</a>, and the user agent updates the policy's <a>received IP address</a> to <code>192.0.2.2</code>. Since the <a>received IP address</a> matches the <a>server</a>'s IP address (which it must for any successful request), it generates the following NEL report: </p> <pre class="example"> { "age": 0, "type": "network-error", "url": "https://example.com/", "body": { "sampling_fraction": 1.0, "server_ip": "192.0.2.2", "protocol": "http/1.1", "method": "GET", "request_headers": {}, "response_headers": {}, "status_code": 200, "elapsed_time": 34, "phase": "application", "type": "ok" } } </pre> </li> <li> <p> The user agent then tries to send a <a>request</a> to <code>192.0.2.3</code>, but isn't able to establish a connection to the server. The user agent still has the <a>NEL policy</a> in the <a>policy cache</a>, and would ideally use this policy to generate a <code>tcp.timed_out</code> report about the <a>failed</a> <a>network request</a>. However, the because policy's <a>received IP address</a> (<code>192.0.2.2</code>) doesn't match the IP address that this <a>request</a> was sent to, the user agent cannot verify that the server at <code>192.0.2.3</code> is actually owned by the owners of <code>example.com</code>. The user agent must therefore downgrade the report to <code>dns.address_changed</code>: </p> <pre class="example"> { "age": 0, "type": "network-error", "url": "https://example.com/", "body": { "sampling_fraction": 1.0, "server_ip": "192.0.2.3", "protocol": "http/1.1", "method": "GET", "request_headers": {}, "response_headers": {}, "status_code": 0, "elapsed_time": 0, "phase": "dns", "type": "dns.address_changed" } } </pre> </li> <li> <p> The user agent then tries to send another <a>request</a> to <code>192.0.2.1</code>, but once again isn't able to establish a connection to the server. Even though the user agent received the <a>NEL policy</a> from <code>192.0.2.1</code> at some point in the past, the policy's <a>received IP address</a> only records where it was <em>most recently</em> received from — in this case, <code>192.0.2.2</code>. The user agent must therefore downgrade the report to <code>dns.address_changed</code>: </p> <pre class="example"> { "age": 0, "type": "network-error", "url": "https://example.com/", "body": { "sampling_fraction": 1.0, "server_ip": "192.0.2.1", "protocol": "http/1.1", "method": "GET", "request_headers": {}, "response_headers": {}, "status_code": 0, "elapsed_time": 0, "phase": "dns", "type": "dns.address_changed" } } </pre> </li> </ol> </section> </section> <section> <h2>Use cases</h2> <section> <h2>Reporting of Navigation Failures</h2> <p>A navigation request initiated by the user (e.g. via a click on a link, direct input via the location bar, script-initiated due to user interaction, etc.) may fail due any number of connectivity reasons: DNS failure, TCP error, TLS protocol violation, and so on. These errors may be caused by network misconfiguration, transient routing issues, server downtime, malware or other attacks against the user, etc.</p> <p>In such cases the destination host is often left unaware of the failed navigation since, by definition, it cannot see the request reach its infrastructure and it is unable to investigate the problem. To address this, the host can register an <a>NEL policy</a> with the user agent, which specifies where reports of such failures should be delivered such that they can be investigated.</p> </section> <section> <h2>Reporting of First-party Subresource Fetch Failures</h2> <p>A typical application requires dozens of resources, the fetching of which is typically initiated via HTML, CSS, or JavaScript. The application requesting such resources can observe failures of most such fetches (e.g. via `onerror` callbacks), but it does not have access to the detailed network error report of why the failure has occurred - e.g. DNS failure, TCP error, TLS protocol violation, etc.</p> <p>To address this, the application can register relevant <a>NEL policies</a> with the user agent for the first-party hosts from which the subresources are being fetched. Then, if such a policy is present and a network error is encountered for a resource from an <a>origin</a> with a registered <a>NEL policy</a>, the user agent will report the detailed network error report and enable the application developers to investigate the error.</p> </section> <section> <h2>Reporting of Third-party Subresource Fetch Failures</h2> <p>In the case where a resource is embedded by a third party, the provider of the resource is often unable to instrument and observe the failure. For example, if `example.com` embeds a `widget.com/thing.js` resource on its site, and the user visiting `example.com` fails to fetch such resource due to a network error, the `widget.com` host is both unaware of the failure and unable to detect it.</p> <p>To address this, `widget.com` can register an NEL policy for its host. Then, if such policy is present and a network error is encountered while fetching a resource — regardless of whether it is being requested from a first-party or third-party origin — from the <a>origin</a> with a registered <a>NEL policy</a>, the user agent will report the network error and enable the provider to investigate the error.</p> </section> </section> <section> <h2>Privacy Considerations</h2> <p> NEL provides network error reports that could expose new information about the user's network configuration. For example, an attacker could abuse NEL reporting to probe the user's network configuration, or to scan for servers on the user's internal network. Also, similar to HSTS, HPKP, and pinned CSP policies, the stored <a>NEL policy</a> could be used as a "supercookie" by setting a distinct policy with a custom (per-user) reporting URI to act as an identifier in combination with (or instead of) HTTP cookies. </p> <p> To mitigate some of the above risks, NEL registration is restricted to <a>potentially trustworthy origins</a>, and delivery of network error reports is similarly restricted to <a>potentially trustworthy origins</a>. This disallows a transient HTTP MITM from trivially abusing NEL as a persistent tracker. </p> <p> Additionally, the NEL <a>policy cache</a> is partitioned using the <a>network partition key</a>, so that a <a>NEL policy</a> stored for a site in one embedding context will not be used in a different context (for instance, when embedded by a different top-level site.) </p> <p> NEL is intended to augment existing server-side monitoring. NEL reports should only be sent to the owner of the service being requested. For errors that occur during <a>DNS resolution</a>, NEL reports are only generated when the <a>NEL policy</a> was received from the owner of the <a>domain namespace tree</a> that contains the <a>policy origin</a>. For errors that occur during <a>secure connection establishment</a> or <a>transmission of request and response</a>, NEL reports are only generated when the <a>NEL policy</a> was received from the owner of the <a>server</a> that the <a>request</a> was sent to. </p> <p> This rationale explains the treatment of the <a>received IP address</a> and <a>subdomains</a> flag of a <a>NEL policy</a>. By checking that the policy's <a>received IP address</a> matches the IP address of the <a>server</a>, NEL extends the trust boundary of the policy to include not just the policy's <a data-lt="policy origin">origin</a>, but also the specific server that the user agent is communicating with. This helps prevent (for instance) DNS rebinding attacks, where an attacker delivers a long-lived <a>NEL policy</a> from a server that they own, and then changes their name servers to resolve the <a>policy origin</a> to a server they don't control. Without the <a>received IP address</a> verification, this would cause user agents to send reports about the second server to the attacker. </p> <p> Similarly, <a data-lt="subdomains">subdomain</a> <a>NEL policies</a> are limited, and can only be used to generate reports about subdomains of the <a>policy origin</a> during the <a>DNS resolution</a> <a>phase</a> of a <a>request</a>. During this <a>phase</a>, there is no <a>server</a> to verify ownership of, and the fact that the policy was received from a superdomain of the <a>request</a>'s <a>origin</a> is enough to establish ownership of the error. This allows the owners of a particular portion of the <a>domain namespace tree</a> to use NEL to detect <a href="#dns-misconfiguration"></a> errors, while preventing them from using malicious DNS entries to collect information about servers they don't control. </p> <p> To prevent information leakage, NEL reports about a <a>request</a> do not contain any information that is not visible to the <a>server</a> when processing the <a>request</a>. For errors during <a>DNS resolution</a>, a NEL report only contains information available from DNS itself. This prevents <a>servers</a> from abusing NEL to collect more information about their users than they already have access to. </p> <p class="note"> As an example, NEL reports specifically do not contain any information about which DNS <a>resolver</a> was used to resolve a <a>request</a>'s <a>domain name</a> into an IP address. </p> <p>In addition to above restrictions, the user agents MUST:</p> <ul> <li>Clear the stored NEL policies when the user clears their browsing data (cookies, site data, history, etc).</li> <li>Refuse to process Set-Cookie response headers when delivering network error reports.</li> </ul> <p>When deploying NEL the developer SHOULD consider privacy implications of NEL reports delivered to the specified collectors. For example, reports may contain URLs with sensitive data (e.g. "Capability URLs") that may need special precautions (see [[CAPABILITY-URLS]]), and may require the developer to operate their own NEL collectors to prevent reporting of such URLs to third parties.</p> </section> <section> <h2>IANA Considerations</h2> <p>The permanent message header field registry should be updated with the following registrations ([[RFC3864]]):</p> <section> <h2><code>NEL</code></h2> <dl> <dt>Header field name</dt> <dd><code>NEL</code></dd> <dt>Applicable protocol</dt> <dd>http</dd> <dt>Status</dt> <dd>standard</dd> <dt>Author/Change controller</dt> <dd>W3C</dd> <dt>Specification document</dt> <dd>This specification (see <a>NEL</a> response header)</dd> </dl> </section> </section> <section class="appendix"> <h2>Acknowledgments</h2> <p>This document reuses text from the [[CSP]] and [[RFC6797]] specification, as permitted by the licenses of those specifications. Additionally, sincere thanks to Julia Tuttle, Chris Bentzel, Todd Reifsteck, Aaron Heady, and Mark Nottingham for their helpful comments and contributions to this work.</p> </section> </body> </html>

CINXE.COM

Network Error Logging