CINXE.COM

<!DOCTYPE html> <html lang="en"> <head> <meta charset="utf-8"> <meta name="viewport" content="width=device-width, initial-scale=1.0"> <meta name="color-scheme" content="light dark"> <title>PEP 485 – A Function for testing approximate equality | peps.python.org</title> <link rel="shortcut icon" href="../_static/py.png"> <link rel="canonical" href="https://peps.python.org/pep-0485/"> <link rel="stylesheet" href="../_static/style.css" type="text/css"> <link rel="stylesheet" href="../_static/mq.css" type="text/css"> <link rel="stylesheet" href="../_static/pygments.css" type="text/css" media="(prefers-color-scheme: light)" id="pyg-light"> <link rel="stylesheet" href="../_static/pygments_dark.css" type="text/css" media="(prefers-color-scheme: dark)" id="pyg-dark"> <link rel="alternate" type="application/rss+xml" title="Latest PEPs" href="https://peps.python.org/peps.rss"> <meta property="og:title" content='PEP 485 – A Function for testing approximate equality | peps.python.org'> <meta property="og:description" content="This PEP proposes the addition of an isclose() function to the standard library math module that determines whether one value is approximately equal or “close” to another value."> <meta property="og:type" content="website"> <meta property="og:url" content="https://peps.python.org/pep-0485/"> <meta property="og:site_name" content="Python Enhancement Proposals (PEPs)"> <meta property="og:image" content="https://peps.python.org/_static/og-image.png"> <meta property="og:image:alt" content="Python PEPs"> <meta property="og:image:width" content="200"> <meta property="og:image:height" content="200"> <meta name="description" content="This PEP proposes the addition of an isclose() function to the standard library math module that determines whether one value is approximately equal or “close” to another value."> <meta name="theme-color" content="#3776ab"> </head> <body> <svg xmlns="http://www.w3.org/2000/svg" style="display: none;"> <symbol id="svg-sun-half" viewBox="0 0 24 24" pointer-events="all"> <title>Following system colour scheme</title> <svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"> <circle cx="12" cy="12" r="9"></circle> <path d="M12 3v18m0-12l4.65-4.65M12 14.3l7.37-7.37M12 19.6l8.85-8.85"></path> </svg> </symbol> <symbol id="svg-moon" viewBox="0 0 24 24" pointer-events="all"> <title>Selected dark colour scheme</title> <svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"> <path stroke="none" d="M0 0h24v24H0z" fill="none"></path> <path d="M12 3c.132 0 .263 0 .393 0a7.5 7.5 0 0 0 7.92 12.446a9 9 0 1 1 -8.313 -12.454z"></path> </svg> </symbol> <symbol id="svg-sun" viewBox="0 0 24 24" pointer-events="all"> <title>Selected light colour scheme</title> <svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"> <circle cx="12" cy="12" r="5"></circle> <line x1="12" y1="1" x2="12" y2="3"></line> <line x1="12" y1="21" x2="12" y2="23"></line> <line x1="4.22" y1="4.22" x2="5.64" y2="5.64"></line> <line x1="18.36" y1="18.36" x2="19.78" y2="19.78"></line> <line x1="1" y1="12" x2="3" y2="12"></line> <line x1="21" y1="12" x2="23" y2="12"></line> <line x1="4.22" y1="19.78" x2="5.64" y2="18.36"></line> <line x1="18.36" y1="5.64" x2="19.78" y2="4.22"></line> </svg> </symbol> </svg> <script> document.documentElement.dataset.colour_scheme = localStorage.getItem("colour_scheme") || "auto" </script> <section id="pep-page-section"> <header> <h1>Python Enhancement Proposals</h1> <ul class="breadcrumbs"> <li><a href="https://www.python.org/" title="The Python Programming Language">Python</a> » </li> <li><a href="../pep-0000/">PEP Index</a> » </li> <li>PEP 485</li> </ul> <button id="colour-scheme-cycler" onClick="setColourScheme(nextColourScheme())"> <svg aria-hidden="true" class="colour-scheme-icon-when-auto"><use href="#svg-sun-half"></use></svg> <svg aria-hidden="true" class="colour-scheme-icon-when-dark"><use href="#svg-moon"></use></svg> <svg aria-hidden="true" class="colour-scheme-icon-when-light"><use href="#svg-sun"></use></svg> <span class="visually-hidden">Toggle light / dark / auto colour theme</span> </button> </header> <article> <section id="pep-content"> <h1 class="page-title">PEP 485 – A Function for testing approximate equality</h1> <dl class="rfc2822 field-list simple"> <dt class="field-odd">Author<span class="colon">:</span></dt> <dd class="field-odd">Christopher Barker <PythonCHB at gmail.com></dd> <dt class="field-even">Status<span class="colon">:</span></dt> <dd class="field-even"><abbr title="Accepted and implementation complete, or no longer active">Final</abbr></dd> <dt class="field-odd">Type<span class="colon">:</span></dt> <dd class="field-odd"><abbr title="Normative PEP with a new feature for Python, implementation change for CPython or interoperability standard for the ecosystem">Standards Track</abbr></dd> <dt class="field-even">Created<span class="colon">:</span></dt> <dd class="field-even">20-Jan-2015</dd> <dt class="field-odd">Python-Version<span class="colon">:</span></dt> <dd class="field-odd">3.5</dd> <dt class="field-even">Post-History<span class="colon">:</span></dt> <dd class="field-even"><p></p></dd> <dt class="field-odd">Resolution<span class="colon">:</span></dt> <dd class="field-odd"><a class="reference external" href="https://mail.python.org/pipermail/python-dev/2015-February/138598.html">Python-Dev message</a></dd> </dl> <hr class="docutils" /> <section id="contents"> <details><summary>Table of Contents</summary><ul class="simple"> <li><a class="reference internal" href="#abstract">Abstract</a></li> <li><a class="reference internal" href="#rationale">Rationale</a><ul> <li><a class="reference internal" href="#existing-implementations">Existing Implementations</a></li> </ul> </li> <li><a class="reference internal" href="#proposed-implementation">Proposed Implementation</a><ul> <li><a class="reference internal" href="#handling-of-non-finite-numbers">Handling of non-finite numbers</a></li> <li><a class="reference internal" href="#non-float-types">Non-float types</a></li> <li><a class="reference internal" href="#behavior-near-zero">Behavior near zero</a></li> <li><a class="reference internal" href="#implementation">Implementation</a></li> </ul> </li> <li><a class="reference internal" href="#relative-difference">Relative Difference</a><ul> <li><a class="reference internal" href="#how-much-difference-does-it-make">How much difference does it make?</a></li> <li><a class="reference internal" href="#symmetry">Symmetry</a></li> <li><a class="reference internal" href="#which-symmetric-test">Which symmetric test?</a></li> <li><a class="reference internal" href="#large-tolerances">Large Tolerances</a></li> </ul> </li> <li><a class="reference internal" href="#defaults">Defaults</a><ul> <li><a class="reference internal" href="#relative-tolerance-default">Relative Tolerance Default</a></li> <li><a class="reference internal" href="#absolute-tolerance-default">Absolute tolerance default</a></li> </ul> </li> <li><a class="reference internal" href="#expected-uses">Expected Uses</a><ul> <li><a class="reference internal" href="#inappropriate-uses">Inappropriate uses</a></li> </ul> </li> <li><a class="reference internal" href="#other-approaches">Other Approaches</a><ul> <li><a class="reference internal" href="#unittest-testcase-assertalmostequal"><code class="docutils literal notranslate"><span class="pre">unittest.TestCase.assertAlmostEqual</span></code></a></li> <li><a class="reference internal" href="#numpy-isclose">numpy <code class="docutils literal notranslate"><span class="pre">isclose()</span></code></a></li> <li><a class="reference internal" href="#boost-floating-point-comparison">Boost floating-point comparison</a></li> <li><a class="reference internal" href="#alternate-proposals">Alternate Proposals</a><ul> <li><a class="reference internal" href="#a-recipe">A Recipe</a></li> <li><a class="reference internal" href="#zero-tol"><code class="docutils literal notranslate"><span class="pre">zero_tol</span></code></a></li> <li><a class="reference internal" href="#no-absolute-tolerance">No absolute tolerance</a></li> <li><a class="reference internal" href="#other-tests">Other tests</a></li> </ul> </li> </ul> </li> <li><a class="reference internal" href="#references">References</a></li> <li><a class="reference internal" href="#copyright">Copyright</a></li> </ul> </details></section> <section id="abstract"> <h2><a class="toc-backref" href="#abstract" role="doc-backlink">Abstract</a></h2> <p>This PEP proposes the addition of an isclose() function to the standard library math module that determines whether one value is approximately equal or “close” to another value.</p> </section> <section id="rationale"> <h2><a class="toc-backref" href="#rationale" role="doc-backlink">Rationale</a></h2> <p>Floating point values contain limited precision, which results in their being unable to exactly represent some values, and for errors to accumulate with repeated computation. As a result, it is common advice to only use an equality comparison in very specific situations. Often an inequality comparison fits the bill, but there are times (often in testing) where the programmer wants to determine whether a computed value is “close” to an expected value, without requiring them to be exactly equal. This is common enough, particularly in testing, and not always obvious how to do it, that it would be useful addition to the standard library.</p> <section id="existing-implementations"> <h3><a class="toc-backref" href="#existing-implementations" role="doc-backlink">Existing Implementations</a></h3> <p>The standard library includes the <code class="docutils literal notranslate"><span class="pre">unittest.TestCase.assertAlmostEqual</span></code> method, but it:</p> <ul class="simple"> <li>Is buried in the unittest.TestCase class</li> <li>Is an assertion, so you can’t use it as a general test at the command line, etc. (easily)</li> <li>Is an absolute difference test. Often the measure of difference requires, particularly for floating point numbers, a relative error, i.e. “Are these two values within x% of each-other?”, rather than an absolute error. Particularly when the magnitude of the values is unknown a priori.</li> </ul> <p>The numpy package has the <code class="docutils literal notranslate"><span class="pre">allclose()</span></code> and <code class="docutils literal notranslate"><span class="pre">isclose()</span></code> functions, but they are only available with numpy.</p> <p>The statistics package tests include an implementation, used for its unit tests.</p> <p>One can also find discussion and sample implementations on Stack Overflow and other help sites.</p> <p>Many other non-python systems provide such a test, including the Boost C++ library and the APL language <a class="footnote-reference brackets" href="#id9" id="id1">[4]</a>.</p> <p>These existing implementations indicate that this is a common need and not trivial to write oneself, making it a candidate for the standard library.</p> </section> </section> <section id="proposed-implementation"> <h2><a class="toc-backref" href="#proposed-implementation" role="doc-backlink">Proposed Implementation</a></h2> <p>NOTE: this PEP is the result of extended discussions on the python-ideas list <a class="footnote-reference brackets" href="#id6" id="id2">[1]</a>.</p> <p>The new function will go into the math module, and have the following signature:</p> <div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">isclose</span><span class="p">(</span><span class="n">a</span><span class="p">,</span> <span class="n">b</span><span class="p">,</span> <span class="n">rel_tol</span><span class="o">=</span><span class="mf">1e-9</span><span class="p">,</span> <span class="n">abs_tol</span><span class="o">=</span><span class="mf">0.0</span><span class="p">)</span> </pre></div> </div> <p><code class="docutils literal notranslate"><span class="pre">a</span></code> and <code class="docutils literal notranslate"><span class="pre">b</span></code>: are the two values to be tested to relative closeness</p> <p><code class="docutils literal notranslate"><span class="pre">rel_tol</span></code>: is the relative tolerance – it is the amount of error allowed, relative to the larger absolute value of a or b. For example, to set a tolerance of 5%, pass tol=0.05. The default tolerance is 1e-9, which assures that the two values are the same within about 9 decimal digits. <code class="docutils literal notranslate"><span class="pre">rel_tol</span></code> must be greater than 0.0</p> <p><code class="docutils literal notranslate"><span class="pre">abs_tol</span></code>: is a minimum absolute tolerance level – useful for comparisons near zero.</p> <p>Modulo error checking, etc, the function will return the result of:</p> <div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="nb">abs</span><span class="p">(</span><span class="n">a</span><span class="o">-</span><span class="n">b</span><span class="p">)</span> <span class="o"><=</span> <span class="nb">max</span><span class="p">(</span> <span class="n">rel_tol</span> <span class="o">*</span> <span class="nb">max</span><span class="p">(</span><span class="nb">abs</span><span class="p">(</span><span class="n">a</span><span class="p">),</span> <span class="nb">abs</span><span class="p">(</span><span class="n">b</span><span class="p">)),</span> <span class="n">abs_tol</span> <span class="p">)</span> </pre></div> </div> <p>The name, <code class="docutils literal notranslate"><span class="pre">isclose</span></code>, is selected for consistency with the existing <code class="docutils literal notranslate"><span class="pre">isnan</span></code> and <code class="docutils literal notranslate"><span class="pre">isinf</span></code>.</p> <section id="handling-of-non-finite-numbers"> <h3><a class="toc-backref" href="#handling-of-non-finite-numbers" role="doc-backlink">Handling of non-finite numbers</a></h3> <p>The IEEE 754 special values of NaN, inf, and -inf will be handled according to IEEE rules. Specifically, NaN is not considered close to any other value, including NaN. inf and -inf are only considered close to themselves.</p> </section> <section id="non-float-types"> <h3><a class="toc-backref" href="#non-float-types" role="doc-backlink">Non-float types</a></h3> <p>The primary use-case is expected to be floating point numbers. However, users may want to compare other numeric types similarly. In theory, it should work for any type that supports <code class="docutils literal notranslate"><span class="pre">abs()</span></code>, multiplication, comparisons, and subtraction. However, the implementation in the math module is written in C, and thus can not (easily) use python’s duck typing. Rather, the values passed into the function will be converted to the float type before the calculation is performed. Passing in types (or values) that cannot be converted to floats will raise an appropriate Exception (TypeError, ValueError, or OverflowError).</p> <p>The code will be tested to accommodate at least some values of these types:</p> <ul class="simple"> <li><code class="docutils literal notranslate"><span class="pre">Decimal</span></code></li> <li><code class="docutils literal notranslate"><span class="pre">int</span></code></li> <li><code class="docutils literal notranslate"><span class="pre">Fraction</span></code></li> <li><code class="docutils literal notranslate"><span class="pre">complex</span></code>: For complex, a companion function will be added to the <code class="docutils literal notranslate"><span class="pre">cmath</span></code> module. In <code class="docutils literal notranslate"><span class="pre">cmath.isclose()</span></code>, the tolerances are specified as floats, and the absolute value of the complex values will be used for scaling and comparison. If a complex tolerance is passed in, the absolute value will be used as the tolerance.</li> </ul> <p>NOTE: it may make sense to add a <code class="docutils literal notranslate"><span class="pre">Decimal.isclose()</span></code> that works properly and completely with the decimal type, but that is not included as part of this PEP.</p> </section> <section id="behavior-near-zero"> <h3><a class="toc-backref" href="#behavior-near-zero" role="doc-backlink">Behavior near zero</a></h3> <p>Relative comparison is problematic if either value is zero. By definition, no value is small relative to zero. And computationally, if either value is zero, the difference is the absolute value of the other value, and the computed absolute tolerance will be <code class="docutils literal notranslate"><span class="pre">rel_tol</span></code> times that value. When <code class="docutils literal notranslate"><span class="pre">rel_tol</span></code> is less than one, the difference will never be less than the tolerance.</p> <p>However, while mathematically correct, there are many use cases where a user will need to know if a computed value is “close” to zero. This calls for an absolute tolerance test. If the user needs to call this function inside a loop or comprehension, where some, but not all, of the expected values may be zero, it is important that both a relative tolerance and absolute tolerance can be tested for with a single function with a single set of parameters.</p> <p>There is a similar issue if the two values to be compared straddle zero: if a is approximately equal to -b, then a and b will never be computed as “close”.</p> <p>To handle this case, an optional parameter, <code class="docutils literal notranslate"><span class="pre">abs_tol</span></code> can be used to set a minimum tolerance used in the case of very small or zero computed relative tolerance. That is, the values will be always be considered close if the difference between them is less than <code class="docutils literal notranslate"><span class="pre">abs_tol</span></code></p> <p>The default absolute tolerance value is set to zero because there is no value that is appropriate for the general case. It is impossible to know an appropriate value without knowing the likely values expected for a given use case. If all the values tested are on order of one, then a value of about 1e-9 might be appropriate, but that would be far too large if expected values are on order of 1e-9 or smaller.</p> <p>Any non-zero default might result in user’s tests passing totally inappropriately. If, on the other hand, a test against zero fails the first time with defaults, a user will be prompted to select an appropriate value for the problem at hand in order to get the test to pass.</p> <p>NOTE: that the author of this PEP has resolved to go back over many of his tests that use the numpy <code class="docutils literal notranslate"><span class="pre">allclose()</span></code> function, which provides a default absolute tolerance, and make sure that the default value is appropriate.</p> <p>If the user sets the rel_tol parameter to 0.0, then only the absolute tolerance will effect the result. While not the goal of the function, it does allow it to be used as a purely absolute tolerance check as well.</p> </section> <section id="implementation"> <h3><a class="toc-backref" href="#implementation" role="doc-backlink">Implementation</a></h3> <p>A sample implementation in python is available (as of Jan 22, 2015) on gitHub:</p> <p><a class="reference external" href="https://github.com/PythonCHB/close_pep/blob/master/is_close.py">https://github.com/PythonCHB/close_pep/blob/master/is_close.py</a></p> <p>This implementation has a flag that lets the user select which relative tolerance test to apply – this PEP does not suggest that that be retained, but rather that the weak test be selected.</p> <p>There are also drafts of this PEP and test code, etc. there:</p> <p><a class="reference external" href="https://github.com/PythonCHB/close_pep">https://github.com/PythonCHB/close_pep</a></p> </section> </section> <section id="relative-difference"> <h2><a class="toc-backref" href="#relative-difference" role="doc-backlink">Relative Difference</a></h2> <p>There are essentially two ways to think about how close two numbers are to each-other:</p> <p>Absolute difference: simply <code class="docutils literal notranslate"><span class="pre">abs(a-b)</span></code></p> <p>Relative difference: <code class="docutils literal notranslate"><span class="pre">abs(a-b)/scale_factor</span></code> <a class="footnote-reference brackets" href="#id7" id="id3">[2]</a>.</p> <p>The absolute difference is trivial enough that this proposal focuses on the relative difference.</p> <p>Usually, the scale factor is some function of the values under consideration, for instance:</p> <ol class="arabic simple"> <li>The absolute value of one of the input values</li> <li>The maximum absolute value of the two</li> <li>The minimum absolute value of the two.</li> <li>The absolute value of the arithmetic mean of the two</li> </ol> <p>These lead to the following possibilities for determining if two values, a and b, are close to each other.</p> <ol class="arabic simple"> <li><code class="docutils literal notranslate"><span class="pre">abs(a-b)</span> <span class="pre"><=</span> <span class="pre">tol*abs(a)</span></code></li> <li><code class="docutils literal notranslate"><span class="pre">abs(a-b)</span> <span class="pre"><=</span> <span class="pre">tol</span> <span class="pre">*</span> <span class="pre">max(</span> <span class="pre">abs(a),</span> <span class="pre">abs(b)</span> <span class="pre">)</span></code></li> <li><code class="docutils literal notranslate"><span class="pre">abs(a-b)</span> <span class="pre"><=</span> <span class="pre">tol</span> <span class="pre">*</span> <span class="pre">min(</span> <span class="pre">abs(a),</span> <span class="pre">abs(b)</span> <span class="pre">)</span></code></li> <li><code class="docutils literal notranslate"><span class="pre">abs(a-b)</span> <span class="pre"><=</span> <span class="pre">tol</span> <span class="pre">*</span> <span class="pre">abs(a</span> <span class="pre">+</span> <span class="pre">b)/2</span></code></li> </ol> <p>NOTE: (2) and (3) can also be written as:</p> <ol class="arabic simple" start="2"> <li><code class="docutils literal notranslate"><span class="pre">(abs(a-b)</span> <span class="pre"><=</span> <span class="pre">abs(tol*a))</span> <span class="pre">or</span> <span class="pre">(abs(a-b)</span> <span class="pre"><=</span> <span class="pre">abs(tol*b))</span></code></li> <li><code class="docutils literal notranslate"><span class="pre">(abs(a-b)</span> <span class="pre"><=</span> <span class="pre">abs(tol*a))</span> <span class="pre">and</span> <span class="pre">(abs(a-b)</span> <span class="pre"><=</span> <span class="pre">abs(tol*b))</span></code></li> </ol> <p>(Boost refers to these as the “weak” and “strong” formulations <a class="footnote-reference brackets" href="#id8" id="id4">[3]</a>) These can be a tiny bit more computationally efficient, and thus are used in the example code.</p> <p>Each of these formulations can lead to slightly different results. However, if the tolerance value is small, the differences are quite small. In fact, often less than available floating point precision.</p> <section id="how-much-difference-does-it-make"> <h3><a class="toc-backref" href="#how-much-difference-does-it-make" role="doc-backlink">How much difference does it make?</a></h3> <p>When selecting a method to determine closeness, one might want to know how much of a difference it could make to use one test or the other – i.e. how many values are there (or what range of values) that will pass one test, but not the other.</p> <p>The largest difference is between options (2) and (3) where the allowable absolute difference is scaled by either the larger or smaller of the values.</p> <p>Define <code class="docutils literal notranslate"><span class="pre">delta</span></code> to be the difference between the allowable absolute tolerance defined by the larger value and that defined by the smaller value. That is, the amount that the two input values need to be different in order to get a different result from the two tests. <code class="docutils literal notranslate"><span class="pre">tol</span></code> is the relative tolerance value.</p> <p>Assume that <code class="docutils literal notranslate"><span class="pre">a</span></code> is the larger value and that both <code class="docutils literal notranslate"><span class="pre">a</span></code> and <code class="docutils literal notranslate"><span class="pre">b</span></code> are positive, to make the analysis a bit easier. <code class="docutils literal notranslate"><span class="pre">delta</span></code> is therefore:</p> <div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">delta</span> <span class="o">=</span> <span class="n">tol</span> <span class="o">*</span> <span class="p">(</span><span class="n">a</span><span class="o">-</span><span class="n">b</span><span class="p">)</span> </pre></div> </div> <p>or:</p> <div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">delta</span> <span class="o">/</span> <span class="n">tol</span> <span class="o">=</span> <span class="p">(</span><span class="n">a</span><span class="o">-</span><span class="n">b</span><span class="p">)</span> </pre></div> </div> <p>The largest absolute difference that would pass the test: <code class="docutils literal notranslate"><span class="pre">(a-b)</span></code>, equals the tolerance times the larger value:</p> <div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="p">(</span><span class="n">a</span><span class="o">-</span><span class="n">b</span><span class="p">)</span> <span class="o">=</span> <span class="n">tol</span> <span class="o">*</span> <span class="n">a</span> </pre></div> </div> <p>Substituting into the expression for delta:</p> <div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">delta</span> <span class="o">/</span> <span class="n">tol</span> <span class="o">=</span> <span class="n">tol</span> <span class="o">*</span> <span class="n">a</span> </pre></div> </div> <p>so:</p> <div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">delta</span> <span class="o">=</span> <span class="n">tol</span><span class="o">**</span><span class="mi">2</span> <span class="o">*</span> <span class="n">a</span> </pre></div> </div> <p>For example, for <code class="docutils literal notranslate"><span class="pre">a</span> <span class="pre">=</span> <span class="pre">10</span></code>, <code class="docutils literal notranslate"><span class="pre">b</span> <span class="pre">=</span> <span class="pre">9</span></code>, <code class="docutils literal notranslate"><span class="pre">tol</span> <span class="pre">=</span> <span class="pre">0.1</span></code> (10%):</p> <p>maximum tolerance <code class="docutils literal notranslate"><span class="pre">tol</span> <span class="pre">*</span> <span class="pre">a</span> <span class="pre">==</span> <span class="pre">0.1</span> <span class="pre">*</span> <span class="pre">10</span> <span class="pre">==</span> <span class="pre">1.0</span></code></p> <p>minimum tolerance <code class="docutils literal notranslate"><span class="pre">tol</span> <span class="pre">*</span> <span class="pre">b</span> <span class="pre">==</span> <span class="pre">0.1</span> <span class="pre">*</span> <span class="pre">9.0</span> <span class="pre">==</span> <span class="pre">0.9</span></code></p> <p>delta = <code class="docutils literal notranslate"><span class="pre">(1.0</span> <span class="pre">-</span> <span class="pre">0.9)</span> <span class="pre">=</span> <span class="pre">0.1</span></code> or <code class="docutils literal notranslate"><span class="pre">tol**2</span> <span class="pre">*</span> <span class="pre">a</span> <span class="pre">=</span> <span class="pre">0.1**2</span> <span class="pre">*</span> <span class="pre">10</span> <span class="pre">=</span> <span class="pre">.1</span></code></p> <p>The absolute difference between the maximum and minimum tolerance tests in this case could be substantial. However, the primary use case for the proposed function is testing the results of computations. In that case a relative tolerance is likely to be selected of much smaller magnitude.</p> <p>For example, a relative tolerance of <code class="docutils literal notranslate"><span class="pre">1e-8</span></code> is about half the precision available in a python float. In that case, the difference between the two tests is <code class="docutils literal notranslate"><span class="pre">1e-8**2</span> <span class="pre">*</span> <span class="pre">a</span></code> or <code class="docutils literal notranslate"><span class="pre">1e-16</span> <span class="pre">*</span> <span class="pre">a</span></code>, which is close to the limit of precision of a python float. If the relative tolerance is set to the proposed default of 1e-9 (or smaller), the difference between the two tests will be lost to the limits of precision of floating point. That is, each of the four methods will yield exactly the same results for all values of a and b.</p> <p>In addition, in common use, tolerances are defined to 1 significant figure – that is, 1e-9 is specifying about 9 decimal digits of accuracy. So the difference between the various possible tests is well below the precision to which the tolerance is specified.</p> </section> <section id="symmetry"> <h3><a class="toc-backref" href="#symmetry" role="doc-backlink">Symmetry</a></h3> <p>A relative comparison can be either symmetric or non-symmetric. For a symmetric algorithm:</p> <p><code class="docutils literal notranslate"><span class="pre">isclose(a,b)</span></code> is always the same as <code class="docutils literal notranslate"><span class="pre">isclose(b,a)</span></code></p> <p>If a relative closeness test uses only one of the values (such as (1) above), then the result is asymmetric, i.e. isclose(a,b) is not necessarily the same as isclose(b,a).</p> <p>Which approach is most appropriate depends on what question is being asked. If the question is: “are these two numbers close to each other?”, there is no obvious ordering, and a symmetric test is most appropriate.</p> <p>However, if the question is: “Is the computed value within x% of this known value?”, then it is appropriate to scale the tolerance to the known value, and an asymmetric test is most appropriate.</p> <p>From the previous section, it is clear that either approach would yield the same or similar results in the common use cases. In that case, the goal of this proposal is to provide a function that is least likely to produce surprising results.</p> <p>The symmetric approach provide an appealing consistency – it mirrors the symmetry of equality, and is less likely to confuse people. A symmetric test also relieves the user of the need to think about the order in which to set the arguments. It was also pointed out that there may be some cases where the order of evaluation may not be well defined, for instance in the case of comparing a set of values all against each other.</p> <p>There may be cases when a user does need to know that a value is within a particular range of a known value. In that case, it is easy enough to simply write the test directly:</p> <div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="k">if</span> <span class="n">a</span><span class="o">-</span><span class="n">b</span> <span class="o"><=</span> <span class="n">tol</span><span class="o">*</span><span class="n">a</span><span class="p">:</span> </pre></div> </div> <p>(assuming a > b in this case). There is little need to provide a function for this particular case.</p> <p>This proposal uses a symmetric test.</p> </section> <section id="which-symmetric-test"> <h3><a class="toc-backref" href="#which-symmetric-test" role="doc-backlink">Which symmetric test?</a></h3> <p>There are three symmetric tests considered:</p> <p>The case that uses the arithmetic mean of the two values requires that the value be either added together before dividing by 2, which could result in extra overflow to inf for very large numbers, or require each value to be divided by two before being added together, which could result in underflow to zero for very small numbers. This effect would only occur at the very limit of float values, but it was decided there was no benefit to the method worth reducing the range of functionality or adding the complexity of checking values to determine the order of computation.</p> <p>This leaves the boost “weak” test (2)– or using the larger value to scale the tolerance, or the Boost “strong” (3) test, which uses the smaller of the values to scale the tolerance. For small tolerance, they yield the same result, but this proposal uses the boost “weak” test case: it is symmetric and provides a more useful result for very large tolerances.</p> </section> <section id="large-tolerances"> <h3><a class="toc-backref" href="#large-tolerances" role="doc-backlink">Large Tolerances</a></h3> <p>The most common use case is expected to be small tolerances – on order of the default 1e-9. However, there may be use cases where a user wants to know if two fairly disparate values are within a particular range of each other: “is a within 200% (rel_tol = 2.0) of b? In this case, the strong test would never indicate that two values are within that range of each other if one of them is zero. The weak case, however would use the larger (non-zero) value for the test, and thus return true if one value is zero. For example: is 0 within 200% of 10? 200% of ten is 20, so the range within 200% of ten is -10 to +30. Zero falls within that range, so it will return True.</p> </section> </section> <section id="defaults"> <h2><a class="toc-backref" href="#defaults" role="doc-backlink">Defaults</a></h2> <p>Default values are required for the relative and absolute tolerance.</p> <section id="relative-tolerance-default"> <h3><a class="toc-backref" href="#relative-tolerance-default" role="doc-backlink">Relative Tolerance Default</a></h3> <p>The relative tolerance required for two values to be considered “close” is entirely use-case dependent. Nevertheless, the relative tolerance needs to be greater than 1e-16 (approximate precision of a python float). The value of 1e-9 was selected because it is the largest relative tolerance for which the various possible methods will yield the same result, and it is also about half of the precision available to a python float. In the general case, a good numerical algorithm is not expected to lose more than about half of available digits of accuracy, and if a much larger tolerance is acceptable, the user should be considering the proper value in that case. Thus 1e-9 is expected to “just work” for many cases.</p> </section> <section id="absolute-tolerance-default"> <h3><a class="toc-backref" href="#absolute-tolerance-default" role="doc-backlink">Absolute tolerance default</a></h3> <p>The absolute tolerance value will be used primarily for comparing to zero. The absolute tolerance required to determine if a value is “close” to zero is entirely use-case dependent. There is also essentially no bounds to the useful range – expected values would conceivably be anywhere within the limits of a python float. Thus a default of 0.0 is selected.</p> <p>If, for a given use case, a user needs to compare to zero, the test will be guaranteed to fail the first time, and the user can select an appropriate value.</p> <p>It was suggested that comparing to zero is, in fact, a common use case (evidence suggest that the numpy functions are often used with zero). In this case, it would be desirable to have a “useful” default. Values around 1e-8 were suggested, being about half of floating point precision for values of around value 1.</p> <p>However, to quote The Zen: “In the face of ambiguity, refuse the temptation to guess.” Guessing that users will most often be concerned with values close to 1.0 would lead to spurious passing tests when used with smaller values – this is potentially more damaging than requiring the user to thoughtfully select an appropriate value.</p> </section> </section> <section id="expected-uses"> <h2><a class="toc-backref" href="#expected-uses" role="doc-backlink">Expected Uses</a></h2> <p>The primary expected use case is various forms of testing – “are the results computed near what I expect as a result?” This sort of test may or may not be part of a formal unit testing suite. Such testing could be used one-off at the command line, in an IPython notebook, part of doctests, or simple asserts in an <code class="docutils literal notranslate"><span class="pre">if</span> <span class="pre">__name__</span> <span class="pre">==</span> <span class="pre">"__main__"</span></code> block.</p> <p>It would also be an appropriate function to use for the termination criteria for a simple iterative solution to an implicit function:</p> <div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">guess</span> <span class="o">=</span> <span class="n">something</span> <span class="k">while</span> <span class="kc">True</span><span class="p">:</span> <span class="n">new_guess</span> <span class="o">=</span> <span class="n">implicit_function</span><span class="p">(</span><span class="n">guess</span><span class="p">,</span> <span class="o">*</span><span class="n">args</span><span class="p">)</span> <span class="k">if</span> <span class="n">isclose</span><span class="p">(</span><span class="n">new_guess</span><span class="p">,</span> <span class="n">guess</span><span class="p">):</span> <span class="k">break</span> <span class="n">guess</span> <span class="o">=</span> <span class="n">new_guess</span> </pre></div> </div> <section id="inappropriate-uses"> <h3><a class="toc-backref" href="#inappropriate-uses" role="doc-backlink">Inappropriate uses</a></h3> <p>One use case for floating point comparison is testing the accuracy of a numerical algorithm. However, in this case, the numerical analyst ideally would be doing careful error propagation analysis, and should understand exactly what to test for. It is also likely that ULP (Unit in the Last Place) comparison may be called for. While this function may prove useful in such situations, It is not intended to be used in that way without careful consideration.</p> </section> </section> <section id="other-approaches"> <h2><a class="toc-backref" href="#other-approaches" role="doc-backlink">Other Approaches</a></h2> <section id="unittest-testcase-assertalmostequal"> <h3><a class="toc-backref" href="#unittest-testcase-assertalmostequal" role="doc-backlink"><code class="docutils literal notranslate"><span class="pre">unittest.TestCase.assertAlmostEqual</span></code></a></h3> <p>(<a class="reference external" href="https://docs.python.org/3/library/unittest.html#unittest.TestCase.assertAlmostEqual">https://docs.python.org/3/library/unittest.html#unittest.TestCase.assertAlmostEqual</a>)</p> <p>Tests that values are approximately (or not approximately) equal by computing the difference, rounding to the given number of decimal places (default 7), and comparing to zero.</p> <p>This method is purely an absolute tolerance test, and does not address the need for a relative tolerance test.</p> </section> <section id="numpy-isclose"> <h3><a class="toc-backref" href="#numpy-isclose" role="doc-backlink">numpy <code class="docutils literal notranslate"><span class="pre">isclose()</span></code></a></h3> <p><a class="reference external" href="http://docs.scipy.org/doc/numpy-dev/reference/generated/numpy.isclose.html">http://docs.scipy.org/doc/numpy-dev/reference/generated/numpy.isclose.html</a></p> <p>The numpy package provides the vectorized functions isclose() and allclose(), for similar use cases as this proposal:</p> <p><code class="docutils literal notranslate"><span class="pre">isclose(a,</span> <span class="pre">b,</span> <span class="pre">rtol=1e-05,</span> <span class="pre">atol=1e-08,</span> <span class="pre">equal_nan=False)</span></code></p> <blockquote> <div>Returns a boolean array where two arrays are element-wise equal within a tolerance.<p>The tolerance values are positive, typically very small numbers. The relative difference (rtol * abs(b)) and the absolute difference atol are added together to compare against the absolute difference between a and b</p> </div></blockquote> <p>In this approach, the absolute and relative tolerance are added together, rather than the <code class="docutils literal notranslate"><span class="pre">or</span></code> method used in this proposal. This is computationally more simple, and if relative tolerance is larger than the absolute tolerance, then the addition will have no effect. However, if the absolute and relative tolerances are of similar magnitude, then the allowed difference will be about twice as large as expected.</p> <p>This makes the function harder to understand, with no computational advantage in this context.</p> <p>Even more critically, if the values passed in are small compared to the absolute tolerance, then the relative tolerance will be completely swamped, perhaps unexpectedly.</p> <p>This is why, in this proposal, the absolute tolerance defaults to zero – the user will be required to choose a value appropriate for the values at hand.</p> </section> <section id="boost-floating-point-comparison"> <h3><a class="toc-backref" href="#boost-floating-point-comparison" role="doc-backlink">Boost floating-point comparison</a></h3> <p>The Boost project ( <a class="footnote-reference brackets" href="#id8" id="id5">[3]</a> ) provides a floating point comparison function. It is a symmetric approach, with both “weak” (larger of the two relative errors) and “strong” (smaller of the two relative errors) options. This proposal uses the Boost “weak” approach. There is no need to complicate the API by providing the option to select different methods when the results will be similar in most cases, and the user is unlikely to know which to select in any case.</p> </section> <section id="alternate-proposals"> <h3><a class="toc-backref" href="#alternate-proposals" role="doc-backlink">Alternate Proposals</a></h3> <section id="a-recipe"> <h4><a class="toc-backref" href="#a-recipe" role="doc-backlink">A Recipe</a></h4> <p>The primary alternate proposal was to not provide a standard library function at all, but rather, provide a recipe for users to refer to. This would have the advantage that the recipe could provide and explain the various options, and let the user select that which is most appropriate. However, that would require anyone needing such a test to, at the very least, copy the function into their code base, and select the comparison method to use.</p> </section> <section id="zero-tol"> <h4><a class="toc-backref" href="#zero-tol" role="doc-backlink"><code class="docutils literal notranslate"><span class="pre">zero_tol</span></code></a></h4> <p>One possibility was to provide a zero tolerance parameter, rather than the absolute tolerance parameter. This would be an absolute tolerance that would only be applied in the case of one of the arguments being exactly zero. This would have the advantage of retaining the full relative tolerance behavior for all non-zero values, while allowing tests against zero to work. However, it would also result in the potentially surprising result that a small value could be “close” to zero, but not “close” to an even smaller value. e.g., 1e-10 is “close” to zero, but not “close” to 1e-11.</p> </section> <section id="no-absolute-tolerance"> <h4><a class="toc-backref" href="#no-absolute-tolerance" role="doc-backlink">No absolute tolerance</a></h4> <p>Given the issues with comparing to zero, another possibility would have been to only provide a relative tolerance, and let comparison to zero fail. In this case, the user would need to do a simple absolute test: <code class="docutils literal notranslate"><span class="pre">abs(val)</span> <span class="pre"><</span> <span class="pre">zero_tol</span></code> in the case where the comparison involved zero.</p> <p>However, this would not allow the same call to be used for a sequence of values, such as in a loop or comprehension. Making the function far less useful. It is noted that the default abs_tol=0.0 achieves the same effect if the default is not overridden.</p> </section> <section id="other-tests"> <h4><a class="toc-backref" href="#other-tests" role="doc-backlink">Other tests</a></h4> <p>The other tests considered are all discussed in the Relative Error section above.</p> </section> </section> </section> <section id="references"> <h2><a class="toc-backref" href="#references" role="doc-backlink">References</a></h2> <aside class="footnote-list brackets"> <aside class="footnote brackets" id="id6" role="doc-footnote"> <dt class="label" id="id6">[<a href="#id2">1</a>]</dt> <dd>Python-ideas list discussion threads<p><a class="reference external" href="https://mail.python.org/pipermail/python-ideas/2015-January/030947.html">https://mail.python.org/pipermail/python-ideas/2015-January/030947.html</a></p> <p><a class="reference external" href="https://mail.python.org/pipermail/python-ideas/2015-January/031124.html">https://mail.python.org/pipermail/python-ideas/2015-January/031124.html</a></p> <p><a class="reference external" href="https://mail.python.org/pipermail/python-ideas/2015-January/031313.html">https://mail.python.org/pipermail/python-ideas/2015-January/031313.html</a></p> </aside> <aside class="footnote brackets" id="id7" role="doc-footnote"> <dt class="label" id="id7">[<a href="#id3">2</a>]</dt> <dd>Wikipedia page on relative difference<p><a class="reference external" href="http://en.wikipedia.org/wiki/Relative_change_and_difference">http://en.wikipedia.org/wiki/Relative_change_and_difference</a></p> </aside> <aside class="footnote brackets" id="id8" role="doc-footnote"> <dt class="label" id="id8">[3]<em> (<a href='#id4'>1</a>, <a href='#id5'>2</a>) </em></dt> <dd>Boost project floating-point comparison algorithms<p><a class="reference external" href="http://www.boost.org/doc/libs/1_35_0/libs/test/doc/components/test_tools/floating_point_comparison.html">http://www.boost.org/doc/libs/1_35_0/libs/test/doc/components/test_tools/floating_point_comparison.html</a></p> </aside> <aside class="footnote brackets" id="id9" role="doc-footnote"> <dt class="label" id="id9">[<a href="#id1">4</a>]</dt> <dd>1976. R. H. Lathwell. APL comparison tolerance. Proceedings of the eighth international conference on APL Pages 255 - 258<p><a class="reference external" href="http://dl.acm.org/citation.cfm?doid=800114.803685">http://dl.acm.org/citation.cfm?doid=800114.803685</a></p> </aside> </aside> </section> <section id="copyright"> <h2><a class="toc-backref" href="#copyright" role="doc-backlink">Copyright</a></h2> <p>This document has been placed in the public domain.</p> </section> </section> <hr class="docutils" /> <p>Source: <a class="reference external" href="https://github.com/python/peps/blob/main/peps/pep-0485.rst">https://github.com/python/peps/blob/main/peps/pep-0485.rst</a></p> <p>Last modified: <a class="reference external" href="https://github.com/python/peps/commits/main/peps/pep-0485.rst">2023-09-09 17:39:29 GMT</a></p> </article> <nav id="pep-sidebar"> <h2>Contents</h2> <ul> <li><a class="reference internal" href="#abstract">Abstract</a></li> <li><a class="reference internal" href="#rationale">Rationale</a><ul> <li><a class="reference internal" href="#existing-implementations">Existing Implementations</a></li> </ul> </li> <li><a class="reference internal" href="#proposed-implementation">Proposed Implementation</a><ul> <li><a class="reference internal" href="#handling-of-non-finite-numbers">Handling of non-finite numbers</a></li> <li><a class="reference internal" href="#non-float-types">Non-float types</a></li> <li><a class="reference internal" href="#behavior-near-zero">Behavior near zero</a></li> <li><a class="reference internal" href="#implementation">Implementation</a></li> </ul> </li> <li><a class="reference internal" href="#relative-difference">Relative Difference</a><ul> <li><a class="reference internal" href="#how-much-difference-does-it-make">How much difference does it make?</a></li> <li><a class="reference internal" href="#symmetry">Symmetry</a></li> <li><a class="reference internal" href="#which-symmetric-test">Which symmetric test?</a></li> <li><a class="reference internal" href="#large-tolerances">Large Tolerances</a></li> </ul> </li> <li><a class="reference internal" href="#defaults">Defaults</a><ul> <li><a class="reference internal" href="#relative-tolerance-default">Relative Tolerance Default</a></li> <li><a class="reference internal" href="#absolute-tolerance-default">Absolute tolerance default</a></li> </ul> </li> <li><a class="reference internal" href="#expected-uses">Expected Uses</a><ul> <li><a class="reference internal" href="#inappropriate-uses">Inappropriate uses</a></li> </ul> </li> <li><a class="reference internal" href="#other-approaches">Other Approaches</a><ul> <li><a class="reference internal" href="#unittest-testcase-assertalmostequal"><code class="docutils literal notranslate"><span class="pre">unittest.TestCase.assertAlmostEqual</span></code></a></li> <li><a class="reference internal" href="#numpy-isclose">numpy <code class="docutils literal notranslate"><span class="pre">isclose()</span></code></a></li> <li><a class="reference internal" href="#boost-floating-point-comparison">Boost floating-point comparison</a></li> <li><a class="reference internal" href="#alternate-proposals">Alternate Proposals</a><ul> <li><a class="reference internal" href="#a-recipe">A Recipe</a></li> <li><a class="reference internal" href="#zero-tol"><code class="docutils literal notranslate"><span class="pre">zero_tol</span></code></a></li> <li><a class="reference internal" href="#no-absolute-tolerance">No absolute tolerance</a></li> <li><a class="reference internal" href="#other-tests">Other tests</a></li> </ul> </li> </ul> </li> <li><a class="reference internal" href="#references">References</a></li> <li><a class="reference internal" href="#copyright">Copyright</a></li> </ul> <br> <a id="source" href="https://github.com/python/peps/blob/main/peps/pep-0485.rst">Page Source (GitHub)</a> </nav> </section> <script src="../_static/colour_scheme.js"></script> <script src="../_static/wrap_tables.js"></script> <script src="../_static/sticky_banner.js"></script> </body> </html>