CINXE.COM
PEP 678 – Enriching Exceptions with Notes | peps.python.org
<!DOCTYPE html> <html lang="en"> <head> <meta charset="utf-8"> <meta name="viewport" content="width=device-width, initial-scale=1.0"> <meta name="color-scheme" content="light dark"> <title>PEP 678 – Enriching Exceptions with Notes | peps.python.org</title> <link rel="shortcut icon" href="../_static/py.png"> <link rel="canonical" href="https://peps.python.org/pep-0678/"> <link rel="stylesheet" href="../_static/style.css" type="text/css"> <link rel="stylesheet" href="../_static/mq.css" type="text/css"> <link rel="stylesheet" href="../_static/pygments.css" type="text/css" media="(prefers-color-scheme: light)" id="pyg-light"> <link rel="stylesheet" href="../_static/pygments_dark.css" type="text/css" media="(prefers-color-scheme: dark)" id="pyg-dark"> <link rel="alternate" type="application/rss+xml" title="Latest PEPs" href="https://peps.python.org/peps.rss"> <meta property="og:title" content='PEP 678 – Enriching Exceptions with Notes | peps.python.org'> <meta property="og:description" content="Exception objects are typically initialized with a message that describes the error which has occurred. Because further information may be available when the exception is caught and re-raised, or included in an ExceptionGroup, this PEP proposes to add ..."> <meta property="og:type" content="website"> <meta property="og:url" content="https://peps.python.org/pep-0678/"> <meta property="og:site_name" content="Python Enhancement Proposals (PEPs)"> <meta property="og:image" content="https://peps.python.org/_static/og-image.png"> <meta property="og:image:alt" content="Python PEPs"> <meta property="og:image:width" content="200"> <meta property="og:image:height" content="200"> <meta name="description" content="Exception objects are typically initialized with a message that describes the error which has occurred. Because further information may be available when the exception is caught and re-raised, or included in an ExceptionGroup, this PEP proposes to add ..."> <meta name="theme-color" content="#3776ab"> </head> <body> <svg xmlns="http://www.w3.org/2000/svg" style="display: none;"> <symbol id="svg-sun-half" viewBox="0 0 24 24" pointer-events="all"> <title>Following system colour scheme</title> <svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"> <circle cx="12" cy="12" r="9"></circle> <path d="M12 3v18m0-12l4.65-4.65M12 14.3l7.37-7.37M12 19.6l8.85-8.85"></path> </svg> </symbol> <symbol id="svg-moon" viewBox="0 0 24 24" pointer-events="all"> <title>Selected dark colour scheme</title> <svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"> <path stroke="none" d="M0 0h24v24H0z" fill="none"></path> <path d="M12 3c.132 0 .263 0 .393 0a7.5 7.5 0 0 0 7.92 12.446a9 9 0 1 1 -8.313 -12.454z"></path> </svg> </symbol> <symbol id="svg-sun" viewBox="0 0 24 24" pointer-events="all"> <title>Selected light colour scheme</title> <svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"> <circle cx="12" cy="12" r="5"></circle> <line x1="12" y1="1" x2="12" y2="3"></line> <line x1="12" y1="21" x2="12" y2="23"></line> <line x1="4.22" y1="4.22" x2="5.64" y2="5.64"></line> <line x1="18.36" y1="18.36" x2="19.78" y2="19.78"></line> <line x1="1" y1="12" x2="3" y2="12"></line> <line x1="21" y1="12" x2="23" y2="12"></line> <line x1="4.22" y1="19.78" x2="5.64" y2="18.36"></line> <line x1="18.36" y1="5.64" x2="19.78" y2="4.22"></line> </svg> </symbol> </svg> <script> document.documentElement.dataset.colour_scheme = localStorage.getItem("colour_scheme") || "auto" </script> <section id="pep-page-section"> <header> <h1>Python Enhancement Proposals</h1> <ul class="breadcrumbs"> <li><a href="https://www.python.org/" title="The Python Programming Language">Python</a> » </li> <li><a href="../pep-0000/">PEP Index</a> » </li> <li>PEP 678</li> </ul> <button id="colour-scheme-cycler" onClick="setColourScheme(nextColourScheme())"> <svg aria-hidden="true" class="colour-scheme-icon-when-auto"><use href="#svg-sun-half"></use></svg> <svg aria-hidden="true" class="colour-scheme-icon-when-dark"><use href="#svg-moon"></use></svg> <svg aria-hidden="true" class="colour-scheme-icon-when-light"><use href="#svg-sun"></use></svg> <span class="visually-hidden">Toggle light / dark / auto colour theme</span> </button> </header> <article> <section id="pep-content"> <h1 class="page-title">PEP 678 – Enriching Exceptions with Notes</h1> <dl class="rfc2822 field-list simple"> <dt class="field-odd">Author<span class="colon">:</span></dt> <dd class="field-odd">Zac Hatfield-Dodds <zac at zhd.dev></dd> <dt class="field-even">Sponsor<span class="colon">:</span></dt> <dd class="field-even">Irit Katriel</dd> <dt class="field-odd">Discussions-To<span class="colon">:</span></dt> <dd class="field-odd"><a class="reference external" href="https://discuss.python.org/t/pep-678-enriching-exceptions-with-notes/13374">Discourse thread</a></dd> <dt class="field-even">Status<span class="colon">:</span></dt> <dd class="field-even"><abbr title="Accepted and implementation complete, or no longer active">Final</abbr></dd> <dt class="field-odd">Type<span class="colon">:</span></dt> <dd class="field-odd"><abbr title="Normative PEP with a new feature for Python, implementation change for CPython or interoperability standard for the ecosystem">Standards Track</abbr></dd> <dt class="field-even">Requires<span class="colon">:</span></dt> <dd class="field-even"><a class="reference external" href="../pep-0654/">654</a></dd> <dt class="field-odd">Created<span class="colon">:</span></dt> <dd class="field-odd">20-Dec-2021</dd> <dt class="field-even">Python-Version<span class="colon">:</span></dt> <dd class="field-even">3.11</dd> <dt class="field-odd">Post-History<span class="colon">:</span></dt> <dd class="field-odd"><a class="reference external" href="https://discuss.python.org/t/pep-678-enriching-exceptions-with-notes/13374" title="Discourse thread">27-Jan-2022</a></dd> <dt class="field-even">Resolution<span class="colon">:</span></dt> <dd class="field-even"><a class="reference external" href="https://discuss.python.org/t/pep-678-enriching-exceptions-with-notes/13374/100">Discourse message</a></dd> </dl> <hr class="docutils" /> <section id="contents"> <details><summary>Table of Contents</summary><ul class="simple"> <li><a class="reference internal" href="#abstract">Abstract</a></li> <li><a class="reference internal" href="#motivation">Motivation</a><ul> <li><a class="reference internal" href="#example-usage">Example usage</a></li> <li><a class="reference internal" href="#non-goals">Non-goals</a></li> </ul> </li> <li><a class="reference internal" href="#specification">Specification</a></li> <li><a class="reference internal" href="#backwards-compatibility">Backwards Compatibility</a></li> <li><a class="reference internal" href="#how-to-teach-this">How to Teach This</a></li> <li><a class="reference internal" href="#reference-implementation">Reference Implementation</a></li> <li><a class="reference internal" href="#rejected-ideas">Rejected Ideas</a><ul> <li><a class="reference internal" href="#use-print-or-logging-etc">Use <code class="docutils literal notranslate"><span class="pre">print()</span></code> (or <code class="docutils literal notranslate"><span class="pre">logging</span></code>, etc.)</a></li> <li><a class="reference internal" href="#raise-wrapper-explanation-from-err"><code class="docutils literal notranslate"><span class="pre">raise</span> <span class="pre">Wrapper(explanation)</span> <span class="pre">from</span> <span class="pre">err</span></code></a></li> <li><a class="reference internal" href="#an-assignable-note-attribute">An assignable <code class="docutils literal notranslate"><span class="pre">__note__</span></code> attribute</a></li> <li><a class="reference internal" href="#subclass-exception-and-add-note-support-downstream">Subclass Exception and add note support downstream</a></li> <li><a class="reference internal" href="#don-t-attach-notes-to-exceptions-just-store-them-in-exceptiongroups">Don’t attach notes to <code class="docutils literal notranslate"><span class="pre">Exception</span></code>s, just store them in <code class="docutils literal notranslate"><span class="pre">ExceptionGroup</span></code>s</a></li> <li><a class="reference internal" href="#add-a-helper-function-contextlib-add-exc-note">Add a helper function <code class="docutils literal notranslate"><span class="pre">contextlib.add_exc_note()</span></code></a></li> <li><a class="reference internal" href="#augment-the-raise-statement">Augment the <code class="docutils literal notranslate"><span class="pre">raise</span></code> statement</a></li> </ul> </li> <li><a class="reference internal" href="#acknowledgements">Acknowledgements</a></li> <li><a class="reference internal" href="#references">References</a></li> <li><a class="reference internal" href="#copyright">Copyright</a></li> </ul> </details></section> <div class="pep-banner canonical-doc sticky-banner admonition important"> <p class="admonition-title">Important</p> <p>This PEP is a historical document. The up-to-date, canonical documentation can now be found at <a class="reference external" href="https://docs.python.org/3/library/exceptions.html#BaseException.add_note" title="(in Python v3.13)"><code class="docutils literal notranslate"><span class="pre">BaseException.add_note()</span></code></a> and <a class="reference external" href="https://docs.python.org/3/library/exceptions.html#BaseException.__notes__" title="(in Python v3.13)"><code class="docutils literal notranslate"><span class="pre">BaseException.__notes__</span></code></a>.</p> <p class="close-button">×</p> <p>See <a class="reference external" href="https://docs.python.org/3/tutorial/errors.html#tut-exception-notes" title="(in Python v3.13)"><span>Enriching Exceptions with Notes</span></a> for a user-focused tutorial.</p> <p>See <a class="pep reference internal" href="../pep-0001/" title="PEP 1 – PEP Purpose and Guidelines">PEP 1</a> for how to propose changes.</p> </div> <section id="abstract"> <h2><a class="toc-backref" href="#abstract" role="doc-backlink">Abstract</a></h2> <p>Exception objects are typically initialized with a message that describes the error which has occurred. Because further information may be available when the exception is caught and re-raised, or included in an <code class="docutils literal notranslate"><span class="pre">ExceptionGroup</span></code>, this PEP proposes to add <code class="docutils literal notranslate"><span class="pre">BaseException.add_note(note)</span></code>, a <code class="docutils literal notranslate"><span class="pre">.__notes__</span></code> attribute holding a list of notes so added, and to update the builtin traceback formatting code to include notes in the formatted traceback following the exception string.</p> <p>This is particularly useful in relation to <a class="pep reference internal" href="../pep-0654/" title="PEP 654 – Exception Groups and except*">PEP 654</a> <code class="docutils literal notranslate"><span class="pre">ExceptionGroup</span></code>s, which make previous workarounds ineffective or confusing. Use cases have been identified in the standard library, Hypothesis and <code class="docutils literal notranslate"><span class="pre">cattrs</span></code> packages, and common code patterns with retries.</p> </section> <section id="motivation"> <h2><a class="toc-backref" href="#motivation" role="doc-backlink">Motivation</a></h2> <p>When an exception is created in order to be raised, it is usually initialized with information that describes the error that has occurred. There are cases where it is useful to add information after the exception was caught. For example,</p> <ul class="simple"> <li>testing libraries may wish to show the values involved in a failing assertion, or the steps to reproduce a failure (e.g. <code class="docutils literal notranslate"><span class="pre">pytest</span></code> and <code class="docutils literal notranslate"><span class="pre">hypothesis</span></code>; example below).</li> <li>code which retries an operation on error may wish to associate an iteration, timestamp, or other explanation with each of several errors - especially if re-raising them in an <code class="docutils literal notranslate"><span class="pre">ExceptionGroup</span></code>.</li> <li>programming environments for novices can provide more detailed descriptions of various errors, and tips for resolving them.</li> </ul> <p>Existing approaches must pass this additional information around while keeping it in sync with the state of raised, and potentially caught or chained, exceptions. This is already error-prone, and made more difficult by <a class="pep reference internal" href="../pep-0654/" title="PEP 654 – Exception Groups and except*">PEP 654</a> <code class="docutils literal notranslate"><span class="pre">ExceptionGroup</span></code>s, so the time is right for a built-in solution. We therefore propose to add:</p> <ul class="simple"> <li>a new method <code class="docutils literal notranslate"><span class="pre">BaseException.add_note(note:</span> <span class="pre">str)</span></code>,</li> <li><code class="docutils literal notranslate"><span class="pre">BaseException.__notes__</span></code>, a list of note strings added using <code class="docutils literal notranslate"><span class="pre">.add_note()</span></code>, and</li> <li>support in the builtin traceback formatting code such that notes are displayed in the formatted traceback following the exception string.</li> </ul> <section id="example-usage"> <h3><a class="toc-backref" href="#example-usage" role="doc-backlink">Example usage</a></h3> <div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="gp">>>> </span><span class="k">try</span><span class="p">:</span> <span class="gp">... </span> <span class="k">raise</span> <span class="ne">TypeError</span><span class="p">(</span><span class="s1">'bad type'</span><span class="p">)</span> <span class="gp">... </span><span class="k">except</span> <span class="ne">Exception</span> <span class="k">as</span> <span class="n">e</span><span class="p">:</span> <span class="gp">... </span> <span class="n">e</span><span class="o">.</span><span class="n">add_note</span><span class="p">(</span><span class="s1">'Add some information'</span><span class="p">)</span> <span class="gp">... </span> <span class="k">raise</span> <span class="gp">...</span> <span class="gt">Traceback (most recent call last):</span> File <span class="nb">"<stdin>"</span>, line <span class="m">2</span>, in <span class="n"><module></span> <span class="gr">TypeError</span>: <span class="n">bad type</span> <span class="x">Add some information</span> <span class="gp">>>></span> </pre></div> </div> <p>When collecting exceptions into an exception group, we may want to add context information for the individual errors. In the following example with <a class="reference external" href="https://github.com/HypothesisWorks/hypothesis/pull/3191">Hypothesis’ proposed support for ExceptionGroup</a>, each exception includes a note of the minimal failing example:</p> <div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="kn">from</span> <span class="nn">hypothesis</span> <span class="kn">import</span> <span class="n">given</span><span class="p">,</span> <span class="n">strategies</span> <span class="k">as</span> <span class="n">st</span><span class="p">,</span> <span class="n">target</span> <span class="nd">@given</span><span class="p">(</span><span class="n">st</span><span class="o">.</span><span class="n">integers</span><span class="p">())</span> <span class="k">def</span> <span class="nf">test</span><span class="p">(</span><span class="n">x</span><span class="p">):</span> <span class="k">assert</span> <span class="n">x</span> <span class="o"><</span> <span class="mi">0</span> <span class="k">assert</span> <span class="n">x</span> <span class="o">></span> <span class="mi">0</span> <span class="o">+</span> <span class="ne">Exception</span> <span class="n">Group</span> <span class="n">Traceback</span> <span class="p">(</span><span class="n">most</span> <span class="n">recent</span> <span class="n">call</span> <span class="n">last</span><span class="p">):</span> <span class="o">|</span> <span class="n">File</span> <span class="s2">"test.py"</span><span class="p">,</span> <span class="n">line</span> <span class="mi">4</span><span class="p">,</span> <span class="ow">in</span> <span class="n">test</span> <span class="o">|</span> <span class="k">def</span> <span class="nf">test</span><span class="p">(</span><span class="n">x</span><span class="p">):</span> <span class="o">|</span> <span class="o">|</span> <span class="n">File</span> <span class="s2">"hypothesis/core.py"</span><span class="p">,</span> <span class="n">line</span> <span class="mi">1202</span><span class="p">,</span> <span class="ow">in</span> <span class="n">wrapped_test</span> <span class="o">|</span> <span class="k">raise</span> <span class="n">the_error_hypothesis_found</span> <span class="o">|</span> <span class="o">^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^</span> <span class="o">|</span> <span class="n">ExceptionGroup</span><span class="p">:</span> <span class="n">Hypothesis</span> <span class="n">found</span> <span class="mi">2</span> <span class="n">distinct</span> <span class="n">failures</span><span class="o">.</span> <span class="o">+-+----------------</span> <span class="mi">1</span> <span class="o">----------------</span> <span class="o">|</span> <span class="n">Traceback</span> <span class="p">(</span><span class="n">most</span> <span class="n">recent</span> <span class="n">call</span> <span class="n">last</span><span class="p">):</span> <span class="o">|</span> <span class="n">File</span> <span class="s2">"test.py"</span><span class="p">,</span> <span class="n">line</span> <span class="mi">6</span><span class="p">,</span> <span class="ow">in</span> <span class="n">test</span> <span class="o">|</span> <span class="k">assert</span> <span class="n">x</span> <span class="o">></span> <span class="mi">0</span> <span class="o">|</span> <span class="o">^^^^^^^^^^^^</span> <span class="o">|</span> <span class="ne">AssertionError</span><span class="p">:</span> <span class="k">assert</span> <span class="o">-</span><span class="mi">1</span> <span class="o">></span> <span class="mi">0</span> <span class="o">|</span> <span class="o">|</span> <span class="n">Falsifying</span> <span class="n">example</span><span class="p">:</span> <span class="n">test</span><span class="p">(</span> <span class="o">|</span> <span class="n">x</span><span class="o">=-</span><span class="mi">1</span><span class="p">,</span> <span class="o">|</span> <span class="p">)</span> <span class="o">+----------------</span> <span class="mi">2</span> <span class="o">----------------</span> <span class="o">|</span> <span class="n">Traceback</span> <span class="p">(</span><span class="n">most</span> <span class="n">recent</span> <span class="n">call</span> <span class="n">last</span><span class="p">):</span> <span class="o">|</span> <span class="n">File</span> <span class="s2">"test.py"</span><span class="p">,</span> <span class="n">line</span> <span class="mi">5</span><span class="p">,</span> <span class="ow">in</span> <span class="n">test</span> <span class="o">|</span> <span class="k">assert</span> <span class="n">x</span> <span class="o"><</span> <span class="mi">0</span> <span class="o">|</span> <span class="o">^^^^^^^^^^^^</span> <span class="o">|</span> <span class="ne">AssertionError</span><span class="p">:</span> <span class="k">assert</span> <span class="mi">0</span> <span class="o"><</span> <span class="mi">0</span> <span class="o">|</span> <span class="o">|</span> <span class="n">Falsifying</span> <span class="n">example</span><span class="p">:</span> <span class="n">test</span><span class="p">(</span> <span class="o">|</span> <span class="n">x</span><span class="o">=</span><span class="mi">0</span><span class="p">,</span> <span class="o">|</span> <span class="p">)</span> <span class="o">+------------------------------------</span> </pre></div> </div> </section> <section id="non-goals"> <h3><a class="toc-backref" href="#non-goals" role="doc-backlink">Non-goals</a></h3> <p>Tracking multiple notes as a list, rather than by concatenating strings when notes are added, is intended to maintain the distinction between the individual notes. This might be required in specialized use cases, such as translation of the notes by packages like <code class="docutils literal notranslate"><span class="pre">friendly-traceback</span></code>.</p> <p>However, <code class="docutils literal notranslate"><span class="pre">__notes__</span></code> is <em>not</em> intended to carry structured data. If your note is for use by a program rather than display to a human, <a class="reference external" href="https://discuss.python.org/t/accepting-pep-654-exception-groups-and-except/10813/26">we recommend</a> instead (or additionally) choosing a convention for an attribute, e.g. <code class="docutils literal notranslate"><span class="pre">err._parse_errors</span> <span class="pre">=</span> <span class="pre">...</span></code> on the error or <code class="docutils literal notranslate"><span class="pre">ExceptionGroup</span></code>.</p> <p>As a rule of thumb, we suggest that you should prefer <a class="reference external" href="https://docs.python.org/3/tutorial/errors.html#exception-chaining">exception chaining</a> when the error is going to be re-raised or handled as an individual error, and prefer <code class="docutils literal notranslate"><span class="pre">.add_note()</span></code> when you want to avoid changing the exception type or are collecting multiple exception objects to handle together. <a class="footnote-reference brackets" href="#id4" id="id1">[1]</a></p> </section> </section> <section id="specification"> <h2><a class="toc-backref" href="#specification" role="doc-backlink">Specification</a></h2> <p><code class="docutils literal notranslate"><span class="pre">BaseException</span></code> gains a new method <code class="docutils literal notranslate"><span class="pre">.add_note(note:</span> <span class="pre">str)</span></code>. If <code class="docutils literal notranslate"><span class="pre">note</span></code> is a string, <code class="docutils literal notranslate"><span class="pre">.add_note(note)</span></code> appends it to the <code class="docutils literal notranslate"><span class="pre">__notes__</span></code> list, creating the attribute if it does not already exist. If <code class="docutils literal notranslate"><span class="pre">note</span></code> is not a string, <code class="docutils literal notranslate"><span class="pre">.add_note()</span></code> raises <code class="docutils literal notranslate"><span class="pre">TypeError</span></code>.</p> <p>Libraries may clear existing notes by modifying or deleting the <code class="docutils literal notranslate"><span class="pre">__notes__</span></code> list, if it has been created, including clearing all notes with <code class="docutils literal notranslate"><span class="pre">del</span> <span class="pre">err.__notes__</span></code>. This allows full control over the attached notes, without overly complicating the API or adding multiple names to <code class="docutils literal notranslate"><span class="pre">BaseException.__dict__</span></code>.</p> <p>When an exception is displayed by the interpreter’s builtin traceback-rendering code, its notes (if there are any) appear immediately after the exception message, in the order in which they were added, with each note starting on a new line.</p> <p>If <code class="docutils literal notranslate"><span class="pre">__notes__</span></code> has been created, <code class="docutils literal notranslate"><span class="pre">BaseExceptionGroup.subgroup</span></code> and <code class="docutils literal notranslate"><span class="pre">BaseExceptionGroup.split</span></code> create a new list for each new instance, containing the same contents as the original exception group’s <code class="docutils literal notranslate"><span class="pre">__notes__</span></code>.</p> <p>We <em>do not</em> specify the expected behaviour when users have assigned a non-list value to <code class="docutils literal notranslate"><span class="pre">__notes__</span></code>, or a list which contains non-string elements. Implementations might choose to emit warnings, discard or ignore bad values, convert them to strings, raise an exception, or do something else entirely.</p> </section> <section id="backwards-compatibility"> <h2><a class="toc-backref" href="#backwards-compatibility" role="doc-backlink">Backwards Compatibility</a></h2> <p>System-defined or “dunder” names (following the pattern <code class="docutils literal notranslate"><span class="pre">__*__</span></code>) are part of the language specification, with <a class="reference external" href="https://docs.python.org/3/reference/lexical_analysis.html#reserved-classes-of-identifiers">unassigned names reserved for future use and subject to breakage without warning</a>. We are also unaware of any code which <em>would</em> be broken by adding <code class="docutils literal notranslate"><span class="pre">__notes__</span></code>.</p> <p>We were also unable to find any code which would be broken by the addition of <code class="docutils literal notranslate"><span class="pre">BaseException.add_note()</span></code>: while searching Google and <a class="reference external" href="https://grep.app/search?q=.add_note%28&filter[lang][0]=Python">GitHub finds several definitions</a> of an <code class="docutils literal notranslate"><span class="pre">.add_note()</span></code> method, none of them are on a subclass of <code class="docutils literal notranslate"><span class="pre">BaseException</span></code>.</p> </section> <section id="how-to-teach-this"> <h2><a class="toc-backref" href="#how-to-teach-this" role="doc-backlink">How to Teach This</a></h2> <p>The <code class="docutils literal notranslate"><span class="pre">add_note()</span></code> method and <code class="docutils literal notranslate"><span class="pre">__notes__</span></code> attribute will be documented as part of the language standard, and explained as part of <a class="reference external" href="https://github.com/python/cpython/pull/30441">the “Errors and Exceptions” tutorial</a>.</p> </section> <section id="reference-implementation"> <h2><a class="toc-backref" href="#reference-implementation" role="doc-backlink">Reference Implementation</a></h2> <p>Following discussions related to <a class="pep reference internal" href="../pep-0654/" title="PEP 654 – Exception Groups and except*">PEP 654</a> <a class="footnote-reference brackets" href="#id5" id="id2">[2]</a>, an early version of this proposal was <a class="reference external" href="https://github.com/python/cpython/pull/29880">implemented in</a> and released in CPython 3.11.0a3, with a mutable string-or-none <code class="docutils literal notranslate"><span class="pre">__note__</span></code> attribute.</p> <p><a class="reference external" href="https://github.com/python/cpython/pull/31317">CPython PR #31317</a> implements <code class="docutils literal notranslate"><span class="pre">.add_note()</span></code> and <code class="docutils literal notranslate"><span class="pre">__notes__</span></code>.</p> </section> <section id="rejected-ideas"> <h2><a class="toc-backref" href="#rejected-ideas" role="doc-backlink">Rejected Ideas</a></h2> <section id="use-print-or-logging-etc"> <span id="print-idea"></span><h3><a class="toc-backref" href="#use-print-or-logging-etc" role="doc-backlink">Use <code class="docutils literal notranslate"><span class="pre">print()</span></code> (or <code class="docutils literal notranslate"><span class="pre">logging</span></code>, etc.)</a></h3> <p>Reporting explanatory or contextual information about an error by printing or logging has historically been an acceptable workaround. However, we dislike the way this separates the content from the exception object it refers to - which can lead to “orphan” reports if the error was caught and handled later, or merely significant difficulties working out which explanation corresponds to which error. The new <code class="docutils literal notranslate"><span class="pre">ExceptionGroup</span></code> type intensifies these existing challenges.</p> <p>Keeping the <code class="docutils literal notranslate"><span class="pre">__notes__</span></code> attached to the exception object, in the same way as the <code class="docutils literal notranslate"><span class="pre">__traceback__</span></code> attribute, eliminates these problems.</p> </section> <section id="raise-wrapper-explanation-from-err"> <h3><a class="toc-backref" href="#raise-wrapper-explanation-from-err" role="doc-backlink"><code class="docutils literal notranslate"><span class="pre">raise</span> <span class="pre">Wrapper(explanation)</span> <span class="pre">from</span> <span class="pre">err</span></code></a></h3> <p>An alternative pattern is to use exception chaining: by raising a ‘wrapper’ exception containing the context or explanation <code class="docutils literal notranslate"><span class="pre">from</span></code> the current exception, we avoid the separation challenges from <code class="docutils literal notranslate"><span class="pre">print()</span></code>. However, this has two key problems.</p> <p>First, it changes the type of the exception, which is often a breaking change for downstream code. We consider <em>always</em> raising a <code class="docutils literal notranslate"><span class="pre">Wrapper</span></code> exception unacceptably inelegant; but because custom exception types might have any number of required arguments we can’t always create an instance of the <em>same</em> type with our explanation. In cases where the exact exception type is known this can work, such as the standard library <code class="docutils literal notranslate"><span class="pre">http.client</span></code> <a class="reference external" href="https://github.com/python/cpython/blob/69ef1b59983065ddb0b712dac3b04107c5059735/Lib/http/client.py#L596-L597">code</a>, but not for libraries which call user code.</p> <p>Second, exception chaining reports several lines of additional detail, which are distracting for experienced users and can be very confusing for beginners. For example, six of the eleven lines reported for this simple example relate to exception chaining, and are unnecessary with <code class="docutils literal notranslate"><span class="pre">BaseException.add_note()</span></code>:</p> <div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="k">class</span> <span class="nc">Explanation</span><span class="p">(</span><span class="ne">Exception</span><span class="p">):</span> <span class="k">def</span> <span class="fm">__str__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span> <span class="k">return</span> <span class="s2">"</span><span class="se">\n</span><span class="s2">"</span> <span class="o">+</span> <span class="nb">str</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">args</span><span class="p">[</span><span class="mi">0</span><span class="p">])</span> <span class="k">try</span><span class="p">:</span> <span class="k">raise</span> <span class="ne">AssertionError</span><span class="p">(</span><span class="s2">"Failed!"</span><span class="p">)</span> <span class="k">except</span> <span class="ne">Exception</span> <span class="k">as</span> <span class="n">e</span><span class="p">:</span> <span class="k">raise</span> <span class="n">Explanation</span><span class="p">(</span><span class="s2">"You can reproduce this error by ..."</span><span class="p">)</span> <span class="kn">from</span> <span class="nn">e</span> </pre></div> </div> <div class="highlight-default notranslate"><div class="highlight"><pre><span></span>$ python example.py Traceback (most recent call last): File "example.py", line 6, in <module> raise AssertionError(why) AssertionError: Failed! # These lines are The above exception was the direct cause of ... # confusing for new # users, and they Traceback (most recent call last): # only exist due File "example.py", line 8, in <module> # to implementation raise Explanation(msg) from e # constraints :-( Explanation: # Hence this PEP! You can reproduce this error by ... </pre></div> </div> <p><strong>In cases where these two problems do not apply, we encourage use of exception chaining rather than</strong> <code class="docutils literal notranslate"><span class="pre">__notes__</span></code>.</p> </section> <section id="an-assignable-note-attribute"> <h3><a class="toc-backref" href="#an-assignable-note-attribute" role="doc-backlink">An assignable <code class="docutils literal notranslate"><span class="pre">__note__</span></code> attribute</a></h3> <p>The first draft and implementation of this PEP defined a single attribute <code class="docutils literal notranslate"><span class="pre">__note__</span></code>, which defaulted to <code class="docutils literal notranslate"><span class="pre">None</span></code> but could have a string assigned. This is substantially simpler if, and only if, there is at most one note.</p> <p>To promote interoperability and support translation of error messages by libraries such as <code class="docutils literal notranslate"><span class="pre">friendly-traceback</span></code>, without resorting to dubious parsing heuristics, we therefore settled on the <code class="docutils literal notranslate"><span class="pre">.add_note()</span></code>-and-<code class="docutils literal notranslate"><span class="pre">__notes__</span></code> API.</p> </section> <section id="subclass-exception-and-add-note-support-downstream"> <h3><a class="toc-backref" href="#subclass-exception-and-add-note-support-downstream" role="doc-backlink">Subclass Exception and add note support downstream</a></h3> <p>Traceback printing is built into the C code, and reimplemented in pure Python in <code class="docutils literal notranslate"><span class="pre">traceback.py</span></code>. To get <code class="docutils literal notranslate"><span class="pre">err.__notes__</span></code> printed from a downstream implementation would <em>also</em> require writing custom traceback-printing code; while this could be shared between projects and reuse some pieces of traceback.py <a class="footnote-reference brackets" href="#id6" id="id3">[3]</a> we prefer to implement this once, upstream.</p> <p>Custom exception types could implement their <code class="docutils literal notranslate"><span class="pre">__str__</span></code> method to include our proposed <code class="docutils literal notranslate"><span class="pre">__notes__</span></code> semantics, but this would be rarely and inconsistently applicable.</p> </section> <section id="don-t-attach-notes-to-exceptions-just-store-them-in-exceptiongroups"> <h3><a class="toc-backref" href="#don-t-attach-notes-to-exceptions-just-store-them-in-exceptiongroups" role="doc-backlink">Don’t attach notes to <code class="docutils literal notranslate"><span class="pre">Exception</span></code>s, just store them in <code class="docutils literal notranslate"><span class="pre">ExceptionGroup</span></code>s</a></h3> <p>The initial motivation for this PEP was to associate a note with each error in an <code class="docutils literal notranslate"><span class="pre">ExceptionGroup</span></code>. At the cost of a remarkably awkward API and the cross-referencing problem discussed <a class="reference external" href="print_idea">above</a>, this use-case could be supported by storing notes on the <code class="docutils literal notranslate"><span class="pre">ExceptionGroup</span></code> instance instead of on each exception it contains.</p> <p>We believe that the cleaner interface, and other use-cases described above, are sufficient to justify the more general feature proposed by this PEP.</p> </section> <section id="add-a-helper-function-contextlib-add-exc-note"> <h3><a class="toc-backref" href="#add-a-helper-function-contextlib-add-exc-note" role="doc-backlink">Add a helper function <code class="docutils literal notranslate"><span class="pre">contextlib.add_exc_note()</span></code></a></h3> <p>It <a class="reference external" href="https://www.reddit.com/r/Python/comments/rmrvxv/pep_678_enriching_exceptions_with_notes/hptbul1/">was suggested</a> that we add a utility such as the one below to the standard library. We do not see this idea as core to the proposal of this PEP, and thus leave it for later or downstream implementation - perhaps based on this example code:</p> <div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="nd">@contextlib</span><span class="o">.</span><span class="n">contextmanager</span> <span class="k">def</span> <span class="nf">add_exc_note</span><span class="p">(</span><span class="n">note</span><span class="p">:</span> <span class="nb">str</span><span class="p">):</span> <span class="k">try</span><span class="p">:</span> <span class="k">yield</span> <span class="k">except</span> <span class="ne">Exception</span> <span class="k">as</span> <span class="n">err</span><span class="p">:</span> <span class="n">err</span><span class="o">.</span><span class="n">add_note</span><span class="p">(</span><span class="n">note</span><span class="p">)</span> <span class="k">raise</span> <span class="k">with</span> <span class="n">add_exc_note</span><span class="p">(</span><span class="sa">f</span><span class="s2">"While attempting to frobnicate </span><span class="si">{</span><span class="n">item</span><span class="si">=}</span><span class="s2">"</span><span class="p">):</span> <span class="n">frobnicate_or_raise</span><span class="p">(</span><span class="n">item</span><span class="p">)</span> </pre></div> </div> </section> <section id="augment-the-raise-statement"> <h3><a class="toc-backref" href="#augment-the-raise-statement" role="doc-backlink">Augment the <code class="docutils literal notranslate"><span class="pre">raise</span></code> statement</a></h3> <p>One discussion proposed <code class="docutils literal notranslate"><span class="pre">raise</span> <span class="pre">Exception()</span> <span class="pre">with</span> <span class="pre">"note</span> <span class="pre">contents"</span></code>, but this does not address the original motivation of compatibility with <code class="docutils literal notranslate"><span class="pre">ExceptionGroup</span></code>.</p> <p>Furthermore, we do not believe that the problem we are solving requires or justifies new language syntax.</p> </section> </section> <section id="acknowledgements"> <h2><a class="toc-backref" href="#acknowledgements" role="doc-backlink">Acknowledgements</a></h2> <p>We wish to thank the many people who have assisted us through conversation, code review, design advice, and implementation: Adam Turner, Alex Grönholm, André Roberge, Barry Warsaw, Brett Cannon, CAM Gerlach, Carol Willing, Damian, Erlend Aasland, Etienne Pot, Gregory Smith, Guido van Rossum, Irit Katriel, Jelle Zijlstra, Ken Jin, Kumar Aditya, Mark Shannon, Matti Picus, Petr Viktorin, Will McGugan, and pseudonymous commenters on Discord and Reddit.</p> </section> <section id="references"> <h2><a class="toc-backref" href="#references" role="doc-backlink">References</a></h2> <aside class="footnote-list brackets"> <aside class="footnote brackets" id="id4" role="doc-footnote"> <dt class="label" id="id4">[<a href="#id1">1</a>]</dt> <dd>this principle was established in the 2003 mail thread which led to <a class="pep reference internal" href="../pep-3134/" title="PEP 3134 – Exception Chaining and Embedded Tracebacks">PEP 3134</a>, and included a proposal for a group-of-exceptions type! <a class="reference external" href="https://mail.python.org/pipermail/python-dev/2003-January/032492.html">https://mail.python.org/pipermail/python-dev/2003-January/032492.html</a></aside> <aside class="footnote brackets" id="id5" role="doc-footnote"> <dt class="label" id="id5">[<a href="#id2">2</a>]</dt> <dd>particularly those at <a class="reference external" href="https://bugs.python.org/issue45607">https://bugs.python.org/issue45607</a>, <a class="reference external" href="https://discuss.python.org/t/accepting-pep-654-exception-groups-and-except/10813/9">https://discuss.python.org/t/accepting-pep-654-exception-groups-and-except/10813/9</a>, <a class="reference external" href="https://github.com/python/cpython/pull/28569#discussion_r721768348">https://github.com/python/cpython/pull/28569#discussion_r721768348</a>, and</aside> <aside class="footnote brackets" id="id6" role="doc-footnote"> <dt class="label" id="id6">[<a href="#id3">3</a>]</dt> <dd>We note that the <code class="docutils literal notranslate"><span class="pre">exceptiongroup</span></code> backport package maintains an exception hook and monkeypatch for <code class="docutils literal notranslate"><span class="pre">TracebackException</span></code> for Pythons older than 3.11, and encourage library authors to avoid creating additional and incompatible backports. We also reiterate our preference for builtin support which makes such measures unnecessary.</aside> </aside> </section> <section id="copyright"> <h2><a class="toc-backref" href="#copyright" role="doc-backlink">Copyright</a></h2> <p>This document is placed in the public domain or under the CC0-1.0-Universal license, whichever is more permissive.</p> </section> </section> <hr class="docutils" /> <p>Source: <a class="reference external" href="https://github.com/python/peps/blob/main/peps/pep-0678.rst">https://github.com/python/peps/blob/main/peps/pep-0678.rst</a></p> <p>Last modified: <a class="reference external" href="https://github.com/python/peps/commits/main/peps/pep-0678.rst">2023-10-10 15:15:34 GMT</a></p> </article> <nav id="pep-sidebar"> <h2>Contents</h2> <ul> <li><a class="reference internal" href="#abstract">Abstract</a></li> <li><a class="reference internal" href="#motivation">Motivation</a><ul> <li><a class="reference internal" href="#example-usage">Example usage</a></li> <li><a class="reference internal" href="#non-goals">Non-goals</a></li> </ul> </li> <li><a class="reference internal" href="#specification">Specification</a></li> <li><a class="reference internal" href="#backwards-compatibility">Backwards Compatibility</a></li> <li><a class="reference internal" href="#how-to-teach-this">How to Teach This</a></li> <li><a class="reference internal" href="#reference-implementation">Reference Implementation</a></li> <li><a class="reference internal" href="#rejected-ideas">Rejected Ideas</a><ul> <li><a class="reference internal" href="#use-print-or-logging-etc">Use <code class="docutils literal notranslate"><span class="pre">print()</span></code> (or <code class="docutils literal notranslate"><span class="pre">logging</span></code>, etc.)</a></li> <li><a class="reference internal" href="#raise-wrapper-explanation-from-err"><code class="docutils literal notranslate"><span class="pre">raise</span> <span class="pre">Wrapper(explanation)</span> <span class="pre">from</span> <span class="pre">err</span></code></a></li> <li><a class="reference internal" href="#an-assignable-note-attribute">An assignable <code class="docutils literal notranslate"><span class="pre">__note__</span></code> attribute</a></li> <li><a class="reference internal" href="#subclass-exception-and-add-note-support-downstream">Subclass Exception and add note support downstream</a></li> <li><a class="reference internal" href="#don-t-attach-notes-to-exceptions-just-store-them-in-exceptiongroups">Don’t attach notes to <code class="docutils literal notranslate"><span class="pre">Exception</span></code>s, just store them in <code class="docutils literal notranslate"><span class="pre">ExceptionGroup</span></code>s</a></li> <li><a class="reference internal" href="#add-a-helper-function-contextlib-add-exc-note">Add a helper function <code class="docutils literal notranslate"><span class="pre">contextlib.add_exc_note()</span></code></a></li> <li><a class="reference internal" href="#augment-the-raise-statement">Augment the <code class="docutils literal notranslate"><span class="pre">raise</span></code> statement</a></li> </ul> </li> <li><a class="reference internal" href="#acknowledgements">Acknowledgements</a></li> <li><a class="reference internal" href="#references">References</a></li> <li><a class="reference internal" href="#copyright">Copyright</a></li> </ul> <br> <a id="source" href="https://github.com/python/peps/blob/main/peps/pep-0678.rst">Page Source (GitHub)</a> </nav> </section> <script src="../_static/colour_scheme.js"></script> <script src="../_static/wrap_tables.js"></script> <script src="../_static/sticky_banner.js"></script> </body> </html>