CINXE.COM

<!DOCTYPE html> <html lang="en"> <head> <meta charset="utf-8"> <meta name="viewport" content="width=device-width, initial-scale=1.0"> <meta name="color-scheme" content="light dark"> <title>PEP 208 – Reworking the Coercion Model | peps.python.org</title> <link rel="shortcut icon" href="../_static/py.png"> <link rel="canonical" href="https://peps.python.org/pep-0208/"> <link rel="stylesheet" href="../_static/style.css" type="text/css"> <link rel="stylesheet" href="../_static/mq.css" type="text/css"> <link rel="stylesheet" href="../_static/pygments.css" type="text/css" media="(prefers-color-scheme: light)" id="pyg-light"> <link rel="stylesheet" href="../_static/pygments_dark.css" type="text/css" media="(prefers-color-scheme: dark)" id="pyg-dark"> <link rel="alternate" type="application/rss+xml" title="Latest PEPs" href="https://peps.python.org/peps.rss"> <meta property="og:title" content='PEP 208 – Reworking the Coercion Model | peps.python.org'> <meta property="og:description" content="Many Python types implement numeric operations. When the arguments of a numeric operation are of different types, the interpreter tries to coerce the arguments into a common type. The numeric operation is then performed using this common type. This P..."> <meta property="og:type" content="website"> <meta property="og:url" content="https://peps.python.org/pep-0208/"> <meta property="og:site_name" content="Python Enhancement Proposals (PEPs)"> <meta property="og:image" content="https://peps.python.org/_static/og-image.png"> <meta property="og:image:alt" content="Python PEPs"> <meta property="og:image:width" content="200"> <meta property="og:image:height" content="200"> <meta name="description" content="Many Python types implement numeric operations. When the arguments of a numeric operation are of different types, the interpreter tries to coerce the arguments into a common type. The numeric operation is then performed using this common type. This P..."> <meta name="theme-color" content="#3776ab"> </head> <body> <svg xmlns="http://www.w3.org/2000/svg" style="display: none;"> <symbol id="svg-sun-half" viewBox="0 0 24 24" pointer-events="all"> <title>Following system colour scheme</title> <svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"> <circle cx="12" cy="12" r="9"></circle> <path d="M12 3v18m0-12l4.65-4.65M12 14.3l7.37-7.37M12 19.6l8.85-8.85"></path> </svg> </symbol> <symbol id="svg-moon" viewBox="0 0 24 24" pointer-events="all"> <title>Selected dark colour scheme</title> <svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"> <path stroke="none" d="M0 0h24v24H0z" fill="none"></path> <path d="M12 3c.132 0 .263 0 .393 0a7.5 7.5 0 0 0 7.92 12.446a9 9 0 1 1 -8.313 -12.454z"></path> </svg> </symbol> <symbol id="svg-sun" viewBox="0 0 24 24" pointer-events="all"> <title>Selected light colour scheme</title> <svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"> <circle cx="12" cy="12" r="5"></circle> <line x1="12" y1="1" x2="12" y2="3"></line> <line x1="12" y1="21" x2="12" y2="23"></line> <line x1="4.22" y1="4.22" x2="5.64" y2="5.64"></line> <line x1="18.36" y1="18.36" x2="19.78" y2="19.78"></line> <line x1="1" y1="12" x2="3" y2="12"></line> <line x1="21" y1="12" x2="23" y2="12"></line> <line x1="4.22" y1="19.78" x2="5.64" y2="18.36"></line> <line x1="18.36" y1="5.64" x2="19.78" y2="4.22"></line> </svg> </symbol> </svg> <script> document.documentElement.dataset.colour_scheme = localStorage.getItem("colour_scheme") || "auto" </script> <section id="pep-page-section"> <header> <h1>Python Enhancement Proposals</h1> <ul class="breadcrumbs"> <li><a href="https://www.python.org/" title="The Python Programming Language">Python</a> » </li> <li><a href="../pep-0000/">PEP Index</a> » </li> <li>PEP 208</li> </ul> <button id="colour-scheme-cycler" onClick="setColourScheme(nextColourScheme())"> <svg aria-hidden="true" class="colour-scheme-icon-when-auto"><use href="#svg-sun-half"></use></svg> <svg aria-hidden="true" class="colour-scheme-icon-when-dark"><use href="#svg-moon"></use></svg> <svg aria-hidden="true" class="colour-scheme-icon-when-light"><use href="#svg-sun"></use></svg> <span class="visually-hidden">Toggle light / dark / auto colour theme</span> </button> </header> <article> <section id="pep-content"> <h1 class="page-title">PEP 208 – Reworking the Coercion Model</h1> <dl class="rfc2822 field-list simple"> <dt class="field-odd">Author<span class="colon">:</span></dt> <dd class="field-odd">Neil Schemenauer <nas at arctrix.com>, Marc-André Lemburg <mal at lemburg.com></dd> <dt class="field-even">Status<span class="colon">:</span></dt> <dd class="field-even"><abbr title="Accepted and implementation complete, or no longer active">Final</abbr></dd> <dt class="field-odd">Type<span class="colon">:</span></dt> <dd class="field-odd"><abbr title="Normative PEP with a new feature for Python, implementation change for CPython or interoperability standard for the ecosystem">Standards Track</abbr></dd> <dt class="field-even">Created<span class="colon">:</span></dt> <dd class="field-even">04-Dec-2000</dd> <dt class="field-odd">Python-Version<span class="colon">:</span></dt> <dd class="field-odd">2.1</dd> <dt class="field-even">Post-History<span class="colon">:</span></dt> <dd class="field-even"><p></p></dd> </dl> <hr class="docutils" /> <section id="contents"> <details><summary>Table of Contents</summary><ul class="simple"> <li><a class="reference internal" href="#abstract">Abstract</a></li> <li><a class="reference internal" href="#rationale">Rationale</a></li> <li><a class="reference internal" href="#specification">Specification</a></li> <li><a class="reference internal" href="#reference-implementation">Reference Implementation</a></li> <li><a class="reference internal" href="#credits">Credits</a></li> <li><a class="reference internal" href="#copyright">Copyright</a></li> <li><a class="reference internal" href="#references">References</a></li> </ul> </details></section> <section id="abstract"> <h2><a class="toc-backref" href="#abstract" role="doc-backlink">Abstract</a></h2> <p>Many Python types implement numeric operations. When the arguments of a numeric operation are of different types, the interpreter tries to coerce the arguments into a common type. The numeric operation is then performed using this common type. This PEP proposes a new type flag to indicate that arguments to a type’s numeric operations should not be coerced. Operations that do not support the supplied types indicate it by returning a new singleton object. Types which do not set the type flag are handled in a backwards compatible manner. Allowing operations handle different types is often simpler, more flexible, and faster than having the interpreter do coercion.</p> </section> <section id="rationale"> <h2><a class="toc-backref" href="#rationale" role="doc-backlink">Rationale</a></h2> <p>When implementing numeric or other related operations, it is often desirable to provide not only operations between operands of one type only, e.g. integer + integer, but to generalize the idea behind the operation to other type combinations as well, e.g. integer + float.</p> <p>A common approach to this mixed type situation is to provide a method of “lifting” the operands to a common type (coercion) and then use that type’s operand method as execution mechanism. Yet, this strategy has a few drawbacks:</p> <ul class="simple"> <li>the “lifting” process creates at least one new (temporary) operand object,</li> <li>since the coercion method is not being told about the operation that is to follow, it is not possible to implement operation specific coercion of types,</li> <li>there is no elegant way to solve situations were a common type is not at hand, and</li> <li>the coercion method will always have to be called prior to the operation’s method itself.</li> </ul> <p>A fix for this situation is obviously needed, since these drawbacks make implementations of types needing these features very cumbersome, if not impossible. As an example, have a look at the <code class="docutils literal notranslate"><span class="pre">DateTime</span></code> and <code class="docutils literal notranslate"><span class="pre">DateTimeDelta</span></code> <a class="footnote-reference brackets" href="#id4" id="id1">[1]</a> types, the first being absolute, the second relative. You can always add a relative value to an absolute one, giving a new absolute value. Yet, there is no common type which the existing coercion mechanism could use to implement that operation.</p> <p>Currently, <code class="docutils literal notranslate"><span class="pre">PyInstance</span></code> types are treated specially by the interpreter in that their numeric methods are passed arguments of different types. Removing this special case simplifies the interpreter and allows other types to implement numeric methods that behave like instance types. This is especially useful for extension types like ExtensionClass.</p> </section> <section id="specification"> <h2><a class="toc-backref" href="#specification" role="doc-backlink">Specification</a></h2> <p>Instead of using a central coercion method, the process of handling different operand types is simply left to the operation. If the operation finds that it cannot handle the given operand type combination, it may return a special singleton as indicator.</p> <p>Note that “numbers” (anything that implements the number protocol, or part of it) written in Python already use the first part of this strategy - it is the C level API that we focus on here.</p> <p>To maintain nearly 100% backward compatibility we have to be very careful to make numbers that don’t know anything about the new strategy (old style numbers) work just as well as those that expect the new scheme (new style numbers). Furthermore, binary compatibility is a must, meaning that the interpreter may only access and use new style operations if the number indicates the availability of these.</p> <p>A new style number is considered by the interpreter as such if and only if it sets the type flag <code class="docutils literal notranslate"><span class="pre">Py_TPFLAGS_CHECKTYPES</span></code>. The main difference between an old style number and a new style one is that the numeric slot functions can no longer assume to be passed arguments of identical type. New style slots must check all arguments for proper type and implement the necessary conversions themselves. This may seem to cause more work on the behalf of the type implementor, but is in fact no more difficult than writing the same kind of routines for an old style coercion slot.</p> <p>If a new style slot finds that it cannot handle the passed argument type combination, it may return a new reference of the special singleton <code class="docutils literal notranslate"><span class="pre">Py_NotImplemented</span></code> to the caller. This will cause the caller to try the other operands operation slots until it finds a slot that does implement the operation for the specific type combination. If none of the possible slots succeed, it raises a <code class="docutils literal notranslate"><span class="pre">TypeError</span></code>.</p> <p>To make the implementation easy to understand (the whole topic is esoteric enough), a new layer in the handling of numeric operations is introduced. This layer takes care of all the different cases that need to be taken into account when dealing with all the possible combinations of old and new style numbers. It is implemented by the two static functions <code class="docutils literal notranslate"><span class="pre">binary_op()</span></code> and <code class="docutils literal notranslate"><span class="pre">ternary_op()</span></code>, which are both internal functions that only the functions in Objects/abstract.c have access to. The numeric API (<code class="docutils literal notranslate"><span class="pre">PyNumber_*</span></code>) is easy to adapt to this new layer.</p> <p>As a side-effect all numeric slots can be NULL-checked (this has to be done anyway, so the added feature comes at no extra cost).</p> <p>The scheme used by the layer to execute a binary operation is as follows:</p> <table class="docutils align-default"> <thead> <tr class="row-odd"><th class="head">v</th> <th class="head">w</th> <th class="head">Action taken</th> </tr> </thead> <tbody> <tr class="row-even"><td>new</td> <td>new</td> <td>v.op(v,w), w.op(v,w)</td> </tr> <tr class="row-odd"><td>new</td> <td>old</td> <td>v.op(v,w), coerce(v,w), v.op(v,w)</td> </tr> <tr class="row-even"><td>old</td> <td>new</td> <td>w.op(v,w), coerce(v,w), v.op(v,w)</td> </tr> <tr class="row-odd"><td>old</td> <td>old</td> <td>coerce(v,w), v.op(v,w)</td> </tr> </tbody> </table> <p>The indicated action sequence is executed from left to right until either the operation succeeds and a valid result (!= <code class="docutils literal notranslate"><span class="pre">Py_NotImplemented</span></code>) is returned or an exception is raised. Exceptions are returned to the calling function as-is. If a slot returns <code class="docutils literal notranslate"><span class="pre">Py_NotImplemented</span></code>, the next item in the sequence is executed.</p> <p>Note that coerce(v,w) will use the old style <code class="docutils literal notranslate"><span class="pre">nb_coerce</span></code> slot methods via a call to <code class="docutils literal notranslate"><span class="pre">PyNumber_Coerce()</span></code>.</p> <p>Ternary operations have a few more cases to handle:</p> <table class="docutils align-default"> <thead> <tr class="row-odd"><th class="head">v</th> <th class="head">w</th> <th class="head">z</th> <th class="head">Action taken</th> </tr> </thead> <tbody> <tr class="row-even"><td>new</td> <td>new</td> <td>new</td> <td>v.op(v,w,z), w.op(v,w,z), z.op(v,w,z)</td> </tr> <tr class="row-odd"><td>new</td> <td>old</td> <td>new</td> <td>v.op(v,w,z), z.op(v,w,z), coerce(v,w,z), v.op(v,w,z)</td> </tr> <tr class="row-even"><td>old</td> <td>new</td> <td>new</td> <td>w.op(v,w,z), z.op(v,w,z), coerce(v,w,z), v.op(v,w,z)</td> </tr> <tr class="row-odd"><td>old</td> <td>old</td> <td>new</td> <td>z.op(v,w,z), coerce(v,w,z), v.op(v,w,z)</td> </tr> <tr class="row-even"><td>new</td> <td>new</td> <td>old</td> <td>v.op(v,w,z), w.op(v,w,z), coerce(v,w,z), v.op(v,w,z)</td> </tr> <tr class="row-odd"><td>new</td> <td>old</td> <td>old</td> <td>v.op(v,w,z), coerce(v,w,z), v.op(v,w,z)</td> </tr> <tr class="row-even"><td>old</td> <td>new</td> <td>old</td> <td>w.op(v,w,z), coerce(v,w,z), v.op(v,w,z)</td> </tr> <tr class="row-odd"><td>old</td> <td>old</td> <td>old</td> <td>coerce(v,w,z), v.op(v,w,z)</td> </tr> </tbody> </table> <p>The same notes as above, except that coerce(v,w,z) actually does:</p> <div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="k">if</span> <span class="n">z</span> <span class="o">!=</span> <span class="n">Py_None</span><span class="p">:</span> <span class="n">coerce</span><span class="p">(</span><span class="n">v</span><span class="p">,</span><span class="n">w</span><span class="p">),</span> <span class="n">coerce</span><span class="p">(</span><span class="n">v</span><span class="p">,</span><span class="n">z</span><span class="p">),</span> <span class="n">coerce</span><span class="p">(</span><span class="n">w</span><span class="p">,</span><span class="n">z</span><span class="p">)</span> <span class="k">else</span><span class="p">:</span> <span class="c1"># treat z as absent variable</span> <span class="n">coerce</span><span class="p">(</span><span class="n">v</span><span class="p">,</span><span class="n">w</span><span class="p">)</span> </pre></div> </div> <p>The current implementation uses this scheme already (there’s only one ternary slot: <code class="docutils literal notranslate"><span class="pre">nb_pow(a,b,c))</span></code>.</p> <p>Note that the numeric protocol is also used for some other related tasks, e.g. sequence concatenation. These can also benefit from the new mechanism by implementing right-hand operations for type combinations that would otherwise fail to work. As an example, take string concatenation: currently you can only do string + string. With the new mechanism, a new string-like type could implement new_type + string and string + new_type, even though strings don’t know anything about new_type.</p> <p>Since comparisons also rely on coercion (every time you compare an integer to a float, the integer is first converted to float and then compared…), a new slot to handle numeric comparisons is needed:</p> <div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">PyObject</span> <span class="o">*</span><span class="n">nb_cmp</span><span class="p">(</span><span class="n">PyObject</span> <span class="o">*</span><span class="n">v</span><span class="p">,</span> <span class="n">PyObject</span> <span class="o">*</span><span class="n">w</span><span class="p">)</span> </pre></div> </div> <p>This slot should compare the two objects and return an integer object stating the result. Currently, this result integer may only be -1, 0, 1. If the slot cannot handle the type combination, it may return a reference to <code class="docutils literal notranslate"><span class="pre">Py_NotImplemented</span></code>. [XXX Note that this slot is still in flux since it should take into account rich comparisons (i.e. <a class="pep reference internal" href="../pep-0207/" title="PEP 207 – Rich Comparisons">PEP 207</a>).]</p> <p>Numeric comparisons are handled by a new numeric protocol API:</p> <div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">PyObject</span> <span class="o">*</span><span class="n">PyNumber_Compare</span><span class="p">(</span><span class="n">PyObject</span> <span class="o">*</span><span class="n">v</span><span class="p">,</span> <span class="n">PyObject</span> <span class="o">*</span><span class="n">w</span><span class="p">)</span> </pre></div> </div> <p>This function compare the two objects as “numbers” and return an integer object stating the result. Currently, this result integer may only be -1, 0, 1. In case the operation cannot be handled by the given objects, a <code class="docutils literal notranslate"><span class="pre">TypeError</span></code> is raised.</p> <p>The <code class="docutils literal notranslate"><span class="pre">PyObject_Compare()</span></code> API needs to adjusted accordingly to make use of this new API.</p> <p>Other changes include adapting some of the built-in functions (e.g. <code class="docutils literal notranslate"><span class="pre">cmp()</span></code>) to use this API as well. Also, <code class="docutils literal notranslate"><span class="pre">PyNumber_CoerceEx()</span></code> will need to check for new style numbers before calling the <code class="docutils literal notranslate"><span class="pre">nb_coerce</span></code> slot. New style numbers don’t provide a coercion slot and thus cannot be explicitly coerced.</p> </section> <section id="reference-implementation"> <h2><a class="toc-backref" href="#reference-implementation" role="doc-backlink">Reference Implementation</a></h2> <p>A preliminary patch for the CVS version of Python is available through the Source Forge patch manager <a class="footnote-reference brackets" href="#id5" id="id2">[2]</a>.</p> </section> <section id="credits"> <h2><a class="toc-backref" href="#credits" role="doc-backlink">Credits</a></h2> <p>This PEP and the patch are heavily based on work done by Marc-André Lemburg <a class="footnote-reference brackets" href="#id6" id="id3">[3]</a>.</p> </section> <section id="copyright"> <h2><a class="toc-backref" href="#copyright" role="doc-backlink">Copyright</a></h2> <p>This document has been placed in the public domain.</p> </section> <section id="references"> <h2><a class="toc-backref" href="#references" role="doc-backlink">References</a></h2> <aside class="footnote-list brackets"> <aside class="footnote brackets" id="id4" role="doc-footnote"> <dt class="label" id="id4">[<a href="#id1">1</a>]</dt> <dd><a class="reference external" href="http://www.lemburg.com/files/python/mxDateTime.html">http://www.lemburg.com/files/python/mxDateTime.html</a></aside> <aside class="footnote brackets" id="id5" role="doc-footnote"> <dt class="label" id="id5">[<a href="#id2">2</a>]</dt> <dd><a class="reference external" href="http://sourceforge.net/patch/?func=detailpatch&patch_id=102652&group_id=5470">http://sourceforge.net/patch/?func=detailpatch&patch_id=102652&group_id=5470</a></aside> <aside class="footnote brackets" id="id6" role="doc-footnote"> <dt class="label" id="id6">[<a href="#id3">3</a>]</dt> <dd><a class="reference external" href="http://www.lemburg.com/files/python/CoercionProposal.html">http://www.lemburg.com/files/python/CoercionProposal.html</a></aside> </aside> </section> </section> <hr class="docutils" /> <p>Source: <a class="reference external" href="https://github.com/python/peps/blob/main/peps/pep-0208.rst">https://github.com/python/peps/blob/main/peps/pep-0208.rst</a></p> <p>Last modified: <a class="reference external" href="https://github.com/python/peps/commits/main/peps/pep-0208.rst">2025-02-01 08:55:40 GMT</a></p> </article> <nav id="pep-sidebar"> <h2>Contents</h2> <ul> <li><a class="reference internal" href="#abstract">Abstract</a></li> <li><a class="reference internal" href="#rationale">Rationale</a></li> <li><a class="reference internal" href="#specification">Specification</a></li> <li><a class="reference internal" href="#reference-implementation">Reference Implementation</a></li> <li><a class="reference internal" href="#credits">Credits</a></li> <li><a class="reference internal" href="#copyright">Copyright</a></li> <li><a class="reference internal" href="#references">References</a></li> </ul> <br> <a id="source" href="https://github.com/python/peps/blob/main/peps/pep-0208.rst">Page Source (GitHub)</a> </nav> </section> <script src="../_static/colour_scheme.js"></script> <script src="../_static/wrap_tables.js"></script> <script src="../_static/sticky_banner.js"></script> </body> </html>