CINXE.COM
PEP 443 – Single-dispatch generic functions | peps.python.org
<!DOCTYPE html> <html lang="en"> <head> <meta charset="utf-8"> <meta name="viewport" content="width=device-width, initial-scale=1.0"> <meta name="color-scheme" content="light dark"> <title>PEP 443 – Single-dispatch generic functions | peps.python.org</title> <link rel="shortcut icon" href="../_static/py.png"> <link rel="canonical" href="https://peps.python.org/pep-0443/"> <link rel="stylesheet" href="../_static/style.css" type="text/css"> <link rel="stylesheet" href="../_static/mq.css" type="text/css"> <link rel="stylesheet" href="../_static/pygments.css" type="text/css" media="(prefers-color-scheme: light)" id="pyg-light"> <link rel="stylesheet" href="../_static/pygments_dark.css" type="text/css" media="(prefers-color-scheme: dark)" id="pyg-dark"> <link rel="alternate" type="application/rss+xml" title="Latest PEPs" href="https://peps.python.org/peps.rss"> <meta property="og:title" content='PEP 443 – Single-dispatch generic functions | peps.python.org'> <meta property="og:description" content="This PEP proposes a new mechanism in the functools standard library module that provides a simple form of generic programming known as single-dispatch generic functions."> <meta property="og:type" content="website"> <meta property="og:url" content="https://peps.python.org/pep-0443/"> <meta property="og:site_name" content="Python Enhancement Proposals (PEPs)"> <meta property="og:image" content="https://peps.python.org/_static/og-image.png"> <meta property="og:image:alt" content="Python PEPs"> <meta property="og:image:width" content="200"> <meta property="og:image:height" content="200"> <meta name="description" content="This PEP proposes a new mechanism in the functools standard library module that provides a simple form of generic programming known as single-dispatch generic functions."> <meta name="theme-color" content="#3776ab"> </head> <body> <svg xmlns="http://www.w3.org/2000/svg" style="display: none;"> <symbol id="svg-sun-half" viewBox="0 0 24 24" pointer-events="all"> <title>Following system colour scheme</title> <svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"> <circle cx="12" cy="12" r="9"></circle> <path d="M12 3v18m0-12l4.65-4.65M12 14.3l7.37-7.37M12 19.6l8.85-8.85"></path> </svg> </symbol> <symbol id="svg-moon" viewBox="0 0 24 24" pointer-events="all"> <title>Selected dark colour scheme</title> <svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"> <path stroke="none" d="M0 0h24v24H0z" fill="none"></path> <path d="M12 3c.132 0 .263 0 .393 0a7.5 7.5 0 0 0 7.92 12.446a9 9 0 1 1 -8.313 -12.454z"></path> </svg> </symbol> <symbol id="svg-sun" viewBox="0 0 24 24" pointer-events="all"> <title>Selected light colour scheme</title> <svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"> <circle cx="12" cy="12" r="5"></circle> <line x1="12" y1="1" x2="12" y2="3"></line> <line x1="12" y1="21" x2="12" y2="23"></line> <line x1="4.22" y1="4.22" x2="5.64" y2="5.64"></line> <line x1="18.36" y1="18.36" x2="19.78" y2="19.78"></line> <line x1="1" y1="12" x2="3" y2="12"></line> <line x1="21" y1="12" x2="23" y2="12"></line> <line x1="4.22" y1="19.78" x2="5.64" y2="18.36"></line> <line x1="18.36" y1="5.64" x2="19.78" y2="4.22"></line> </svg> </symbol> </svg> <script> document.documentElement.dataset.colour_scheme = localStorage.getItem("colour_scheme") || "auto" </script> <section id="pep-page-section"> <header> <h1>Python Enhancement Proposals</h1> <ul class="breadcrumbs"> <li><a href="https://www.python.org/" title="The Python Programming Language">Python</a> » </li> <li><a href="../pep-0000/">PEP Index</a> » </li> <li>PEP 443</li> </ul> <button id="colour-scheme-cycler" onClick="setColourScheme(nextColourScheme())"> <svg aria-hidden="true" class="colour-scheme-icon-when-auto"><use href="#svg-sun-half"></use></svg> <svg aria-hidden="true" class="colour-scheme-icon-when-dark"><use href="#svg-moon"></use></svg> <svg aria-hidden="true" class="colour-scheme-icon-when-light"><use href="#svg-sun"></use></svg> <span class="visually-hidden">Toggle light / dark / auto colour theme</span> </button> </header> <article> <section id="pep-content"> <h1 class="page-title">PEP 443 – Single-dispatch generic functions</h1> <dl class="rfc2822 field-list simple"> <dt class="field-odd">Author<span class="colon">:</span></dt> <dd class="field-odd">Łukasz Langa <lukasz at python.org></dd> <dt class="field-even">Discussions-To<span class="colon">:</span></dt> <dd class="field-even"><a class="reference external" href="https://mail.python.org/archives/list/python-dev@python.org/">Python-Dev list</a></dd> <dt class="field-odd">Status<span class="colon">:</span></dt> <dd class="field-odd"><abbr title="Accepted and implementation complete, or no longer active">Final</abbr></dd> <dt class="field-even">Type<span class="colon">:</span></dt> <dd class="field-even"><abbr title="Normative PEP with a new feature for Python, implementation change for CPython or interoperability standard for the ecosystem">Standards Track</abbr></dd> <dt class="field-odd">Created<span class="colon">:</span></dt> <dd class="field-odd">22-May-2013</dd> <dt class="field-even">Python-Version<span class="colon">:</span></dt> <dd class="field-even">3.4</dd> <dt class="field-odd">Post-History<span class="colon">:</span></dt> <dd class="field-odd">22-May-2013, 25-May-2013, 31-May-2013</dd> <dt class="field-even">Replaces<span class="colon">:</span></dt> <dd class="field-even"><a class="reference external" href="../pep-0245/">245</a>, <a class="reference external" href="../pep-0246/">246</a>, <a class="reference external" href="../pep-3124/">3124</a></dd> </dl> <hr class="docutils" /> <section id="contents"> <details><summary>Table of Contents</summary><ul class="simple"> <li><a class="reference internal" href="#abstract">Abstract</a></li> <li><a class="reference internal" href="#rationale-and-goals">Rationale and Goals</a></li> <li><a class="reference internal" href="#user-api">User API</a></li> <li><a class="reference internal" href="#implementation-notes">Implementation Notes</a><ul> <li><a class="reference internal" href="#abstract-base-classes">Abstract Base Classes</a></li> </ul> </li> <li><a class="reference internal" href="#usage-patterns">Usage Patterns</a></li> <li><a class="reference internal" href="#alternative-approaches">Alternative approaches</a></li> <li><a class="reference internal" href="#acknowledgements">Acknowledgements</a></li> <li><a class="reference internal" href="#references">References</a></li> <li><a class="reference internal" href="#copyright">Copyright</a></li> </ul> </details></section> <section id="abstract"> <h2><a class="toc-backref" href="#abstract" role="doc-backlink">Abstract</a></h2> <p>This PEP proposes a new mechanism in the <code class="docutils literal notranslate"><span class="pre">functools</span></code> standard library module that provides a simple form of generic programming known as single-dispatch generic functions.</p> <p>A <strong>generic function</strong> is composed of multiple functions implementing the same operation for different types. Which implementation should be used during a call is determined by the dispatch algorithm. When the implementation is chosen based on the type of a single argument, this is known as <strong>single dispatch</strong>.</p> </section> <section id="rationale-and-goals"> <h2><a class="toc-backref" href="#rationale-and-goals" role="doc-backlink">Rationale and Goals</a></h2> <p>Python has always provided a variety of built-in and standard-library generic functions, such as <code class="docutils literal notranslate"><span class="pre">len()</span></code>, <code class="docutils literal notranslate"><span class="pre">iter()</span></code>, <code class="docutils literal notranslate"><span class="pre">pprint.pprint()</span></code>, <code class="docutils literal notranslate"><span class="pre">copy.copy()</span></code>, and most of the functions in the <code class="docutils literal notranslate"><span class="pre">operator</span></code> module. However, it currently:</p> <ol class="arabic simple"> <li>does not have a simple or straightforward way for developers to create new generic functions,</li> <li>does not have a standard way for methods to be added to existing generic functions (i.e., some are added using registration functions, others require defining <code class="docutils literal notranslate"><span class="pre">__special__</span></code> methods, possibly by monkeypatching).</li> </ol> <p>In addition, it is currently a common anti-pattern for Python code to inspect the types of received arguments, in order to decide what to do with the objects.</p> <p>For example, code may wish to accept either an object of some type, or a sequence of objects of that type. Currently, the “obvious way” to do this is by type inspection, but this is brittle and closed to extension.</p> <p>Abstract Base Classes make it easier to discover present behaviour, but don’t help adding new behaviour. A developer using an already-written library may be unable to change how their objects are treated by such code, especially if the objects they are using were created by a third party.</p> <p>Therefore, this PEP proposes a uniform API to address dynamic overloading using decorators.</p> </section> <section id="user-api"> <h2><a class="toc-backref" href="#user-api" role="doc-backlink">User API</a></h2> <p>To define a generic function, decorate it with the <code class="docutils literal notranslate"><span class="pre">@singledispatch</span></code> decorator. Note that the dispatch happens on the type of the first argument. Create your function accordingly:</p> <div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="gp">>>> </span><span class="kn">from</span> <span class="nn">functools</span> <span class="kn">import</span> <span class="n">singledispatch</span> <span class="gp">>>> </span><span class="nd">@singledispatch</span> <span class="gp">... </span><span class="k">def</span> <span class="nf">fun</span><span class="p">(</span><span class="n">arg</span><span class="p">,</span> <span class="n">verbose</span><span class="o">=</span><span class="kc">False</span><span class="p">):</span> <span class="gp">... </span> <span class="k">if</span> <span class="n">verbose</span><span class="p">:</span> <span class="gp">... </span> <span class="nb">print</span><span class="p">(</span><span class="s2">"Let me just say,"</span><span class="p">,</span> <span class="n">end</span><span class="o">=</span><span class="s2">" "</span><span class="p">)</span> <span class="gp">... </span> <span class="nb">print</span><span class="p">(</span><span class="n">arg</span><span class="p">)</span> </pre></div> </div> <p>To add overloaded implementations to the function, use the <code class="docutils literal notranslate"><span class="pre">register()</span></code> attribute of the generic function. This is a decorator, taking a type parameter and decorating a function implementing the operation for that type:</p> <div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="gp">>>> </span><span class="nd">@fun</span><span class="o">.</span><span class="n">register</span><span class="p">(</span><span class="nb">int</span><span class="p">)</span> <span class="gp">... </span><span class="k">def</span> <span class="nf">_</span><span class="p">(</span><span class="n">arg</span><span class="p">,</span> <span class="n">verbose</span><span class="o">=</span><span class="kc">False</span><span class="p">):</span> <span class="gp">... </span> <span class="k">if</span> <span class="n">verbose</span><span class="p">:</span> <span class="gp">... </span> <span class="nb">print</span><span class="p">(</span><span class="s2">"Strength in numbers, eh?"</span><span class="p">,</span> <span class="n">end</span><span class="o">=</span><span class="s2">" "</span><span class="p">)</span> <span class="gp">... </span> <span class="nb">print</span><span class="p">(</span><span class="n">arg</span><span class="p">)</span> <span class="gp">...</span> <span class="gp">>>> </span><span class="nd">@fun</span><span class="o">.</span><span class="n">register</span><span class="p">(</span><span class="nb">list</span><span class="p">)</span> <span class="gp">... </span><span class="k">def</span> <span class="nf">_</span><span class="p">(</span><span class="n">arg</span><span class="p">,</span> <span class="n">verbose</span><span class="o">=</span><span class="kc">False</span><span class="p">):</span> <span class="gp">... </span> <span class="k">if</span> <span class="n">verbose</span><span class="p">:</span> <span class="gp">... </span> <span class="nb">print</span><span class="p">(</span><span class="s2">"Enumerate this:"</span><span class="p">)</span> <span class="gp">... </span> <span class="k">for</span> <span class="n">i</span><span class="p">,</span> <span class="n">elem</span> <span class="ow">in</span> <span class="nb">enumerate</span><span class="p">(</span><span class="n">arg</span><span class="p">):</span> <span class="gp">... </span> <span class="nb">print</span><span class="p">(</span><span class="n">i</span><span class="p">,</span> <span class="n">elem</span><span class="p">)</span> </pre></div> </div> <p>To enable registering lambdas and pre-existing functions, the <code class="docutils literal notranslate"><span class="pre">register()</span></code> attribute can be used in a functional form:</p> <div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="gp">>>> </span><span class="k">def</span> <span class="nf">nothing</span><span class="p">(</span><span class="n">arg</span><span class="p">,</span> <span class="n">verbose</span><span class="o">=</span><span class="kc">False</span><span class="p">):</span> <span class="gp">... </span> <span class="nb">print</span><span class="p">(</span><span class="s2">"Nothing."</span><span class="p">)</span> <span class="gp">...</span> <span class="gp">>>> </span><span class="n">fun</span><span class="o">.</span><span class="n">register</span><span class="p">(</span><span class="nb">type</span><span class="p">(</span><span class="kc">None</span><span class="p">),</span> <span class="n">nothing</span><span class="p">)</span> </pre></div> </div> <p>The <code class="docutils literal notranslate"><span class="pre">register()</span></code> attribute returns the undecorated function. This enables decorator stacking, pickling, as well as creating unit tests for each variant independently:</p> <div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="gp">>>> </span><span class="nd">@fun</span><span class="o">.</span><span class="n">register</span><span class="p">(</span><span class="nb">float</span><span class="p">)</span> <span class="gp">... </span><span class="nd">@fun</span><span class="o">.</span><span class="n">register</span><span class="p">(</span><span class="n">Decimal</span><span class="p">)</span> <span class="gp">... </span><span class="k">def</span> <span class="nf">fun_num</span><span class="p">(</span><span class="n">arg</span><span class="p">,</span> <span class="n">verbose</span><span class="o">=</span><span class="kc">False</span><span class="p">):</span> <span class="gp">... </span> <span class="k">if</span> <span class="n">verbose</span><span class="p">:</span> <span class="gp">... </span> <span class="nb">print</span><span class="p">(</span><span class="s2">"Half of your number:"</span><span class="p">,</span> <span class="n">end</span><span class="o">=</span><span class="s2">" "</span><span class="p">)</span> <span class="gp">... </span> <span class="nb">print</span><span class="p">(</span><span class="n">arg</span> <span class="o">/</span> <span class="mi">2</span><span class="p">)</span> <span class="gp">...</span> <span class="gp">>>> </span><span class="n">fun_num</span> <span class="ow">is</span> <span class="n">fun</span> <span class="go">False</span> </pre></div> </div> <p>When called, the generic function dispatches on the type of the first argument:</p> <div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="gp">>>> </span><span class="n">fun</span><span class="p">(</span><span class="s2">"Hello, world."</span><span class="p">)</span> <span class="go">Hello, world.</span> <span class="gp">>>> </span><span class="n">fun</span><span class="p">(</span><span class="s2">"test."</span><span class="p">,</span> <span class="n">verbose</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span> <span class="go">Let me just say, test.</span> <span class="gp">>>> </span><span class="n">fun</span><span class="p">(</span><span class="mi">42</span><span class="p">,</span> <span class="n">verbose</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span> <span class="go">Strength in numbers, eh? 42</span> <span class="gp">>>> </span><span class="n">fun</span><span class="p">([</span><span class="s1">'spam'</span><span class="p">,</span> <span class="s1">'spam'</span><span class="p">,</span> <span class="s1">'eggs'</span><span class="p">,</span> <span class="s1">'spam'</span><span class="p">],</span> <span class="n">verbose</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span> <span class="go">Enumerate this:</span> <span class="go">0 spam</span> <span class="go">1 spam</span> <span class="go">2 eggs</span> <span class="go">3 spam</span> <span class="gp">>>> </span><span class="n">fun</span><span class="p">(</span><span class="kc">None</span><span class="p">)</span> <span class="go">Nothing.</span> <span class="gp">>>> </span><span class="n">fun</span><span class="p">(</span><span class="mf">1.23</span><span class="p">)</span> <span class="go">0.615</span> </pre></div> </div> <p>Where there is no registered implementation for a specific type, its method resolution order is used to find a more generic implementation. The original function decorated with <code class="docutils literal notranslate"><span class="pre">@singledispatch</span></code> is registered for the base <code class="docutils literal notranslate"><span class="pre">object</span></code> type, which means it is used if no better implementation is found.</p> <p>To check which implementation will the generic function choose for a given type, use the <code class="docutils literal notranslate"><span class="pre">dispatch()</span></code> attribute:</p> <div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="gp">>>> </span><span class="n">fun</span><span class="o">.</span><span class="n">dispatch</span><span class="p">(</span><span class="nb">float</span><span class="p">)</span> <span class="go"><function fun_num at 0x104319058></span> <span class="gp">>>> </span><span class="n">fun</span><span class="o">.</span><span class="n">dispatch</span><span class="p">(</span><span class="nb">dict</span><span class="p">)</span> <span class="c1"># note: default implementation</span> <span class="go"><function fun at 0x103fe0000></span> </pre></div> </div> <p>To access all registered implementations, use the read-only <code class="docutils literal notranslate"><span class="pre">registry</span></code> attribute:</p> <div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="gp">>>> </span><span class="n">fun</span><span class="o">.</span><span class="n">registry</span><span class="o">.</span><span class="n">keys</span><span class="p">()</span> <span class="go">dict_keys([<class 'NoneType'>, <class 'int'>, <class 'object'>,</span> <span class="go"> <class 'decimal.Decimal'>, <class 'list'>,</span> <span class="go"> <class 'float'>])</span> <span class="gp">>>> </span><span class="n">fun</span><span class="o">.</span><span class="n">registry</span><span class="p">[</span><span class="nb">float</span><span class="p">]</span> <span class="go"><function fun_num at 0x1035a2840></span> <span class="gp">>>> </span><span class="n">fun</span><span class="o">.</span><span class="n">registry</span><span class="p">[</span><span class="nb">object</span><span class="p">]</span> <span class="go"><function fun at 0x103fe0000></span> </pre></div> </div> <p>The proposed API is intentionally limited and opinionated, as to ensure it is easy to explain and use, as well as to maintain consistency with existing members in the <code class="docutils literal notranslate"><span class="pre">functools</span></code> module.</p> </section> <section id="implementation-notes"> <h2><a class="toc-backref" href="#implementation-notes" role="doc-backlink">Implementation Notes</a></h2> <p>The functionality described in this PEP is already implemented in the <code class="docutils literal notranslate"><span class="pre">pkgutil</span></code> standard library module as <code class="docutils literal notranslate"><span class="pre">simplegeneric</span></code>. Because this implementation is mature, the goal is to move it largely as-is. The reference implementation is available on hg.python.org <a class="footnote-reference brackets" href="#ref-impl" id="id1">[1]</a>.</p> <p>The dispatch type is specified as a decorator argument. An alternative form using function annotations was considered but its inclusion has been rejected. As of May 2013, this usage pattern is out of scope for the standard library <a class="footnote-reference brackets" href="#pep-0008" id="id2">[2]</a>, and the best practices for annotation usage are still debated.</p> <p>Based on the current <code class="docutils literal notranslate"><span class="pre">pkgutil.simplegeneric</span></code> implementation, and following the convention on registering virtual subclasses on Abstract Base Classes, the dispatch registry will not be thread-safe.</p> <section id="abstract-base-classes"> <h3><a class="toc-backref" href="#abstract-base-classes" role="doc-backlink">Abstract Base Classes</a></h3> <p>The <code class="docutils literal notranslate"><span class="pre">pkgutil.simplegeneric</span></code> implementation relied on several forms of method resolution order (MRO). <code class="docutils literal notranslate"><span class="pre">@singledispatch</span></code> removes special handling of old-style classes and Zope’s ExtensionClasses. More importantly, it introduces support for Abstract Base Classes (ABC).</p> <p>When a generic function implementation is registered for an ABC, the dispatch algorithm switches to an extended form of C3 linearization, which includes the relevant ABCs in the MRO of the provided argument. The algorithm inserts ABCs where their functionality is introduced, i.e. <code class="docutils literal notranslate"><span class="pre">issubclass(cls,</span> <span class="pre">abc)</span></code> returns <code class="docutils literal notranslate"><span class="pre">True</span></code> for the class itself but returns <code class="docutils literal notranslate"><span class="pre">False</span></code> for all its direct base classes. Implicit ABCs for a given class (either registered or inferred from the presence of a special method like <code class="docutils literal notranslate"><span class="pre">__len__()</span></code>) are inserted directly after the last ABC explicitly listed in the MRO of said class.</p> <p>In its most basic form, this linearization returns the MRO for the given type:</p> <div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="gp">>>> </span><span class="n">_compose_mro</span><span class="p">(</span><span class="nb">dict</span><span class="p">,</span> <span class="p">[])</span> <span class="go">[<class 'dict'>, <class 'object'>]</span> </pre></div> </div> <p>When the second argument contains ABCs that the specified type is a subclass of, they are inserted in a predictable order:</p> <div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="gp">>>> </span><span class="n">_compose_mro</span><span class="p">(</span><span class="nb">dict</span><span class="p">,</span> <span class="p">[</span><span class="n">Sized</span><span class="p">,</span> <span class="n">MutableMapping</span><span class="p">,</span> <span class="nb">str</span><span class="p">,</span> <span class="gp">... </span> <span class="n">Sequence</span><span class="p">,</span> <span class="n">Iterable</span><span class="p">])</span> <span class="go">[<class 'dict'>, <class 'collections.abc.MutableMapping'>,</span> <span class="go"> <class 'collections.abc.Mapping'>, <class 'collections.abc.Sized'>,</span> <span class="go"> <class 'collections.abc.Iterable'>, <class 'collections.abc.Container'>,</span> <span class="go"> <class 'object'>]</span> </pre></div> </div> <p>While this mode of operation is significantly slower, all dispatch decisions are cached. The cache is invalidated on registering new implementations on the generic function or when user code calls <code class="docutils literal notranslate"><span class="pre">register()</span></code> on an ABC to implicitly subclass it. In the latter case, it is possible to create a situation with ambiguous dispatch, for instance:</p> <div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="gp">>>> </span><span class="kn">from</span> <span class="nn">collections.abc</span> <span class="kn">import</span> <span class="n">Iterable</span><span class="p">,</span> <span class="n">Container</span> <span class="gp">>>> </span><span class="k">class</span> <span class="nc">P</span><span class="p">:</span> <span class="gp">... </span> <span class="k">pass</span> <span class="gp">>>> </span><span class="n">Iterable</span><span class="o">.</span><span class="n">register</span><span class="p">(</span><span class="n">P</span><span class="p">)</span> <span class="go"><class '__main__.P'></span> <span class="gp">>>> </span><span class="n">Container</span><span class="o">.</span><span class="n">register</span><span class="p">(</span><span class="n">P</span><span class="p">)</span> <span class="go"><class '__main__.P'></span> </pre></div> </div> <p>Faced with ambiguity, <code class="docutils literal notranslate"><span class="pre">@singledispatch</span></code> refuses the temptation to guess:</p> <div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="gp">>>> </span><span class="nd">@singledispatch</span> <span class="gp">... </span><span class="k">def</span> <span class="nf">g</span><span class="p">(</span><span class="n">arg</span><span class="p">):</span> <span class="gp">... </span> <span class="k">return</span> <span class="s2">"base"</span> <span class="gp">...</span> <span class="gp">>>> </span><span class="n">g</span><span class="o">.</span><span class="n">register</span><span class="p">(</span><span class="n">Iterable</span><span class="p">,</span> <span class="k">lambda</span> <span class="n">arg</span><span class="p">:</span> <span class="s2">"iterable"</span><span class="p">)</span> <span class="go"><function <lambda> at 0x108b49110></span> <span class="gp">>>> </span><span class="n">g</span><span class="o">.</span><span class="n">register</span><span class="p">(</span><span class="n">Container</span><span class="p">,</span> <span class="k">lambda</span> <span class="n">arg</span><span class="p">:</span> <span class="s2">"container"</span><span class="p">)</span> <span class="go"><function <lambda> at 0x108b491c8></span> <span class="gp">>>> </span><span class="n">g</span><span class="p">(</span><span class="n">P</span><span class="p">())</span> <span class="gt">Traceback (most recent call last):</span> <span class="c">...</span> <span class="gr">RuntimeError</span>: <span class="n">Ambiguous dispatch: <class 'collections.abc.Container'></span> <span class="x">or <class 'collections.abc.Iterable'></span> </pre></div> </div> <p>Note that this exception would not be raised if one or more ABCs had been provided explicitly as base classes during class definition. In this case dispatch happens in the MRO order:</p> <div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="gp">>>> </span><span class="k">class</span> <span class="nc">Ten</span><span class="p">(</span><span class="n">Iterable</span><span class="p">,</span> <span class="n">Container</span><span class="p">):</span> <span class="gp">... </span> <span class="k">def</span> <span class="fm">__iter__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span> <span class="gp">... </span> <span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">10</span><span class="p">):</span> <span class="gp">... </span> <span class="k">yield</span> <span class="n">i</span> <span class="gp">... </span> <span class="k">def</span> <span class="fm">__contains__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">value</span><span class="p">):</span> <span class="gp">... </span> <span class="k">return</span> <span class="n">value</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">10</span><span class="p">)</span> <span class="gp">...</span> <span class="gp">>>> </span><span class="n">g</span><span class="p">(</span><span class="n">Ten</span><span class="p">())</span> <span class="go">'iterable'</span> </pre></div> </div> <p>A similar conflict arises when subclassing an ABC is inferred from the presence of a special method like <code class="docutils literal notranslate"><span class="pre">__len__()</span></code> or <code class="docutils literal notranslate"><span class="pre">__contains__()</span></code>:</p> <div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="gp">>>> </span><span class="k">class</span> <span class="nc">Q</span><span class="p">:</span> <span class="gp">... </span> <span class="k">def</span> <span class="fm">__contains__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">value</span><span class="p">):</span> <span class="gp">... </span> <span class="k">return</span> <span class="kc">False</span> <span class="gp">...</span> <span class="gp">>>> </span><span class="nb">issubclass</span><span class="p">(</span><span class="n">Q</span><span class="p">,</span> <span class="n">Container</span><span class="p">)</span> <span class="go">True</span> <span class="gp">>>> </span><span class="n">Iterable</span><span class="o">.</span><span class="n">register</span><span class="p">(</span><span class="n">Q</span><span class="p">)</span> <span class="gp">>>> </span><span class="n">g</span><span class="p">(</span><span class="n">Q</span><span class="p">())</span> <span class="gt">Traceback (most recent call last):</span> <span class="c">...</span> <span class="gr">RuntimeError</span>: <span class="n">Ambiguous dispatch: <class 'collections.abc.Container'></span> <span class="x">or <class 'collections.abc.Iterable'></span> </pre></div> </div> <p>An early version of the PEP contained a custom approach that was simpler but created a number of edge cases with surprising results <a class="footnote-reference brackets" href="#why-c3" id="id3">[3]</a>.</p> </section> </section> <section id="usage-patterns"> <h2><a class="toc-backref" href="#usage-patterns" role="doc-backlink">Usage Patterns</a></h2> <p>This PEP proposes extending behaviour only of functions specifically marked as generic. Just as a base class method may be overridden by a subclass, so too a function may be overloaded to provide custom functionality for a given type.</p> <p>Universal overloading does not equal <em>arbitrary</em> overloading, in the sense that we need not expect people to randomly redefine the behavior of existing functions in unpredictable ways. To the contrary, generic function usage in actual programs tends to follow very predictable patterns and registered implementations are highly-discoverable in the common case.</p> <p>If a module is defining a new generic operation, it will usually also define any required implementations for existing types in the same place. Likewise, if a module is defining a new type, then it will usually define implementations there for any generic functions that it knows or cares about. As a result, the vast majority of registered implementations can be found adjacent to either the function being overloaded, or to a newly-defined type for which the implementation is adding support.</p> <p>It is only in rather infrequent cases that one will have implementations registered in a module that contains neither the function nor the type(s) for which the implementation is added. In the absence of incompetence or deliberate intention to be obscure, the few implementations that are not registered adjacent to the relevant type(s) or function(s), will generally not need to be understood or known about outside the scope where those implementations are defined. (Except in the “support modules” case, where best practice suggests naming them accordingly.)</p> <p>As mentioned earlier, single-dispatch generics are already prolific throughout the standard library. A clean, standard way of doing them provides a way forward to refactor those custom implementations to use a common one, opening them up for user extensibility at the same time.</p> </section> <section id="alternative-approaches"> <h2><a class="toc-backref" href="#alternative-approaches" role="doc-backlink">Alternative approaches</a></h2> <p>In <a class="pep reference internal" href="../pep-3124/" title="PEP 3124 – Overloading, Generic Functions, Interfaces, and Adaptation">PEP 3124</a> Phillip J. Eby proposes a full-grown solution with overloading based on arbitrary rule sets (with the default implementation dispatching on argument types), as well as interfaces, adaptation and method combining. PEAK-Rules <a class="footnote-reference brackets" href="#peak-rules" id="id4">[4]</a> is a reference implementation of the concepts described in PJE’s PEP.</p> <p>Such a broad approach is inherently complex, which makes reaching a consensus hard. In contrast, this PEP focuses on a single piece of functionality that is simple to reason about. It’s important to note this does not preclude the use of other approaches now or in the future.</p> <p>In a 2005 article on Artima <a class="footnote-reference brackets" href="#artima2005" id="id5">[5]</a> Guido van Rossum presents a generic function implementation that dispatches on types of all arguments on a function. The same approach was chosen in Andrey Popp’s <code class="docutils literal notranslate"><span class="pre">generic</span></code> package available on PyPI <a class="footnote-reference brackets" href="#pypi-generic" id="id6">[6]</a>, as well as David Mertz’s <code class="docutils literal notranslate"><span class="pre">gnosis.magic.multimethods</span></code> <a class="footnote-reference brackets" href="#gnosis-multimethods" id="id7">[7]</a>.</p> <p>While this seems desirable at first, I agree with Fredrik Lundh’s comment that “if you design APIs with pages of logic just to sort out what code a function should execute, you should probably hand over the API design to someone else”. In other words, the single argument approach proposed in this PEP is not only easier to implement but also clearly communicates that dispatching on a more complex state is an anti-pattern. It also has the virtue of corresponding directly with the familiar method dispatch mechanism in object oriented programming. The only difference is whether the custom implementation is associated more closely with the data (object-oriented methods) or the algorithm (single-dispatch overloading).</p> <p>PyPy’s RPython offers <code class="docutils literal notranslate"><span class="pre">extendabletype</span></code> <a class="footnote-reference brackets" href="#pairtype" id="id8">[8]</a>, a metaclass which enables classes to be externally extended. In combination with <code class="docutils literal notranslate"><span class="pre">pairtype()</span></code> and <code class="docutils literal notranslate"><span class="pre">pair()</span></code> factories, this offers a form of single-dispatch generics.</p> </section> <section id="acknowledgements"> <h2><a class="toc-backref" href="#acknowledgements" role="doc-backlink">Acknowledgements</a></h2> <p>Apart from Phillip J. Eby’s work on <a class="pep reference internal" href="../pep-3124/" title="PEP 3124 – Overloading, Generic Functions, Interfaces, and Adaptation">PEP 3124</a> and PEAK-Rules, influences include Paul Moore’s original issue <a class="footnote-reference brackets" href="#issue-5135" id="id9">[9]</a> that proposed exposing <code class="docutils literal notranslate"><span class="pre">pkgutil.simplegeneric</span></code> as part of the <code class="docutils literal notranslate"><span class="pre">functools</span></code> API, Guido van Rossum’s article on multimethods <a class="footnote-reference brackets" href="#artima2005" id="id10">[5]</a>, and discussions with Raymond Hettinger on a general pprint rewrite. Huge thanks to Alyssa Coghlan for encouraging me to create this PEP and providing initial feedback.</p> </section> <section id="references"> <h2><a class="toc-backref" href="#references" role="doc-backlink">References</a></h2> <aside class="footnote-list brackets"> <aside class="footnote brackets" id="ref-impl" role="doc-footnote"> <dt class="label" id="ref-impl">[<a href="#id1">1</a>]</dt> <dd><a class="reference external" href="http://hg.python.org/features/pep-443/file/tip/Lib/functools.py#l359">http://hg.python.org/features/pep-443/file/tip/Lib/functools.py#l359</a></aside> <aside class="footnote brackets" id="pep-0008" role="doc-footnote"> <dt class="label" id="pep-0008">[<a href="#id2">2</a>]</dt> <dd><a class="pep reference internal" href="../pep-0008/" title="PEP 8 – Style Guide for Python Code">PEP 8</a> states in the “Programming Recommendations” section that “the Python standard library will not use function annotations as that would result in a premature commitment to a particular annotation style”.</aside> <aside class="footnote brackets" id="why-c3" role="doc-footnote"> <dt class="label" id="why-c3">[<a href="#id3">3</a>]</dt> <dd><a class="reference external" href="http://bugs.python.org/issue18244">http://bugs.python.org/issue18244</a></aside> <aside class="footnote brackets" id="peak-rules" role="doc-footnote"> <dt class="label" id="peak-rules">[<a href="#id4">4</a>]</dt> <dd><a class="reference external" href="http://peak.telecommunity.com/DevCenter/PEAK_2dRules">http://peak.telecommunity.com/DevCenter/PEAK_2dRules</a></aside> <aside class="footnote brackets" id="artima2005" role="doc-footnote"> <dt class="label" id="artima2005">[5]<em> (<a href='#id5'>1</a>, <a href='#id10'>2</a>) </em></dt> <dd><a class="reference external" href="http://www.artima.com/weblogs/viewpost.jsp?thread=101605">http://www.artima.com/weblogs/viewpost.jsp?thread=101605</a></aside> <aside class="footnote brackets" id="pypi-generic" role="doc-footnote"> <dt class="label" id="pypi-generic">[<a href="#id6">6</a>]</dt> <dd><a class="reference external" href="http://pypi.python.org/pypi/generic">http://pypi.python.org/pypi/generic</a></aside> <aside class="footnote brackets" id="gnosis-multimethods" role="doc-footnote"> <dt class="label" id="gnosis-multimethods">[<a href="#id7">7</a>]</dt> <dd><a class="reference external" href="http://gnosis.cx/publish/programming/charming_python_b12.html">http://gnosis.cx/publish/programming/charming_python_b12.html</a></aside> <aside class="footnote brackets" id="pairtype" role="doc-footnote"> <dt class="label" id="pairtype">[<a href="#id8">8</a>]</dt> <dd><a class="reference external" href="https://bitbucket.org/pypy/pypy/raw/default/rpython/tool/pairtype.py">https://bitbucket.org/pypy/pypy/raw/default/rpython/tool/pairtype.py</a></aside> <aside class="footnote brackets" id="issue-5135" role="doc-footnote"> <dt class="label" id="issue-5135">[<a href="#id9">9</a>]</dt> <dd><a class="reference external" href="http://bugs.python.org/issue5135">http://bugs.python.org/issue5135</a></aside> </aside> </section> <section id="copyright"> <h2><a class="toc-backref" href="#copyright" role="doc-backlink">Copyright</a></h2> <p>This document has been placed in the public domain.</p> </section> </section> <hr class="docutils" /> <p>Source: <a class="reference external" href="https://github.com/python/peps/blob/main/peps/pep-0443.rst">https://github.com/python/peps/blob/main/peps/pep-0443.rst</a></p> <p>Last modified: <a class="reference external" href="https://github.com/python/peps/commits/main/peps/pep-0443.rst">2023-10-11 12:05:51 GMT</a></p> </article> <nav id="pep-sidebar"> <h2>Contents</h2> <ul> <li><a class="reference internal" href="#abstract">Abstract</a></li> <li><a class="reference internal" href="#rationale-and-goals">Rationale and Goals</a></li> <li><a class="reference internal" href="#user-api">User API</a></li> <li><a class="reference internal" href="#implementation-notes">Implementation Notes</a><ul> <li><a class="reference internal" href="#abstract-base-classes">Abstract Base Classes</a></li> </ul> </li> <li><a class="reference internal" href="#usage-patterns">Usage Patterns</a></li> <li><a class="reference internal" href="#alternative-approaches">Alternative approaches</a></li> <li><a class="reference internal" href="#acknowledgements">Acknowledgements</a></li> <li><a class="reference internal" href="#references">References</a></li> <li><a class="reference internal" href="#copyright">Copyright</a></li> </ul> <br> <a id="source" href="https://github.com/python/peps/blob/main/peps/pep-0443.rst">Page Source (GitHub)</a> </nav> </section> <script src="../_static/colour_scheme.js"></script> <script src="../_static/wrap_tables.js"></script> <script src="../_static/sticky_banner.js"></script> </body> </html>