CINXE.COM
PEP 560 – Core support for typing module and generic types | peps.python.org
<!DOCTYPE html> <html lang="en"> <head> <meta charset="utf-8"> <meta name="viewport" content="width=device-width, initial-scale=1.0"> <meta name="color-scheme" content="light dark"> <title>PEP 560 – Core support for typing module and generic types | peps.python.org</title> <link rel="shortcut icon" href="../_static/py.png"> <link rel="canonical" href="https://peps.python.org/pep-0560/"> <link rel="stylesheet" href="../_static/style.css" type="text/css"> <link rel="stylesheet" href="../_static/mq.css" type="text/css"> <link rel="stylesheet" href="../_static/pygments.css" type="text/css" media="(prefers-color-scheme: light)" id="pyg-light"> <link rel="stylesheet" href="../_static/pygments_dark.css" type="text/css" media="(prefers-color-scheme: dark)" id="pyg-dark"> <link rel="alternate" type="application/rss+xml" title="Latest PEPs" href="https://peps.python.org/peps.rss"> <meta property="og:title" content='PEP 560 – Core support for typing module and generic types | peps.python.org'> <meta property="og:description" content="Initially PEP 484 was designed in such way that it would not introduce any changes to the core CPython interpreter. Now type hints and the typing module are extensively used by the community, e.g. PEP 526 and PEP 557 extend the usage of type hints, and ..."> <meta property="og:type" content="website"> <meta property="og:url" content="https://peps.python.org/pep-0560/"> <meta property="og:site_name" content="Python Enhancement Proposals (PEPs)"> <meta property="og:image" content="https://peps.python.org/_static/og-image.png"> <meta property="og:image:alt" content="Python PEPs"> <meta property="og:image:width" content="200"> <meta property="og:image:height" content="200"> <meta name="description" content="Initially PEP 484 was designed in such way that it would not introduce any changes to the core CPython interpreter. Now type hints and the typing module are extensively used by the community, e.g. PEP 526 and PEP 557 extend the usage of type hints, and ..."> <meta name="theme-color" content="#3776ab"> </head> <body> <svg xmlns="http://www.w3.org/2000/svg" style="display: none;"> <symbol id="svg-sun-half" viewBox="0 0 24 24" pointer-events="all"> <title>Following system colour scheme</title> <svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"> <circle cx="12" cy="12" r="9"></circle> <path d="M12 3v18m0-12l4.65-4.65M12 14.3l7.37-7.37M12 19.6l8.85-8.85"></path> </svg> </symbol> <symbol id="svg-moon" viewBox="0 0 24 24" pointer-events="all"> <title>Selected dark colour scheme</title> <svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"> <path stroke="none" d="M0 0h24v24H0z" fill="none"></path> <path d="M12 3c.132 0 .263 0 .393 0a7.5 7.5 0 0 0 7.92 12.446a9 9 0 1 1 -8.313 -12.454z"></path> </svg> </symbol> <symbol id="svg-sun" viewBox="0 0 24 24" pointer-events="all"> <title>Selected light colour scheme</title> <svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"> <circle cx="12" cy="12" r="5"></circle> <line x1="12" y1="1" x2="12" y2="3"></line> <line x1="12" y1="21" x2="12" y2="23"></line> <line x1="4.22" y1="4.22" x2="5.64" y2="5.64"></line> <line x1="18.36" y1="18.36" x2="19.78" y2="19.78"></line> <line x1="1" y1="12" x2="3" y2="12"></line> <line x1="21" y1="12" x2="23" y2="12"></line> <line x1="4.22" y1="19.78" x2="5.64" y2="18.36"></line> <line x1="18.36" y1="5.64" x2="19.78" y2="4.22"></line> </svg> </symbol> </svg> <script> document.documentElement.dataset.colour_scheme = localStorage.getItem("colour_scheme") || "auto" </script> <section id="pep-page-section"> <header> <h1>Python Enhancement Proposals</h1> <ul class="breadcrumbs"> <li><a href="https://www.python.org/" title="The Python Programming Language">Python</a> » </li> <li><a href="../pep-0000/">PEP Index</a> » </li> <li>PEP 560</li> </ul> <button id="colour-scheme-cycler" onClick="setColourScheme(nextColourScheme())"> <svg aria-hidden="true" class="colour-scheme-icon-when-auto"><use href="#svg-sun-half"></use></svg> <svg aria-hidden="true" class="colour-scheme-icon-when-dark"><use href="#svg-moon"></use></svg> <svg aria-hidden="true" class="colour-scheme-icon-when-light"><use href="#svg-sun"></use></svg> <span class="visually-hidden">Toggle light / dark / auto colour theme</span> </button> </header> <article> <section id="pep-content"> <h1 class="page-title">PEP 560 – Core support for typing module and generic types</h1> <dl class="rfc2822 field-list simple"> <dt class="field-odd">Author<span class="colon">:</span></dt> <dd class="field-odd">Ivan Levkivskyi <levkivskyi at gmail.com></dd> <dt class="field-even">Status<span class="colon">:</span></dt> <dd class="field-even"><abbr title="Accepted and implementation complete, or no longer active">Final</abbr></dd> <dt class="field-odd">Type<span class="colon">:</span></dt> <dd class="field-odd"><abbr title="Normative PEP with a new feature for Python, implementation change for CPython or interoperability standard for the ecosystem">Standards Track</abbr></dd> <dt class="field-even">Topic<span class="colon">:</span></dt> <dd class="field-even"><a class="reference external" href="../topic/typing/">Typing</a></dd> <dt class="field-odd">Created<span class="colon">:</span></dt> <dd class="field-odd">03-Sep-2017</dd> <dt class="field-even">Python-Version<span class="colon">:</span></dt> <dd class="field-even">3.7</dd> <dt class="field-odd">Post-History<span class="colon">:</span></dt> <dd class="field-odd">09-Sep-2017, 14-Nov-2017</dd> <dt class="field-even">Resolution<span class="colon">:</span></dt> <dd class="field-even"><a class="reference external" href="https://mail.python.org/pipermail/python-dev/2017-December/151038.html">Python-Dev message</a></dd> </dl> <hr class="docutils" /> <section id="contents"> <details><summary>Table of Contents</summary><ul class="simple"> <li><a class="reference internal" href="#abstract">Abstract</a></li> <li><a class="reference internal" href="#rationale">Rationale</a><ul> <li><a class="reference internal" href="#performance">Performance</a></li> <li><a class="reference internal" href="#metaclass-conflicts">Metaclass conflicts</a></li> <li><a class="reference internal" href="#hacks-and-bugs-that-will-be-removed-by-this-proposal">Hacks and bugs that will be removed by this proposal</a></li> </ul> </li> <li><a class="reference internal" href="#specification">Specification</a><ul> <li><a class="reference internal" href="#class-getitem"><code class="docutils literal notranslate"><span class="pre">__class_getitem__</span></code></a></li> <li><a class="reference internal" href="#mro-entries"><code class="docutils literal notranslate"><span class="pre">__mro_entries__</span></code></a></li> <li><a class="reference internal" href="#dynamic-class-creation-and-types-resolve-bases">Dynamic class creation and <code class="docutils literal notranslate"><span class="pre">types.resolve_bases</span></code></a></li> <li><a class="reference internal" href="#using-class-getitem-in-c-extensions">Using <code class="docutils literal notranslate"><span class="pre">__class_getitem__</span></code> in C extensions</a></li> </ul> </li> <li><a class="reference internal" href="#backwards-compatibility-and-impact-on-users-who-don-t-use-typing">Backwards compatibility and impact on users who don’t use <code class="docutils literal notranslate"><span class="pre">typing</span></code></a></li> <li><a class="reference internal" href="#references">References</a></li> <li><a class="reference internal" href="#copyright">Copyright</a></li> </ul> </details></section> <div class="pep-banner canonical-doc sticky-banner admonition important"> <p class="admonition-title">Important</p> <p>This PEP is a historical document. The up-to-date, canonical documentation can now be found at the documentation for <a class="reference external" href="https://docs.python.org/3/reference/datamodel.html#object.__class_getitem__" title="(in Python v3.13)"><code class="xref py py-meth docutils literal notranslate"><span class="pre">__class_getitem__()</span></code></a> and <a class="reference external" href="https://docs.python.org/3/reference/datamodel.html#object.__mro_entries__" title="(in Python v3.13)"><code class="xref py py-meth docutils literal notranslate"><span class="pre">__mro_entries__()</span></code></a>.</p> <p class="close-button">×</p> <p>See <a class="pep reference internal" href="../pep-0001/" title="PEP 1 – PEP Purpose and Guidelines">PEP 1</a> for how to propose changes.</p> </div> <section id="abstract"> <h2><a class="toc-backref" href="#abstract" role="doc-backlink">Abstract</a></h2> <p>Initially <a class="pep reference internal" href="../pep-0484/" title="PEP 484 – Type Hints">PEP 484</a> was designed in such way that it would not introduce <em>any</em> changes to the core CPython interpreter. Now type hints and the <code class="docutils literal notranslate"><span class="pre">typing</span></code> module are extensively used by the community, e.g. <a class="pep reference internal" href="../pep-0526/" title="PEP 526 – Syntax for Variable Annotations">PEP 526</a> and <a class="pep reference internal" href="../pep-0557/" title="PEP 557 – Data Classes">PEP 557</a> extend the usage of type hints, and the backport of <code class="docutils literal notranslate"><span class="pre">typing</span></code> on PyPI has 1M downloads/month. Therefore, this restriction can be removed. It is proposed to add two special methods <code class="docutils literal notranslate"><span class="pre">__class_getitem__</span></code> and <code class="docutils literal notranslate"><span class="pre">__mro_entries__</span></code> to the core CPython for better support of generic types.</p> </section> <section id="rationale"> <h2><a class="toc-backref" href="#rationale" role="doc-backlink">Rationale</a></h2> <p>The restriction to not modify the core CPython interpreter led to some design decisions that became questionable when the <code class="docutils literal notranslate"><span class="pre">typing</span></code> module started to be widely used. There are three main points of concern: performance of the <code class="docutils literal notranslate"><span class="pre">typing</span></code> module, metaclass conflicts, and the large number of hacks currently used in <code class="docutils literal notranslate"><span class="pre">typing</span></code>.</p> <section id="performance"> <h3><a class="toc-backref" href="#performance" role="doc-backlink">Performance</a></h3> <p>The <code class="docutils literal notranslate"><span class="pre">typing</span></code> module is one of the heaviest and slowest modules in the standard library even with all the optimizations made. Mainly this is because subscripted generic types (see <a class="pep reference internal" href="../pep-0484/" title="PEP 484 – Type Hints">PEP 484</a> for definition of terms used in this PEP) are class objects (see also <a class="footnote-reference brackets" href="#id7" id="id1">[1]</a>). There are three main ways how the performance can be improved with the help of the proposed special methods:</p> <ul class="simple"> <li>Creation of generic classes is slow since the <code class="docutils literal notranslate"><span class="pre">GenericMeta.__new__</span></code> is very slow; we will not need it anymore.</li> <li>Very long method resolution orders (MROs) for generic classes will be half as long; they are present because we duplicate the <code class="docutils literal notranslate"><span class="pre">collections.abc</span></code> inheritance chain in <code class="docutils literal notranslate"><span class="pre">typing</span></code>.</li> <li>Instantiation of generic classes will be faster (this is minor however).</li> </ul> </section> <section id="metaclass-conflicts"> <h3><a class="toc-backref" href="#metaclass-conflicts" role="doc-backlink">Metaclass conflicts</a></h3> <p>All generic types are instances of <code class="docutils literal notranslate"><span class="pre">GenericMeta</span></code>, so if a user uses a custom metaclass, then it is hard to make a corresponding class generic. This is particularly hard for library classes that a user doesn’t control. A workaround is to always mix-in <code class="docutils literal notranslate"><span class="pre">GenericMeta</span></code>:</p> <div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="k">class</span><span class="w"> </span><span class="nc">AdHocMeta</span><span class="p">(</span><span class="n">GenericMeta</span><span class="p">,</span> <span class="n">LibraryMeta</span><span class="p">):</span> <span class="k">pass</span> <span class="k">class</span><span class="w"> </span><span class="nc">UserClass</span><span class="p">(</span><span class="n">LibraryBase</span><span class="p">,</span> <span class="n">Generic</span><span class="p">[</span><span class="n">T</span><span class="p">],</span> <span class="n">metaclass</span><span class="o">=</span><span class="n">AdHocMeta</span><span class="p">):</span> <span class="o">...</span> </pre></div> </div> <p>but this is not always practical or even possible. With the help of the proposed special attributes the <code class="docutils literal notranslate"><span class="pre">GenericMeta</span></code> metaclass will not be needed.</p> </section> <section id="hacks-and-bugs-that-will-be-removed-by-this-proposal"> <h3><a class="toc-backref" href="#hacks-and-bugs-that-will-be-removed-by-this-proposal" role="doc-backlink">Hacks and bugs that will be removed by this proposal</a></h3> <ul class="simple"> <li><code class="docutils literal notranslate"><span class="pre">_generic_new</span></code> hack that exists because <code class="docutils literal notranslate"><span class="pre">__init__</span></code> is not called on instances with a type differing from the type whose <code class="docutils literal notranslate"><span class="pre">__new__</span></code> was called, <code class="docutils literal notranslate"><span class="pre">C[int]().__class__</span> <span class="pre">is</span> <span class="pre">C</span></code>.</li> <li><code class="docutils literal notranslate"><span class="pre">_next_in_mro</span></code> speed hack will be not necessary since subscription will not create new classes.</li> <li>Ugly <code class="docutils literal notranslate"><span class="pre">sys._getframe</span></code> hack. This one is particularly nasty since it looks like we can’t remove it without changes outside <code class="docutils literal notranslate"><span class="pre">typing</span></code>.</li> <li>Currently generics do dangerous things with private ABC caches to fix large memory consumption that grows at least as O(N<sup>2</sup>), see <a class="footnote-reference brackets" href="#id8" id="id2">[2]</a>. This point is also important because it was recently proposed to re-implement <code class="docutils literal notranslate"><span class="pre">ABCMeta</span></code> in C.</li> <li>Problems with sharing attributes between subscripted generics, see <a class="footnote-reference brackets" href="#id9" id="id3">[3]</a>. The current solution already uses <code class="docutils literal notranslate"><span class="pre">__getattr__</span></code> and <code class="docutils literal notranslate"><span class="pre">__setattr__</span></code>, but it is still incomplete, and solving this without the current proposal will be hard and will need <code class="docutils literal notranslate"><span class="pre">__getattribute__</span></code>.</li> <li><code class="docutils literal notranslate"><span class="pre">_no_slots_copy</span></code> hack, where we clean up the class dictionary on every subscription thus allowing generics with <code class="docutils literal notranslate"><span class="pre">__slots__</span></code>.</li> <li>General complexity of the <code class="docutils literal notranslate"><span class="pre">typing</span></code> module. The new proposal will not only allow to remove the above-mentioned hacks/bugs, but also simplify the implementation, so that it will be easier to maintain.</li> </ul> </section> </section> <section id="specification"> <h2><a class="toc-backref" href="#specification" role="doc-backlink">Specification</a></h2> <section id="class-getitem"> <h3><a class="toc-backref" href="#class-getitem" role="doc-backlink"><code class="docutils literal notranslate"><span class="pre">__class_getitem__</span></code></a></h3> <p>The idea of <code class="docutils literal notranslate"><span class="pre">__class_getitem__</span></code> is simple: it is an exact analog of <code class="docutils literal notranslate"><span class="pre">__getitem__</span></code> with an exception that it is called on a class that defines it, not on its instances. This allows us to avoid <code class="docutils literal notranslate"><span class="pre">GenericMeta.__getitem__</span></code> for things like <code class="docutils literal notranslate"><span class="pre">Iterable[int]</span></code>. The <code class="docutils literal notranslate"><span class="pre">__class_getitem__</span></code> is automatically a class method and does not require <code class="docutils literal notranslate"><span class="pre">@classmethod</span></code> decorator (similar to <code class="docutils literal notranslate"><span class="pre">__init_subclass__</span></code>) and is inherited like normal attributes. For example:</p> <div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="k">class</span><span class="w"> </span><span class="nc">MyList</span><span class="p">:</span> <span class="k">def</span><span class="w"> </span><span class="fm">__getitem__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">index</span><span class="p">):</span> <span class="k">return</span> <span class="n">index</span> <span class="o">+</span> <span class="mi">1</span> <span class="k">def</span><span class="w"> </span><span class="nf">__class_getitem__</span><span class="p">(</span><span class="bp">cls</span><span class="p">,</span> <span class="n">item</span><span class="p">):</span> <span class="k">return</span> <span class="sa">f</span><span class="s2">"</span><span class="si">{</span><span class="bp">cls</span><span class="o">.</span><span class="vm">__name__</span><span class="si">}</span><span class="s2">[</span><span class="si">{</span><span class="n">item</span><span class="o">.</span><span class="vm">__name__</span><span class="si">}</span><span class="s2">]"</span> <span class="k">class</span><span class="w"> </span><span class="nc">MyOtherList</span><span class="p">(</span><span class="n">MyList</span><span class="p">):</span> <span class="k">pass</span> <span class="k">assert</span> <span class="n">MyList</span><span class="p">()[</span><span class="mi">0</span><span class="p">]</span> <span class="o">==</span> <span class="mi">1</span> <span class="k">assert</span> <span class="n">MyList</span><span class="p">[</span><span class="nb">int</span><span class="p">]</span> <span class="o">==</span> <span class="s2">"MyList[int]"</span> <span class="k">assert</span> <span class="n">MyOtherList</span><span class="p">()[</span><span class="mi">0</span><span class="p">]</span> <span class="o">==</span> <span class="mi">1</span> <span class="k">assert</span> <span class="n">MyOtherList</span><span class="p">[</span><span class="nb">int</span><span class="p">]</span> <span class="o">==</span> <span class="s2">"MyOtherList[int]"</span> </pre></div> </div> <p>Note that this method is used as a fallback, so if a metaclass defines <code class="docutils literal notranslate"><span class="pre">__getitem__</span></code>, then that will have the priority.</p> </section> <section id="mro-entries"> <h3><a class="toc-backref" href="#mro-entries" role="doc-backlink"><code class="docutils literal notranslate"><span class="pre">__mro_entries__</span></code></a></h3> <p>If an object that is not a class object appears in the tuple of bases of a class definition, then method <code class="docutils literal notranslate"><span class="pre">__mro_entries__</span></code> is searched on it. If found, it is called with the original tuple of bases as an argument. The result of the call must be a tuple, that is unpacked in the base classes in place of this object. (If the tuple is empty, this means that the original bases is simply discarded.) If there are more than one object with <code class="docutils literal notranslate"><span class="pre">__mro_entries__</span></code>, then all of them are called with the same original tuple of bases. This step happens first in the process of creation of a class, all other steps, including checks for duplicate bases and MRO calculation, happen normally with the updated bases.</p> <p>Using the method API instead of just an attribute is necessary to avoid inconsistent MRO errors, and perform other manipulations that are currently done by <code class="docutils literal notranslate"><span class="pre">GenericMeta.__new__</span></code>. The original bases are stored as <code class="docutils literal notranslate"><span class="pre">__orig_bases__</span></code> in the class namespace (currently this is also done by the metaclass). For example:</p> <div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="k">class</span><span class="w"> </span><span class="nc">GenericAlias</span><span class="p">:</span> <span class="k">def</span><span class="w"> </span><span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">origin</span><span class="p">,</span> <span class="n">item</span><span class="p">):</span> <span class="bp">self</span><span class="o">.</span><span class="n">origin</span> <span class="o">=</span> <span class="n">origin</span> <span class="bp">self</span><span class="o">.</span><span class="n">item</span> <span class="o">=</span> <span class="n">item</span> <span class="k">def</span><span class="w"> </span><span class="nf">__mro_entries__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">bases</span><span class="p">):</span> <span class="k">return</span> <span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">origin</span><span class="p">,)</span> <span class="k">class</span><span class="w"> </span><span class="nc">NewList</span><span class="p">:</span> <span class="k">def</span><span class="w"> </span><span class="nf">__class_getitem__</span><span class="p">(</span><span class="bp">cls</span><span class="p">,</span> <span class="n">item</span><span class="p">):</span> <span class="k">return</span> <span class="n">GenericAlias</span><span class="p">(</span><span class="bp">cls</span><span class="p">,</span> <span class="n">item</span><span class="p">)</span> <span class="k">class</span><span class="w"> </span><span class="nc">Tokens</span><span class="p">(</span><span class="n">NewList</span><span class="p">[</span><span class="nb">int</span><span class="p">]):</span> <span class="o">...</span> <span class="k">assert</span> <span class="n">Tokens</span><span class="o">.</span><span class="vm">__bases__</span> <span class="o">==</span> <span class="p">(</span><span class="n">NewList</span><span class="p">,)</span> <span class="k">assert</span> <span class="n">Tokens</span><span class="o">.</span><span class="n">__orig_bases__</span> <span class="o">==</span> <span class="p">(</span><span class="n">NewList</span><span class="p">[</span><span class="nb">int</span><span class="p">],)</span> <span class="k">assert</span> <span class="n">Tokens</span><span class="o">.</span><span class="vm">__mro__</span> <span class="o">==</span> <span class="p">(</span><span class="n">Tokens</span><span class="p">,</span> <span class="n">NewList</span><span class="p">,</span> <span class="nb">object</span><span class="p">)</span> </pre></div> </div> <p>Resolution using <code class="docutils literal notranslate"><span class="pre">__mro_entries__</span></code> happens <em>only</em> in bases of a class definition statement. In all other situations where a class object is expected, no such resolution will happen, this includes <code class="docutils literal notranslate"><span class="pre">isinstance</span></code> and <code class="docutils literal notranslate"><span class="pre">issubclass</span></code> built-in functions.</p> <p>NOTE: These two method names are reserved for use by the <code class="docutils literal notranslate"><span class="pre">typing</span></code> module and the generic types machinery, and any other use is discouraged. The reference implementation (with tests) can be found in <a class="footnote-reference brackets" href="#id10" id="id4">[4]</a>, and the proposal was originally posted and discussed on the <code class="docutils literal notranslate"><span class="pre">typing</span></code> tracker, see <a class="footnote-reference brackets" href="#id11" id="id5">[5]</a>.</p> </section> <section id="dynamic-class-creation-and-types-resolve-bases"> <h3><a class="toc-backref" href="#dynamic-class-creation-and-types-resolve-bases" role="doc-backlink">Dynamic class creation and <code class="docutils literal notranslate"><span class="pre">types.resolve_bases</span></code></a></h3> <p><code class="docutils literal notranslate"><span class="pre">type.__new__</span></code> will not perform any MRO entry resolution. So that a direct call <code class="docutils literal notranslate"><span class="pre">type('Tokens',</span> <span class="pre">(List[int],),</span> <span class="pre">{})</span></code> will fail. This is done for performance reasons and to minimize the number of implicit transformations. Instead, a helper function <code class="docutils literal notranslate"><span class="pre">resolve_bases</span></code> will be added to the <code class="docutils literal notranslate"><span class="pre">types</span></code> module to allow an explicit <code class="docutils literal notranslate"><span class="pre">__mro_entries__</span></code> resolution in the context of dynamic class creation. Correspondingly, <code class="docutils literal notranslate"><span class="pre">types.new_class</span></code> will be updated to reflect the new class creation steps while maintaining the backwards compatibility:</p> <div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="k">def</span><span class="w"> </span><span class="nf">new_class</span><span class="p">(</span><span class="n">name</span><span class="p">,</span> <span class="n">bases</span><span class="o">=</span><span class="p">(),</span> <span class="n">kwds</span><span class="o">=</span><span class="kc">None</span><span class="p">,</span> <span class="n">exec_body</span><span class="o">=</span><span class="kc">None</span><span class="p">):</span> <span class="n">resolved_bases</span> <span class="o">=</span> <span class="n">resolve_bases</span><span class="p">(</span><span class="n">bases</span><span class="p">)</span> <span class="c1"># This step is added</span> <span class="n">meta</span><span class="p">,</span> <span class="n">ns</span><span class="p">,</span> <span class="n">kwds</span> <span class="o">=</span> <span class="n">prepare_class</span><span class="p">(</span><span class="n">name</span><span class="p">,</span> <span class="n">resolved_bases</span><span class="p">,</span> <span class="n">kwds</span><span class="p">)</span> <span class="k">if</span> <span class="n">exec_body</span> <span class="ow">is</span> <span class="ow">not</span> <span class="kc">None</span><span class="p">:</span> <span class="n">exec_body</span><span class="p">(</span><span class="n">ns</span><span class="p">)</span> <span class="n">ns</span><span class="p">[</span><span class="s1">'__orig_bases__'</span><span class="p">]</span> <span class="o">=</span> <span class="n">bases</span> <span class="c1"># This step is added</span> <span class="k">return</span> <span class="n">meta</span><span class="p">(</span><span class="n">name</span><span class="p">,</span> <span class="n">resolved_bases</span><span class="p">,</span> <span class="n">ns</span><span class="p">,</span> <span class="o">**</span><span class="n">kwds</span><span class="p">)</span> </pre></div> </div> </section> <section id="using-class-getitem-in-c-extensions"> <h3><a class="toc-backref" href="#using-class-getitem-in-c-extensions" role="doc-backlink">Using <code class="docutils literal notranslate"><span class="pre">__class_getitem__</span></code> in C extensions</a></h3> <p>As mentioned above, <code class="docutils literal notranslate"><span class="pre">__class_getitem__</span></code> is automatically a class method if defined in Python code. To define this method in a C extension, one should use flags <code class="docutils literal notranslate"><span class="pre">METH_O|METH_CLASS</span></code>. For example, a simple way to make an extension class generic is to use a method that simply returns the original class objects, thus fully erasing the type information at runtime, and deferring all check to static type checkers only:</p> <div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">typedef</span> <span class="n">struct</span> <span class="p">{</span> <span class="n">PyObject_HEAD</span> <span class="o">/*</span> <span class="o">...</span> <span class="n">your</span> <span class="n">code</span> <span class="o">...</span> <span class="o">*/</span> <span class="p">}</span> <span class="n">SimpleGeneric</span><span class="p">;</span> <span class="n">static</span> <span class="n">PyObject</span> <span class="o">*</span> <span class="n">simple_class_getitem</span><span class="p">(</span><span class="n">PyObject</span> <span class="o">*</span><span class="nb">type</span><span class="p">,</span> <span class="n">PyObject</span> <span class="o">*</span><span class="n">item</span><span class="p">)</span> <span class="p">{</span> <span class="n">Py_INCREF</span><span class="p">(</span><span class="nb">type</span><span class="p">);</span> <span class="k">return</span> <span class="nb">type</span><span class="p">;</span> <span class="p">}</span> <span class="n">static</span> <span class="n">PyMethodDef</span> <span class="n">simple_generic_methods</span><span class="p">[]</span> <span class="o">=</span> <span class="p">{</span> <span class="p">{</span><span class="s2">"__class_getitem__"</span><span class="p">,</span> <span class="n">simple_class_getitem</span><span class="p">,</span> <span class="n">METH_O</span><span class="o">|</span><span class="n">METH_CLASS</span><span class="p">,</span> <span class="n">NULL</span><span class="p">},</span> <span class="o">/*</span> <span class="o">...</span> <span class="n">other</span> <span class="n">methods</span> <span class="o">...</span> <span class="o">*/</span> <span class="p">};</span> <span class="n">PyTypeObject</span> <span class="n">SimpleGeneric_Type</span> <span class="o">=</span> <span class="p">{</span> <span class="n">PyVarObject_HEAD_INIT</span><span class="p">(</span><span class="n">NULL</span><span class="p">,</span> <span class="mi">0</span><span class="p">)</span> <span class="s2">"SimpleGeneric"</span><span class="p">,</span> <span class="n">sizeof</span><span class="p">(</span><span class="n">SimpleGeneric</span><span class="p">),</span> <span class="mi">0</span><span class="p">,</span> <span class="o">.</span><span class="n">tp_flags</span> <span class="o">=</span> <span class="n">Py_TPFLAGS_DEFAULT</span> <span class="o">|</span> <span class="n">Py_TPFLAGS_BASETYPE</span><span class="p">,</span> <span class="o">.</span><span class="n">tp_methods</span> <span class="o">=</span> <span class="n">simple_generic_methods</span><span class="p">,</span> <span class="p">};</span> </pre></div> </div> <p>Such class can be used as a normal generic in Python type annotations (a corresponding stub file should be provided for static type checkers, see <a class="pep reference internal" href="../pep-0484/" title="PEP 484 – Type Hints">PEP 484</a> for details):</p> <div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="kn">from</span><span class="w"> </span><span class="nn">simple_extension</span><span class="w"> </span><span class="kn">import</span> <span class="n">SimpleGeneric</span> <span class="kn">from</span><span class="w"> </span><span class="nn">typing</span><span class="w"> </span><span class="kn">import</span> <span class="n">TypeVar</span> <span class="n">T</span> <span class="o">=</span> <span class="n">TypeVar</span><span class="p">(</span><span class="s1">'T'</span><span class="p">)</span> <span class="n">Alias</span> <span class="o">=</span> <span class="n">SimpleGeneric</span><span class="p">[</span><span class="nb">str</span><span class="p">,</span> <span class="n">T</span><span class="p">]</span> <span class="k">class</span><span class="w"> </span><span class="nc">SubClass</span><span class="p">(</span><span class="n">SimpleGeneric</span><span class="p">[</span><span class="n">T</span><span class="p">,</span> <span class="nb">int</span><span class="p">]):</span> <span class="o">...</span> <span class="n">data</span><span class="p">:</span> <span class="n">Alias</span><span class="p">[</span><span class="nb">int</span><span class="p">]</span> <span class="c1"># Works at runtime</span> <span class="n">more_data</span><span class="p">:</span> <span class="n">SubClass</span><span class="p">[</span><span class="nb">str</span><span class="p">]</span> <span class="c1"># Also works at runtime</span> </pre></div> </div> </section> </section> <section id="backwards-compatibility-and-impact-on-users-who-don-t-use-typing"> <h2><a class="toc-backref" href="#backwards-compatibility-and-impact-on-users-who-don-t-use-typing" role="doc-backlink">Backwards compatibility and impact on users who don’t use <code class="docutils literal notranslate"><span class="pre">typing</span></code></a></h2> <p>This proposal may break code that currently uses the names <code class="docutils literal notranslate"><span class="pre">__class_getitem__</span></code> and <code class="docutils literal notranslate"><span class="pre">__mro_entries__</span></code>. (But the language reference explicitly reserves <em>all</em> undocumented dunder names, and allows “breakage without warning”; see <a class="footnote-reference brackets" href="#id12" id="id6">[6]</a>.)</p> <p>This proposal will support almost complete backwards compatibility with the current public generic types API; moreover the <code class="docutils literal notranslate"><span class="pre">typing</span></code> module is still provisional. The only two exceptions are that currently <code class="docutils literal notranslate"><span class="pre">issubclass(List[int],</span> <span class="pre">List)</span></code> returns True, while with this proposal it will raise <code class="docutils literal notranslate"><span class="pre">TypeError</span></code>, and <code class="docutils literal notranslate"><span class="pre">repr()</span></code> of unsubscripted user-defined generics cannot be tweaked and will coincide with <code class="docutils literal notranslate"><span class="pre">repr()</span></code> of normal (non-generic) classes.</p> <p>With the reference implementation I measured negligible performance effects (under 1% on a micro-benchmark) for regular (non-generic) classes. At the same time performance of generics is significantly improved:</p> <ul class="simple"> <li><code class="docutils literal notranslate"><span class="pre">importlib.reload(typing)</span></code> is up to 7x faster</li> <li>Creation of user defined generic classes is up to 4x faster (on a micro-benchmark with an empty body)</li> <li>Instantiation of generic classes is up to 5x faster (on a micro-benchmark with an empty <code class="docutils literal notranslate"><span class="pre">__init__</span></code>)</li> <li>Other operations with generic types and instances (like method lookup and <code class="docutils literal notranslate"><span class="pre">isinstance()</span></code> checks) are improved by around 10-20%</li> <li>The only aspect that gets slower with the current proof of concept implementation is the subscripted generics cache look-up. However it was already very efficient, so this aspect gives negligible overall impact.</li> </ul> </section> <section id="references"> <h2><a class="toc-backref" href="#references" role="doc-backlink">References</a></h2> <aside class="footnote-list brackets"> <aside class="footnote brackets" id="id7" role="doc-footnote"> <dt class="label" id="id7">[<a href="#id1">1</a>]</dt> <dd>Discussion following Mark Shannon’s presentation at Language Summit (<a class="reference external" href="https://github.com/python/typing/issues/432">https://github.com/python/typing/issues/432</a>)</aside> <aside class="footnote brackets" id="id8" role="doc-footnote"> <dt class="label" id="id8">[<a href="#id2">2</a>]</dt> <dd>Pull Request to implement shared generic ABC caches (merged) (<a class="reference external" href="https://github.com/python/typing/pull/383">https://github.com/python/typing/pull/383</a>)</aside> <aside class="footnote brackets" id="id9" role="doc-footnote"> <dt class="label" id="id9">[<a href="#id3">3</a>]</dt> <dd>An old bug with setting/accessing attributes on generic types (<a class="reference external" href="https://github.com/python/typing/issues/392">https://github.com/python/typing/issues/392</a>)</aside> <aside class="footnote brackets" id="id10" role="doc-footnote"> <dt class="label" id="id10">[<a href="#id4">4</a>]</dt> <dd>The reference implementation (<a class="reference external" href="https://github.com/ilevkivskyi/cpython/pull/2/files">https://github.com/ilevkivskyi/cpython/pull/2/files</a>, <a class="reference external" href="https://github.com/ilevkivskyi/cpython/tree/new-typing">https://github.com/ilevkivskyi/cpython/tree/new-typing</a>)</aside> <aside class="footnote brackets" id="id11" role="doc-footnote"> <dt class="label" id="id11">[<a href="#id5">5</a>]</dt> <dd>Original proposal (<a class="reference external" href="https://github.com/python/typing/issues/468">https://github.com/python/typing/issues/468</a>)</aside> <aside class="footnote brackets" id="id12" role="doc-footnote"> <dt class="label" id="id12">[<a href="#id6">6</a>]</dt> <dd>Reserved classes of identifiers (<a class="reference external" href="https://docs.python.org/3/reference/lexical_analysis.html#reserved-classes-of-identifiers">https://docs.python.org/3/reference/lexical_analysis.html#reserved-classes-of-identifiers</a>)</aside> </aside> </section> <section id="copyright"> <h2><a class="toc-backref" href="#copyright" role="doc-backlink">Copyright</a></h2> <p>This document has been placed in the public domain.</p> </section> </section> <hr class="docutils" /> <p>Source: <a class="reference external" href="https://github.com/python/peps/blob/main/peps/pep-0560.rst">https://github.com/python/peps/blob/main/peps/pep-0560.rst</a></p> <p>Last modified: <a class="reference external" href="https://github.com/python/peps/commits/main/peps/pep-0560.rst">2024-06-11 22:12:09 GMT</a></p> </article> <nav id="pep-sidebar"> <h2>Contents</h2> <ul> <li><a class="reference internal" href="#abstract">Abstract</a></li> <li><a class="reference internal" href="#rationale">Rationale</a><ul> <li><a class="reference internal" href="#performance">Performance</a></li> <li><a class="reference internal" href="#metaclass-conflicts">Metaclass conflicts</a></li> <li><a class="reference internal" href="#hacks-and-bugs-that-will-be-removed-by-this-proposal">Hacks and bugs that will be removed by this proposal</a></li> </ul> </li> <li><a class="reference internal" href="#specification">Specification</a><ul> <li><a class="reference internal" href="#class-getitem"><code class="docutils literal notranslate"><span class="pre">__class_getitem__</span></code></a></li> <li><a class="reference internal" href="#mro-entries"><code class="docutils literal notranslate"><span class="pre">__mro_entries__</span></code></a></li> <li><a class="reference internal" href="#dynamic-class-creation-and-types-resolve-bases">Dynamic class creation and <code class="docutils literal notranslate"><span class="pre">types.resolve_bases</span></code></a></li> <li><a class="reference internal" href="#using-class-getitem-in-c-extensions">Using <code class="docutils literal notranslate"><span class="pre">__class_getitem__</span></code> in C extensions</a></li> </ul> </li> <li><a class="reference internal" href="#backwards-compatibility-and-impact-on-users-who-don-t-use-typing">Backwards compatibility and impact on users who don’t use <code class="docutils literal notranslate"><span class="pre">typing</span></code></a></li> <li><a class="reference internal" href="#references">References</a></li> <li><a class="reference internal" href="#copyright">Copyright</a></li> </ul> <br> <a id="source" href="https://github.com/python/peps/blob/main/peps/pep-0560.rst">Page Source (GitHub)</a> </nav> </section> <script src="../_static/colour_scheme.js"></script> <script src="../_static/wrap_tables.js"></script> <script src="../_static/sticky_banner.js"></script> </body> </html>