CINXE.COM

PEP 403 – General purpose decorator clause (aka “@in” clause) | peps.python.org

<!DOCTYPE html> <html lang="en"> <head> <meta charset="utf-8"> <meta name="viewport" content="width=device-width, initial-scale=1.0"> <meta name="color-scheme" content="light dark"> <title>PEP 403 – General purpose decorator clause (aka “@in” clause) | peps.python.org</title> <link rel="shortcut icon" href="../_static/py.png"> <link rel="canonical" href="https://peps.python.org/pep-0403/"> <link rel="stylesheet" href="../_static/style.css" type="text/css"> <link rel="stylesheet" href="../_static/mq.css" type="text/css"> <link rel="stylesheet" href="../_static/pygments.css" type="text/css" media="(prefers-color-scheme: light)" id="pyg-light"> <link rel="stylesheet" href="../_static/pygments_dark.css" type="text/css" media="(prefers-color-scheme: dark)" id="pyg-dark"> <link rel="alternate" type="application/rss+xml" title="Latest PEPs" href="https://peps.python.org/peps.rss"> <meta property="og:title" content='PEP 403 – General purpose decorator clause (aka “@in” clause) | peps.python.org'> <meta property="og:description" content="This PEP proposes the addition of a new @in decorator clause that makes it possible to override the name binding step of a function or class definition."> <meta property="og:type" content="website"> <meta property="og:url" content="https://peps.python.org/pep-0403/"> <meta property="og:site_name" content="Python Enhancement Proposals (PEPs)"> <meta property="og:image" content="https://peps.python.org/_static/og-image.png"> <meta property="og:image:alt" content="Python PEPs"> <meta property="og:image:width" content="200"> <meta property="og:image:height" content="200"> <meta name="description" content="This PEP proposes the addition of a new @in decorator clause that makes it possible to override the name binding step of a function or class definition."> <meta name="theme-color" content="#3776ab"> </head> <body> <svg xmlns="http://www.w3.org/2000/svg" style="display: none;"> <symbol id="svg-sun-half" viewBox="0 0 24 24" pointer-events="all"> <title>Following system colour scheme</title> <svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"> <circle cx="12" cy="12" r="9"></circle> <path d="M12 3v18m0-12l4.65-4.65M12 14.3l7.37-7.37M12 19.6l8.85-8.85"></path> </svg> </symbol> <symbol id="svg-moon" viewBox="0 0 24 24" pointer-events="all"> <title>Selected dark colour scheme</title> <svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"> <path stroke="none" d="M0 0h24v24H0z" fill="none"></path> <path d="M12 3c.132 0 .263 0 .393 0a7.5 7.5 0 0 0 7.92 12.446a9 9 0 1 1 -8.313 -12.454z"></path> </svg> </symbol> <symbol id="svg-sun" viewBox="0 0 24 24" pointer-events="all"> <title>Selected light colour scheme</title> <svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"> <circle cx="12" cy="12" r="5"></circle> <line x1="12" y1="1" x2="12" y2="3"></line> <line x1="12" y1="21" x2="12" y2="23"></line> <line x1="4.22" y1="4.22" x2="5.64" y2="5.64"></line> <line x1="18.36" y1="18.36" x2="19.78" y2="19.78"></line> <line x1="1" y1="12" x2="3" y2="12"></line> <line x1="21" y1="12" x2="23" y2="12"></line> <line x1="4.22" y1="19.78" x2="5.64" y2="18.36"></line> <line x1="18.36" y1="5.64" x2="19.78" y2="4.22"></line> </svg> </symbol> </svg> <script> document.documentElement.dataset.colour_scheme = localStorage.getItem("colour_scheme") || "auto" </script> <section id="pep-page-section"> <header> <h1>Python Enhancement Proposals</h1> <ul class="breadcrumbs"> <li><a href="https://www.python.org/" title="The Python Programming Language">Python</a> &raquo; </li> <li><a href="../pep-0000/">PEP Index</a> &raquo; </li> <li>PEP 403</li> </ul> <button id="colour-scheme-cycler" onClick="setColourScheme(nextColourScheme())"> <svg aria-hidden="true" class="colour-scheme-icon-when-auto"><use href="#svg-sun-half"></use></svg> <svg aria-hidden="true" class="colour-scheme-icon-when-dark"><use href="#svg-moon"></use></svg> <svg aria-hidden="true" class="colour-scheme-icon-when-light"><use href="#svg-sun"></use></svg> <span class="visually-hidden">Toggle light / dark / auto colour theme</span> </button> </header> <article> <section id="pep-content"> <h1 class="page-title">PEP 403 – General purpose decorator clause (aka “&#64;in” clause)</h1> <dl class="rfc2822 field-list simple"> <dt class="field-odd">Author<span class="colon">:</span></dt> <dd class="field-odd">Alyssa Coghlan &lt;ncoghlan&#32;&#97;t&#32;gmail.com&gt;</dd> <dt class="field-even">Status<span class="colon">:</span></dt> <dd class="field-even"><abbr title="Inactive draft that may be taken up again at a later time">Deferred</abbr></dd> <dt class="field-odd">Type<span class="colon">:</span></dt> <dd class="field-odd"><abbr title="Normative PEP with a new feature for Python, implementation change for CPython or interoperability standard for the ecosystem">Standards Track</abbr></dd> <dt class="field-even">Created<span class="colon">:</span></dt> <dd class="field-even">13-Oct-2011</dd> <dt class="field-odd">Python-Version<span class="colon">:</span></dt> <dd class="field-odd">3.4</dd> <dt class="field-even">Post-History<span class="colon">:</span></dt> <dd class="field-even">13-Oct-2011</dd> </dl> <hr class="docutils" /> <section id="contents"> <details><summary>Table of Contents</summary><ul class="simple"> <li><a class="reference internal" href="#abstract">Abstract</a></li> <li><a class="reference internal" href="#basic-examples">Basic Examples</a></li> <li><a class="reference internal" href="#proposal">Proposal</a><ul> <li><a class="reference internal" href="#syntax-change">Syntax Change</a></li> </ul> </li> <li><a class="reference internal" href="#design-discussion">Design Discussion</a><ul> <li><a class="reference internal" href="#background">Background</a></li> <li><a class="reference internal" href="#relation-to-pep-3150">Relation to PEP 3150</a></li> <li><a class="reference internal" href="#keyword-choice">Keyword Choice</a></li> <li><a class="reference internal" href="#better-debugging-support-for-functions-and-classes-with-short-names">Better Debugging Support for Functions and Classes with Short Names</a></li> <li><a class="reference internal" href="#possible-implementation-strategy">Possible Implementation Strategy</a></li> <li><a class="reference internal" href="#explaining-container-comprehensions-and-generator-expressions">Explaining Container Comprehensions and Generator Expressions</a></li> </ul> </li> <li><a class="reference internal" href="#more-examples">More Examples</a></li> <li><a class="reference internal" href="#reference-implementation">Reference Implementation</a></li> <li><a class="reference internal" href="#acknowledgements">Acknowledgements</a></li> <li><a class="reference internal" href="#rejected-concepts">Rejected Concepts</a><ul> <li><a class="reference internal" href="#omitting-the-decorator-prefix-character">Omitting the decorator prefix character</a></li> <li><a class="reference internal" href="#anonymous-forward-references">Anonymous Forward References</a></li> <li><a class="reference internal" href="#using-a-nested-suite">Using a nested suite</a></li> </ul> </li> <li><a class="reference internal" href="#references">References</a></li> <li><a class="reference internal" href="#copyright">Copyright</a></li> </ul> </details></section> <section id="abstract"> <h2><a class="toc-backref" href="#abstract" role="doc-backlink">Abstract</a></h2> <p>This PEP proposes the addition of a new <code class="docutils literal notranslate"><span class="pre">&#64;in</span></code> decorator clause that makes it possible to override the name binding step of a function or class definition.</p> <p>The new clause accepts a single simple statement that can make a forward reference to decorated function or class definition.</p> <p>This new clause is designed to be used whenever a “one-shot” function or class is needed, and placing the function or class definition before the statement that uses it actually makes the code harder to read. It also avoids any name shadowing concerns by making sure the new name is visible only to the statement in the <code class="docutils literal notranslate"><span class="pre">&#64;in</span></code> clause.</p> <p>This PEP is based heavily on many of the ideas in <a class="pep reference internal" href="../pep-3150/" title="PEP 3150 – Statement local namespaces (aka “given” clause)">PEP 3150</a> (Statement Local Namespaces) so some elements of the rationale will be familiar to readers of that PEP. Both PEPs remain deferred for the time being, primarily due to the lack of compelling real world use cases in either PEP.</p> </section> <section id="basic-examples"> <h2><a class="toc-backref" href="#basic-examples" role="doc-backlink">Basic Examples</a></h2> <p>Before diving into the long history of this problem and the detailed rationale for this specific proposed solution, here are a few simple examples of the kind of code it is designed to simplify.</p> <p>As a trivial example, a weakref callback could be defined as follows:</p> <div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="nd">@in</span> <span class="n">x</span> <span class="o">=</span> <span class="n">weakref</span><span class="o">.</span><span class="n">ref</span><span class="p">(</span><span class="n">target</span><span class="p">,</span> <span class="n">report_destruction</span><span class="p">)</span> <span class="k">def</span><span class="w"> </span><span class="nf">report_destruction</span><span class="p">(</span><span class="n">obj</span><span class="p">):</span> <span class="nb">print</span><span class="p">(</span><span class="s2">&quot;</span><span class="si">{}</span><span class="s2"> is being destroyed&quot;</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">obj</span><span class="p">))</span> </pre></div> </div> <p>This contrasts with the current (conceptually) “out of order” syntax for this operation:</p> <div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="k">def</span><span class="w"> </span><span class="nf">report_destruction</span><span class="p">(</span><span class="n">obj</span><span class="p">):</span> <span class="nb">print</span><span class="p">(</span><span class="s2">&quot;</span><span class="si">{}</span><span class="s2"> is being destroyed&quot;</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">obj</span><span class="p">))</span> <span class="n">x</span> <span class="o">=</span> <span class="n">weakref</span><span class="o">.</span><span class="n">ref</span><span class="p">(</span><span class="n">target</span><span class="p">,</span> <span class="n">report_destruction</span><span class="p">)</span> </pre></div> </div> <p>That structure is OK when you’re using the callable multiple times, but it’s irritating to be forced into it for one-off operations.</p> <p>If the repetition of the name seems especially annoying, then a throwaway name like <code class="docutils literal notranslate"><span class="pre">f</span></code> can be used instead:</p> <div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="nd">@in</span> <span class="n">x</span> <span class="o">=</span> <span class="n">weakref</span><span class="o">.</span><span class="n">ref</span><span class="p">(</span><span class="n">target</span><span class="p">,</span> <span class="n">f</span><span class="p">)</span> <span class="k">def</span><span class="w"> </span><span class="nf">f</span><span class="p">(</span><span class="n">obj</span><span class="p">):</span> <span class="nb">print</span><span class="p">(</span><span class="s2">&quot;</span><span class="si">{}</span><span class="s2"> is being destroyed&quot;</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">obj</span><span class="p">))</span> </pre></div> </div> <p>Similarly, a sorted operation on a particularly poorly defined type could now be defined as:</p> <div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="nd">@in</span> <span class="n">sorted_list</span> <span class="o">=</span> <span class="nb">sorted</span><span class="p">(</span><span class="n">original</span><span class="p">,</span> <span class="n">key</span><span class="o">=</span><span class="n">f</span><span class="p">)</span> <span class="k">def</span><span class="w"> </span><span class="nf">f</span><span class="p">(</span><span class="n">item</span><span class="p">):</span> <span class="k">try</span><span class="p">:</span> <span class="k">return</span> <span class="n">item</span><span class="o">.</span><span class="n">calc_sort_order</span><span class="p">()</span> <span class="k">except</span> <span class="n">NotSortableError</span><span class="p">:</span> <span class="k">return</span> <span class="nb">float</span><span class="p">(</span><span class="s1">&#39;inf&#39;</span><span class="p">)</span> </pre></div> </div> <p>Rather than:</p> <div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="k">def</span><span class="w"> </span><span class="nf">force_sort</span><span class="p">(</span><span class="n">item</span><span class="p">):</span> <span class="k">try</span><span class="p">:</span> <span class="k">return</span> <span class="n">item</span><span class="o">.</span><span class="n">calc_sort_order</span><span class="p">()</span> <span class="k">except</span> <span class="n">NotSortableError</span><span class="p">:</span> <span class="k">return</span> <span class="nb">float</span><span class="p">(</span><span class="s1">&#39;inf&#39;</span><span class="p">)</span> <span class="n">sorted_list</span> <span class="o">=</span> <span class="nb">sorted</span><span class="p">(</span><span class="n">original</span><span class="p">,</span> <span class="n">key</span><span class="o">=</span><span class="n">force_sort</span><span class="p">)</span> </pre></div> </div> <p>And early binding semantics in a list comprehension could be attained via:</p> <div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="nd">@in</span> <span class="n">funcs</span> <span class="o">=</span> <span class="p">[</span><span class="n">adder</span><span class="p">(</span><span class="n">i</span><span class="p">)</span> <span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">10</span><span class="p">)]</span> <span class="k">def</span><span class="w"> </span><span class="nf">adder</span><span class="p">(</span><span class="n">i</span><span class="p">):</span> <span class="k">return</span> <span class="k">lambda</span> <span class="n">x</span><span class="p">:</span> <span class="n">x</span> <span class="o">+</span> <span class="n">i</span> </pre></div> </div> </section> <section id="proposal"> <h2><a class="toc-backref" href="#proposal" role="doc-backlink">Proposal</a></h2> <p>This PEP proposes the addition of a new <code class="docutils literal notranslate"><span class="pre">&#64;in</span></code> clause that is a variant of the existing class and function decorator syntax.</p> <p>The new <code class="docutils literal notranslate"><span class="pre">&#64;in</span></code> clause precedes the decorator lines, and allows forward references to the trailing function or class definition.</p> <p>The trailing function or class definition is always named - the name of the trailing definition is then used to make the forward reference from the <code class="docutils literal notranslate"><span class="pre">&#64;in</span></code> clause.</p> <p>The <code class="docutils literal notranslate"><span class="pre">&#64;in</span></code> clause is allowed to contain any simple statement (including those that don’t make any sense in that context, such as <code class="docutils literal notranslate"><span class="pre">pass</span></code> - while such code would be legal, there wouldn’t be any point in writing it). This permissive structure is easier to define and easier to explain, but a more restrictive approach that only permits operations that “make sense” would also be possible (see <a class="pep reference internal" href="../pep-3150/" title="PEP 3150 – Statement local namespaces (aka “given” clause)">PEP 3150</a> for a list of possible candidates).</p> <p>The <code class="docutils literal notranslate"><span class="pre">&#64;in</span></code> clause will not create a new scope - all name binding operations aside from the trailing function or class definition will affect the containing scope.</p> <p>The name used in the trailing function or class definition is only visible from the associated <code class="docutils literal notranslate"><span class="pre">&#64;in</span></code> clause, and behaves as if it was an ordinary variable defined in that scope. If any nested scopes are created in either the <code class="docutils literal notranslate"><span class="pre">&#64;in</span></code> clause or the trailing function or class definition, those scopes will see the trailing function or class definition rather than any other bindings for that name in the containing scope.</p> <p>In a very real sense, this proposal is about making it possible to override the implicit “name = &lt;defined function or class&gt;” name binding operation that is part of every function or class definition, specifically in those cases where the local name binding isn’t actually needed.</p> <p>Under this PEP, an ordinary class or function definition:</p> <div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="nd">@deco2</span> <span class="nd">@deco1</span> <span class="k">def</span><span class="w"> </span><span class="nf">name</span><span class="p">():</span> <span class="o">...</span> </pre></div> </div> <p>can be explained as being roughly equivalent to:</p> <div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="nd">@in</span> <span class="n">name</span> <span class="o">=</span> <span class="n">deco2</span><span class="p">(</span><span class="n">deco1</span><span class="p">(</span><span class="n">name</span><span class="p">))</span> <span class="k">def</span><span class="w"> </span><span class="nf">name</span><span class="p">():</span> <span class="o">...</span> </pre></div> </div> <section id="syntax-change"> <h3><a class="toc-backref" href="#syntax-change" role="doc-backlink">Syntax Change</a></h3> <p>Syntactically, only one new grammar rule is needed:</p> <div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">in_stmt</span><span class="p">:</span> <span class="s1">&#39;@in&#39;</span> <span class="n">simple_stmt</span> <span class="n">decorated</span> </pre></div> </div> <p>Grammar: <a class="reference external" href="http://hg.python.org/cpython/file/default/Grammar/Grammar">http://hg.python.org/cpython/file/default/Grammar/Grammar</a></p> </section> </section> <section id="design-discussion"> <h2><a class="toc-backref" href="#design-discussion" role="doc-backlink">Design Discussion</a></h2> <section id="background"> <h3><a class="toc-backref" href="#background" role="doc-backlink">Background</a></h3> <p>The question of “multi-line lambdas” has been a vexing one for many Python users for a very long time, and it took an exploration of Ruby’s block functionality for me to finally understand why this bugs people so much: Python’s demand that the function be named and introduced before the operation that needs it breaks the developer’s flow of thought. They get to a point where they go “I need a one-shot operation that does &lt;X&gt;”, and instead of being able to just <em>say</em> that directly, they instead have to back up, name a function to do &lt;X&gt;, then call that function from the operation they actually wanted to do in the first place. Lambda expressions can help sometimes, but they’re no substitute for being able to use a full suite.</p> <p>Ruby’s block syntax also heavily inspired the style of the solution in this PEP, by making it clear that even when limited to <em>one</em> anonymous function per statement, anonymous functions could still be incredibly useful. Consider how many constructs Python has where one expression is responsible for the bulk of the heavy lifting:</p> <ul class="simple"> <li>comprehensions, generator expressions, map(), filter()</li> <li>key arguments to sorted(), min(), max()</li> <li>partial function application</li> <li>provision of callbacks (e.g. for weak references or asynchronous IO)</li> <li>array broadcast operations in NumPy</li> </ul> <p>However, adopting Ruby’s block syntax directly won’t work for Python, since the effectiveness of Ruby’s blocks relies heavily on various conventions in the way functions are <em>defined</em> (specifically, using Ruby’s <code class="docutils literal notranslate"><span class="pre">yield</span></code> syntax to call blocks directly and the <code class="docutils literal notranslate"><span class="pre">&amp;arg</span></code> mechanism to accept a block as a function’s final argument).</p> <p>Since Python has relied on named functions for so long, the signatures of APIs that accept callbacks are far more diverse, thus requiring a solution that allows one-shot functions to be slotted in at the appropriate location.</p> <p>The approach taken in this PEP is to retain the requirement to name the function explicitly, but allow the relative order of the definition and the statement that references it to be changed to match the developer’s flow of thought. The rationale is essentially the same as that used when introducing decorators, but covering a broader set of applications.</p> </section> <section id="relation-to-pep-3150"> <h3><a class="toc-backref" href="#relation-to-pep-3150" role="doc-backlink">Relation to PEP 3150</a></h3> <p><a class="pep reference internal" href="../pep-3150/" title="PEP 3150 – Statement local namespaces (aka “given” clause)">PEP 3150</a> (Statement Local Namespaces) describes its primary motivation as being to elevate ordinary assignment statements to be on par with <code class="docutils literal notranslate"><span class="pre">class</span></code> and <code class="docutils literal notranslate"><span class="pre">def</span></code> statements where the name of the item to be defined is presented to the reader in advance of the details of how the value of that item is calculated. This PEP achieves the same goal in a different way, by allowing the simple name binding of a standard function definition to be replaced with something else (like assigning the result of the function to a value).</p> <p>Despite having the same author, the two PEPs are in direct competition with each other. <a class="pep reference internal" href="../pep-0403/" title="PEP 403 – General purpose decorator clause (aka “&#64;in” clause)">PEP 403</a> represents a minimalist approach that attempts to achieve useful functionality with a minimum of change from the status quo. This PEP instead aims for a more flexible standalone statement design, which requires a larger degree of change to the language.</p> <p>Note that where <a class="pep reference internal" href="../pep-0403/" title="PEP 403 – General purpose decorator clause (aka “&#64;in” clause)">PEP 403</a> is better suited to explaining the behaviour of generator expressions correctly, this PEP is better able to explain the behaviour of decorator clauses in general. Both PEPs support adequate explanations for the semantics of container comprehensions.</p> </section> <section id="keyword-choice"> <h3><a class="toc-backref" href="#keyword-choice" role="doc-backlink">Keyword Choice</a></h3> <p>The proposal definitely requires <em>some</em> kind of prefix to avoid parsing ambiguity and backwards compatibility problems with existing constructs. It also needs to be clearly highlighted to readers, since it declares that the following piece of code is going to be executed only after the trailing function or class definition has been executed.</p> <p>The <code class="docutils literal notranslate"><span class="pre">in</span></code> keyword was chosen as an existing keyword that can be used to denote the concept of a forward reference.</p> <p>The <code class="docutils literal notranslate"><span class="pre">&#64;</span></code> prefix was included in order to exploit the fact that Python programmers are already used to decorator syntax as an indication of out of order execution, where the function or class is actually defined <em>first</em> and then decorators are applied in reverse order.</p> <p>For functions, the construct is intended to be read as “in &lt;this statement that references NAME&gt; define NAME as a function that does &lt;operation&gt;”.</p> <p>The mapping to English prose isn’t as obvious for the class definition case, but the concept remains the same.</p> </section> <section id="better-debugging-support-for-functions-and-classes-with-short-names"> <h3><a class="toc-backref" href="#better-debugging-support-for-functions-and-classes-with-short-names" role="doc-backlink">Better Debugging Support for Functions and Classes with Short Names</a></h3> <p>One of the objections to widespread use of lambda expressions is that they have a negative effect on traceback intelligibility and other aspects of introspection. Similar objections are raised regarding constructs that promote short, cryptic function names (including this one, which requires that the name of the trailing definition be supplied at least twice, encouraging the use of shorthand placeholder names like <code class="docutils literal notranslate"><span class="pre">f</span></code>).</p> <p>However, the introduction of qualified names in <a class="pep reference internal" href="../pep-3155/" title="PEP 3155 – Qualified name for classes and functions">PEP 3155</a> means that even anonymous classes and functions will now have different representations if they occur in different scopes. For example:</p> <div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="gp">&gt;&gt;&gt; </span><span class="k">def</span><span class="w"> </span><span class="nf">f</span><span class="p">():</span> <span class="gp">... </span> <span class="k">return</span> <span class="k">lambda</span><span class="p">:</span> <span class="n">y</span> <span class="gp">...</span> <span class="gp">&gt;&gt;&gt; </span><span class="n">f</span><span class="p">()</span> <span class="go">&lt;function f.&lt;locals&gt;.&lt;lambda&gt; at 0x7f6f46faeae0&gt;</span> </pre></div> </div> <p>Anonymous functions (or functions that share a name) within the <em>same</em> scope will still share representations (aside from the object ID), but this is still a major improvement over the historical situation where everything <em>except</em> the object ID was identical.</p> </section> <section id="possible-implementation-strategy"> <h3><a class="toc-backref" href="#possible-implementation-strategy" role="doc-backlink">Possible Implementation Strategy</a></h3> <p>This proposal has at least one titanic advantage over <a class="pep reference internal" href="../pep-3150/" title="PEP 3150 – Statement local namespaces (aka “given” clause)">PEP 3150</a>: implementation should be relatively straightforward.</p> <p>The <code class="docutils literal notranslate"><span class="pre">&#64;in</span></code> clause will be included in the AST for the associated function or class definition and the statement that references it. When the <code class="docutils literal notranslate"><span class="pre">&#64;in</span></code> clause is present, it will be emitted in place of the local name binding operation normally implied by a function or class definition.</p> <p>The one potentially tricky part is changing the meaning of the references to the statement local function or namespace while within the scope of the <code class="docutils literal notranslate"><span class="pre">in</span></code> statement, but that shouldn’t be too hard to address by maintaining some additional state within the compiler (it’s much easier to handle this for a single name than it is for an unknown number of names in a full nested suite).</p> </section> <section id="explaining-container-comprehensions-and-generator-expressions"> <h3><a class="toc-backref" href="#explaining-container-comprehensions-and-generator-expressions" role="doc-backlink">Explaining Container Comprehensions and Generator Expressions</a></h3> <p>One interesting feature of the proposed construct is that it can be used as a primitive to explain the scoping and execution order semantics of both generator expressions and container comprehensions:</p> <div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">seq2</span> <span class="o">=</span> <span class="p">[</span><span class="n">x</span> <span class="k">for</span> <span class="n">x</span> <span class="ow">in</span> <span class="n">y</span> <span class="k">if</span> <span class="n">q</span><span class="p">(</span><span class="n">x</span><span class="p">)</span> <span class="k">for</span> <span class="n">y</span> <span class="ow">in</span> <span class="n">seq</span> <span class="k">if</span> <span class="n">p</span><span class="p">(</span><span class="n">y</span><span class="p">)]</span> <span class="c1"># would be equivalent to</span> <span class="nd">@in</span> <span class="n">seq2</span> <span class="o">=</span> <span class="n">f</span><span class="p">(</span><span class="n">seq</span><span class="p">):</span> <span class="k">def</span><span class="w"> </span><span class="nf">f</span><span class="p">(</span><span class="n">seq</span><span class="p">)</span> <span class="n">result</span> <span class="o">=</span> <span class="p">[]</span> <span class="k">for</span> <span class="n">y</span> <span class="ow">in</span> <span class="n">seq</span><span class="p">:</span> <span class="k">if</span> <span class="n">p</span><span class="p">(</span><span class="n">y</span><span class="p">):</span> <span class="k">for</span> <span class="n">x</span> <span class="ow">in</span> <span class="n">y</span><span class="p">:</span> <span class="k">if</span> <span class="n">q</span><span class="p">(</span><span class="n">x</span><span class="p">):</span> <span class="n">result</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">x</span><span class="p">)</span> <span class="k">return</span> <span class="n">result</span> </pre></div> </div> <p>The important point in this expansion is that it explains why comprehensions appear to misbehave at class scope: only the outermost iterator is evaluated at class scope, while all predicates, nested iterators and value expressions are evaluated inside a nested scope.</p> <p>An equivalent expansion is possible for generator expressions:</p> <div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">gen</span> <span class="o">=</span> <span class="p">(</span><span class="n">x</span> <span class="k">for</span> <span class="n">x</span> <span class="ow">in</span> <span class="n">y</span> <span class="k">if</span> <span class="n">q</span><span class="p">(</span><span class="n">x</span><span class="p">)</span> <span class="k">for</span> <span class="n">y</span> <span class="ow">in</span> <span class="n">seq</span> <span class="k">if</span> <span class="n">p</span><span class="p">(</span><span class="n">y</span><span class="p">))</span> <span class="c1"># would be equivalent to</span> <span class="nd">@in</span> <span class="n">gen</span> <span class="o">=</span> <span class="n">g</span><span class="p">(</span><span class="n">seq</span><span class="p">):</span> <span class="k">def</span><span class="w"> </span><span class="nf">g</span><span class="p">(</span><span class="n">seq</span><span class="p">)</span> <span class="k">for</span> <span class="n">y</span> <span class="ow">in</span> <span class="n">seq</span><span class="p">:</span> <span class="k">if</span> <span class="n">p</span><span class="p">(</span><span class="n">y</span><span class="p">):</span> <span class="k">for</span> <span class="n">x</span> <span class="ow">in</span> <span class="n">y</span><span class="p">:</span> <span class="k">if</span> <span class="n">q</span><span class="p">(</span><span class="n">x</span><span class="p">):</span> <span class="k">yield</span> <span class="n">x</span> </pre></div> </div> </section> </section> <section id="more-examples"> <h2><a class="toc-backref" href="#more-examples" role="doc-backlink">More Examples</a></h2> <p>Calculating attributes without polluting the local namespace (from os.py):</p> <div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="c1"># Current Python (manual namespace cleanup)</span> <span class="k">def</span><span class="w"> </span><span class="nf">_createenviron</span><span class="p">():</span> <span class="o">...</span> <span class="c1"># 27 line function</span> <span class="n">environ</span> <span class="o">=</span> <span class="n">_createenviron</span><span class="p">()</span> <span class="k">del</span> <span class="n">_createenviron</span> <span class="c1"># Becomes:</span> <span class="nd">@in</span> <span class="n">environ</span> <span class="o">=</span> <span class="n">_createenviron</span><span class="p">()</span> <span class="k">def</span><span class="w"> </span><span class="nf">_createenviron</span><span class="p">():</span> <span class="o">...</span> <span class="c1"># 27 line function</span> </pre></div> </div> <p>Loop early binding:</p> <div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="c1"># Current Python (default argument hack)</span> <span class="n">funcs</span> <span class="o">=</span> <span class="p">[(</span><span class="k">lambda</span> <span class="n">x</span><span class="p">,</span> <span class="n">i</span><span class="o">=</span><span class="n">i</span><span class="p">:</span> <span class="n">x</span> <span class="o">+</span> <span class="n">i</span><span class="p">)</span> <span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">10</span><span class="p">)]</span> <span class="c1"># Becomes:</span> <span class="nd">@in</span> <span class="n">funcs</span> <span class="o">=</span> <span class="p">[</span><span class="n">adder</span><span class="p">(</span><span class="n">i</span><span class="p">)</span> <span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">10</span><span class="p">)]</span> <span class="k">def</span><span class="w"> </span><span class="nf">adder</span><span class="p">(</span><span class="n">i</span><span class="p">):</span> <span class="k">return</span> <span class="k">lambda</span> <span class="n">x</span><span class="p">:</span> <span class="n">x</span> <span class="o">+</span> <span class="n">i</span> <span class="c1"># Or even:</span> <span class="nd">@in</span> <span class="n">funcs</span> <span class="o">=</span> <span class="p">[</span><span class="n">adder</span><span class="p">(</span><span class="n">i</span><span class="p">)</span> <span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">10</span><span class="p">)]</span> <span class="k">def</span><span class="w"> </span><span class="nf">adder</span><span class="p">(</span><span class="n">i</span><span class="p">):</span> <span class="nd">@in</span> <span class="k">return</span> <span class="n">incr</span> <span class="k">def</span><span class="w"> </span><span class="nf">incr</span><span class="p">(</span><span class="n">x</span><span class="p">):</span> <span class="k">return</span> <span class="n">x</span> <span class="o">+</span> <span class="n">i</span> </pre></div> </div> <p>A trailing class can be used as a statement local namespace:</p> <div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="c1"># Evaluate subexpressions only once</span> <span class="nd">@in</span> <span class="n">c</span> <span class="o">=</span> <span class="n">math</span><span class="o">.</span><span class="n">sqrt</span><span class="p">(</span><span class="n">x</span><span class="o">.</span><span class="n">a</span><span class="o">*</span><span class="n">x</span><span class="o">.</span><span class="n">a</span> <span class="o">+</span> <span class="n">x</span><span class="o">.</span><span class="n">b</span><span class="o">*</span><span class="n">x</span><span class="o">.</span><span class="n">b</span><span class="p">)</span> <span class="k">class</span><span class="w"> </span><span class="nc">x</span><span class="p">:</span> <span class="n">a</span> <span class="o">=</span> <span class="n">calculate_a</span><span class="p">()</span> <span class="n">b</span> <span class="o">=</span> <span class="n">calculate_b</span><span class="p">()</span> </pre></div> </div> <p>A function can be bound directly to a location which isn’t a valid identifier:</p> <div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="nd">@in</span> <span class="n">dispatch</span><span class="p">[</span><span class="n">MyClass</span><span class="p">]</span> <span class="o">=</span> <span class="n">f</span> <span class="k">def</span><span class="w"> </span><span class="nf">f</span><span class="p">():</span> <span class="o">...</span> </pre></div> </div> <p>Constructs that verge on decorator abuse can be eliminated:</p> <div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="c1"># Current Python</span> <span class="nd">@call</span> <span class="k">def</span><span class="w"> </span><span class="nf">f</span><span class="p">():</span> <span class="o">...</span> <span class="c1"># Becomes:</span> <span class="nd">@in</span> <span class="n">f</span><span class="p">()</span> <span class="k">def</span><span class="w"> </span><span class="nf">f</span><span class="p">():</span> <span class="o">...</span> </pre></div> </div> </section> <section id="reference-implementation"> <h2><a class="toc-backref" href="#reference-implementation" role="doc-backlink">Reference Implementation</a></h2> <p>None as yet.</p> </section> <section id="acknowledgements"> <h2><a class="toc-backref" href="#acknowledgements" role="doc-backlink">Acknowledgements</a></h2> <p>Huge thanks to Gary Bernhardt for being blunt in pointing out that I had no idea what I was talking about in criticising Ruby’s blocks, kicking off a rather enlightening process of investigation.</p> </section> <section id="rejected-concepts"> <h2><a class="toc-backref" href="#rejected-concepts" role="doc-backlink">Rejected Concepts</a></h2> <p>To avoid retreading previously covered ground, some rejected alternatives are documented in this section.</p> <section id="omitting-the-decorator-prefix-character"> <h3><a class="toc-backref" href="#omitting-the-decorator-prefix-character" role="doc-backlink">Omitting the decorator prefix character</a></h3> <p>Earlier versions of this proposal omitted the <code class="docutils literal notranslate"><span class="pre">&#64;</span></code> prefix. However, without that prefix, the bare <code class="docutils literal notranslate"><span class="pre">in</span></code> keyword didn’t associate the clause strongly enough with the subsequent function or class definition. Reusing the decorator prefix and explicitly characterising the new construct as a kind of decorator clause is intended to help users link the two concepts and see them as two variants of the same idea.</p> </section> <section id="anonymous-forward-references"> <h3><a class="toc-backref" href="#anonymous-forward-references" role="doc-backlink">Anonymous Forward References</a></h3> <p>A previous incarnation of this PEP (see <a class="footnote-reference brackets" href="#id2" id="id1">[1]</a>) proposed a syntax where the new clause was introduced with <code class="docutils literal notranslate"><span class="pre">:</span></code> and the forward reference was written using <code class="docutils literal notranslate"><span class="pre">&#64;</span></code>. Feedback on this variant was almost universally negative, as it was considered both ugly and excessively magical:</p> <div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="p">:</span><span class="n">x</span> <span class="o">=</span> <span class="n">weakref</span><span class="o">.</span><span class="n">ref</span><span class="p">(</span><span class="n">target</span><span class="p">,</span> <span class="o">@</span><span class="p">)</span> <span class="k">def</span><span class="w"> </span><span class="nf">report_destruction</span><span class="p">(</span><span class="n">obj</span><span class="p">):</span> <span class="nb">print</span><span class="p">(</span><span class="s2">&quot;</span><span class="si">{}</span><span class="s2"> is being destroyed&quot;</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">obj</span><span class="p">))</span> </pre></div> </div> <p>A more recent variant always used <code class="docutils literal notranslate"><span class="pre">...</span></code> for forward references, along with genuinely anonymous function and class definitions. However, this degenerated quickly into a mass of unintelligible dots in more complex cases:</p> <div class="highlight-default notranslate"><div class="highlight"><pre><span></span>in funcs = [...(i) for i in range(10)] def ...(i): in return ... def ...(x): return x + i in c = math.sqrt(....a*....a + ....b*....b) class ...: a = calculate_a() b = calculate_b() </pre></div> </div> </section> <section id="using-a-nested-suite"> <h3><a class="toc-backref" href="#using-a-nested-suite" role="doc-backlink">Using a nested suite</a></h3> <p>The problems with using a full nested suite are best described in <a class="pep reference internal" href="../pep-3150/" title="PEP 3150 – Statement local namespaces (aka “given” clause)">PEP 3150</a>. It’s comparatively difficult to implement properly, the scoping semantics are harder to explain and it creates quite a few situations where there are two ways to do it without clear guidelines for choosing between them (as almost any construct that can be expressed with ordinary imperative code could instead be expressed using a given statement). While the PEP does propose some new <a class="pep reference internal" href="../pep-0008/" title="PEP 8 – Style Guide for Python Code">PEP 8</a> guidelines to help address that last problem, the difficulties in implementation are not so easily dealt with.</p> <p>By contrast, the decorator inspired syntax in this PEP explicitly limits the new feature to cases where it should actually improve readability, rather than harming it. As in the case of the original introduction of decorators, the idea of this new syntax is that if it <em>can</em> be used (i.e. the local name binding of the function is completely unnecessary) then it probably <em>should</em> be used.</p> <p>Another possible variant of this idea is to keep the decorator based <em>semantics</em> of this PEP, while adopting the prettier syntax from <a class="pep reference internal" href="../pep-3150/" title="PEP 3150 – Statement local namespaces (aka “given” clause)">PEP 3150</a>:</p> <div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">x</span> <span class="o">=</span> <span class="n">weakref</span><span class="o">.</span><span class="n">ref</span><span class="p">(</span><span class="n">target</span><span class="p">,</span> <span class="n">report_destruction</span><span class="p">)</span> <span class="n">given</span><span class="p">:</span> <span class="k">def</span><span class="w"> </span><span class="nf">report_destruction</span><span class="p">(</span><span class="n">obj</span><span class="p">):</span> <span class="nb">print</span><span class="p">(</span><span class="s2">&quot;</span><span class="si">{}</span><span class="s2"> is being destroyed&quot;</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">obj</span><span class="p">))</span> </pre></div> </div> <p>There are a couple of problems with this approach. The main issue is that this syntax variant uses something that looks like a suite, but really isn’t one. A secondary concern is that it’s not clear how the compiler will know which name(s) in the leading expression are forward references (although that could potentially be addressed through a suitable definition of the suite-that-is-not-a-suite in the language grammar).</p> <p>However, a nested suite has not yet been ruled out completely. The latest version of <a class="pep reference internal" href="../pep-3150/" title="PEP 3150 – Statement local namespaces (aka “given” clause)">PEP 3150</a> uses explicit forward reference and name binding schemes that greatly simplify the semantics of the statement, and it does offer the advantage of allowing the definition of arbitrary subexpressions rather than being restricted to a single function or class definition.</p> </section> </section> <section id="references"> <h2><a class="toc-backref" href="#references" role="doc-backlink">References</a></h2> <aside class="footnote-list brackets"> <aside class="footnote brackets" id="id2" role="doc-footnote"> <dt class="label" id="id2">[<a href="#id1">1</a>]</dt> <dd>Start of python-ideas thread: <a class="reference external" href="https://mail.python.org/pipermail/python-ideas/2011-October/012276.html">https://mail.python.org/pipermail/python-ideas/2011-October/012276.html</a></aside> </aside> </section> <section id="copyright"> <h2><a class="toc-backref" href="#copyright" role="doc-backlink">Copyright</a></h2> <p>This document has been placed in the public domain.</p> </section> </section> <hr class="docutils" /> <p>Source: <a class="reference external" href="https://github.com/python/peps/blob/main/peps/pep-0403.rst">https://github.com/python/peps/blob/main/peps/pep-0403.rst</a></p> <p>Last modified: <a class="reference external" href="https://github.com/python/peps/commits/main/peps/pep-0403.rst">2025-02-01 08:59:27 GMT</a></p> </article> <nav id="pep-sidebar"> <h2>Contents</h2> <ul> <li><a class="reference internal" href="#abstract">Abstract</a></li> <li><a class="reference internal" href="#basic-examples">Basic Examples</a></li> <li><a class="reference internal" href="#proposal">Proposal</a><ul> <li><a class="reference internal" href="#syntax-change">Syntax Change</a></li> </ul> </li> <li><a class="reference internal" href="#design-discussion">Design Discussion</a><ul> <li><a class="reference internal" href="#background">Background</a></li> <li><a class="reference internal" href="#relation-to-pep-3150">Relation to PEP 3150</a></li> <li><a class="reference internal" href="#keyword-choice">Keyword Choice</a></li> <li><a class="reference internal" href="#better-debugging-support-for-functions-and-classes-with-short-names">Better Debugging Support for Functions and Classes with Short Names</a></li> <li><a class="reference internal" href="#possible-implementation-strategy">Possible Implementation Strategy</a></li> <li><a class="reference internal" href="#explaining-container-comprehensions-and-generator-expressions">Explaining Container Comprehensions and Generator Expressions</a></li> </ul> </li> <li><a class="reference internal" href="#more-examples">More Examples</a></li> <li><a class="reference internal" href="#reference-implementation">Reference Implementation</a></li> <li><a class="reference internal" href="#acknowledgements">Acknowledgements</a></li> <li><a class="reference internal" href="#rejected-concepts">Rejected Concepts</a><ul> <li><a class="reference internal" href="#omitting-the-decorator-prefix-character">Omitting the decorator prefix character</a></li> <li><a class="reference internal" href="#anonymous-forward-references">Anonymous Forward References</a></li> <li><a class="reference internal" href="#using-a-nested-suite">Using a nested suite</a></li> </ul> </li> <li><a class="reference internal" href="#references">References</a></li> <li><a class="reference internal" href="#copyright">Copyright</a></li> </ul> <br> <a id="source" href="https://github.com/python/peps/blob/main/peps/pep-0403.rst">Page Source (GitHub)</a> </nav> </section> <script src="../_static/colour_scheme.js"></script> <script src="../_static/wrap_tables.js"></script> <script src="../_static/sticky_banner.js"></script> </body> </html>

Pages: 1 2 3 4 5 6 7 8 9 10