CINXE.COM

<!DOCTYPE html> <html lang="en"> <head> <meta charset="utf-8"> <meta name="viewport" content="width=device-width, initial-scale=1.0"> <meta name="color-scheme" content="light dark"> <title>PEP 3150 – Statement local namespaces (aka “given” clause) | peps.python.org</title> <link rel="shortcut icon" href="../_static/py.png"> <link rel="canonical" href="https://peps.python.org/pep-3150/"> <link rel="stylesheet" href="../_static/style.css" type="text/css"> <link rel="stylesheet" href="../_static/mq.css" type="text/css"> <link rel="stylesheet" href="../_static/pygments.css" type="text/css" media="(prefers-color-scheme: light)" id="pyg-light"> <link rel="stylesheet" href="../_static/pygments_dark.css" type="text/css" media="(prefers-color-scheme: dark)" id="pyg-dark"> <link rel="alternate" type="application/rss+xml" title="Latest PEPs" href="https://peps.python.org/peps.rss"> <meta property="og:title" content='PEP 3150 – Statement local namespaces (aka “given” clause) | peps.python.org'> <meta property="og:description" content="This PEP proposes the addition of an optional given clause to several Python statements that do not currently have an associated code suite. This clause will create a statement local namespace for additional names that are accessible in the associated s..."> <meta property="og:type" content="website"> <meta property="og:url" content="https://peps.python.org/pep-3150/"> <meta property="og:site_name" content="Python Enhancement Proposals (PEPs)"> <meta property="og:image" content="https://peps.python.org/_static/og-image.png"> <meta property="og:image:alt" content="Python PEPs"> <meta property="og:image:width" content="200"> <meta property="og:image:height" content="200"> <meta name="description" content="This PEP proposes the addition of an optional given clause to several Python statements that do not currently have an associated code suite. This clause will create a statement local namespace for additional names that are accessible in the associated s..."> <meta name="theme-color" content="#3776ab"> </head> <body> <svg xmlns="http://www.w3.org/2000/svg" style="display: none;"> <symbol id="svg-sun-half" viewBox="0 0 24 24" pointer-events="all"> <title>Following system colour scheme</title> <svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"> <circle cx="12" cy="12" r="9"></circle> <path d="M12 3v18m0-12l4.65-4.65M12 14.3l7.37-7.37M12 19.6l8.85-8.85"></path> </svg> </symbol> <symbol id="svg-moon" viewBox="0 0 24 24" pointer-events="all"> <title>Selected dark colour scheme</title> <svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"> <path stroke="none" d="M0 0h24v24H0z" fill="none"></path> <path d="M12 3c.132 0 .263 0 .393 0a7.5 7.5 0 0 0 7.92 12.446a9 9 0 1 1 -8.313 -12.454z"></path> </svg> </symbol> <symbol id="svg-sun" viewBox="0 0 24 24" pointer-events="all"> <title>Selected light colour scheme</title> <svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"> <circle cx="12" cy="12" r="5"></circle> <line x1="12" y1="1" x2="12" y2="3"></line> <line x1="12" y1="21" x2="12" y2="23"></line> <line x1="4.22" y1="4.22" x2="5.64" y2="5.64"></line> <line x1="18.36" y1="18.36" x2="19.78" y2="19.78"></line> <line x1="1" y1="12" x2="3" y2="12"></line> <line x1="21" y1="12" x2="23" y2="12"></line> <line x1="4.22" y1="19.78" x2="5.64" y2="18.36"></line> <line x1="18.36" y1="5.64" x2="19.78" y2="4.22"></line> </svg> </symbol> </svg> <script> document.documentElement.dataset.colour_scheme = localStorage.getItem("colour_scheme") || "auto" </script> <section id="pep-page-section"> <header> <h1>Python Enhancement Proposals</h1> <ul class="breadcrumbs"> <li><a href="https://www.python.org/" title="The Python Programming Language">Python</a> » </li> <li><a href="../pep-0000/">PEP Index</a> » </li> <li>PEP 3150</li> </ul> <button id="colour-scheme-cycler" onClick="setColourScheme(nextColourScheme())"> <svg aria-hidden="true" class="colour-scheme-icon-when-auto"><use href="#svg-sun-half"></use></svg> <svg aria-hidden="true" class="colour-scheme-icon-when-dark"><use href="#svg-moon"></use></svg> <svg aria-hidden="true" class="colour-scheme-icon-when-light"><use href="#svg-sun"></use></svg> Toggle light / dark / auto colour theme </button> </header> <article> <section id="pep-content"> <h1 class="page-title">PEP 3150 – Statement local namespaces (aka “given” clause)</h1> <dl class="rfc2822 field-list simple"> <dt class="field-odd">Author:</dt> <dd class="field-odd">Alyssa Coghlan <ncoghlan at gmail.com></dd> <dt class="field-even">Status:</dt> <dd class="field-even"><abbr title="Inactive draft that may be taken up again at a later time">Deferred</abbr></dd> <dt class="field-odd">Type:</dt> <dd class="field-odd"><abbr title="Normative PEP with a new feature for Python, implementation change for CPython or interoperability standard for the ecosystem">Standards Track</abbr></dd> <dt class="field-even">Created:</dt> <dd class="field-even">09-Jul-2010</dd> <dt class="field-odd">Python-Version:</dt> <dd class="field-odd">3.4</dd> <dt class="field-even">Post-History:</dt> <dd class="field-even">14-Jul-2010, 21-Apr-2011, 13-Jun-2011</dd> </dl> <hr class="docutils" /> <section id="contents"> <details><summary>Table of Contents</summary><ul class="simple"> <li><a class="reference internal" href="#abstract">Abstract</a></li> <li><a class="reference internal" href="#proposal">Proposal</a><ul> <li><a class="reference internal" href="#semantics">Semantics</a></li> <li><a class="reference internal" href="#syntax-change">Syntax Change</a></li> <li><a class="reference internal" href="#new-pep-8-guidelines">New PEP 8 Guidelines</a></li> </ul> </li> <li><a class="reference internal" href="#rationale">Rationale</a></li> <li><a class="reference internal" href="#design-discussion">Design Discussion</a><ul> <li><a class="reference internal" href="#keyword-choice">Keyword Choice</a></li> <li><a class="reference internal" href="#relation-to-pep-403">Relation to PEP 403</a></li> <li><a class="reference internal" href="#explaining-container-comprehensions-and-generator-expressions">Explaining Container Comprehensions and Generator Expressions</a></li> <li><a class="reference internal" href="#explaining-decorator-clause-evaluation-and-application">Explaining Decorator Clause Evaluation and Application</a></li> <li><a class="reference internal" href="#anticipated-objections">Anticipated Objections</a><ul> <li><a class="reference internal" href="#two-ways-to-do-it">Two Ways To Do It</a></li> <li><a class="reference internal" href="#out-of-order-execution">Out of Order Execution</a></li> <li><a class="reference internal" href="#harmful-to-introspection">Harmful to Introspection</a></li> <li><a class="reference internal" href="#lack-of-real-world-impact-assessment">Lack of Real World Impact Assessment</a></li> </ul> </li> </ul> </li> <li><a class="reference internal" href="#open-questions">Open Questions</a><ul> <li><a class="reference internal" href="#syntax-for-forward-references">Syntax for Forward References</a></li> <li><a class="reference internal" href="#handling-of-nonlocal-and-global">Handling of <code class="docutils literal notranslate">nonlocal</code> and <code class="docutils literal notranslate">global</code></a></li> <li><a class="reference internal" href="#handling-of-break-and-continue">Handling of <code class="docutils literal notranslate">break</code> and <code class="docutils literal notranslate">continue</code></a></li> <li><a class="reference internal" href="#handling-of-return-and-yield">Handling of <code class="docutils literal notranslate">return</code> and <code class="docutils literal notranslate">yield</code></a></li> </ul> </li> <li><a class="reference internal" href="#examples">Examples</a></li> <li><a class="reference internal" href="#possible-additions">Possible Additions</a></li> <li><a class="reference internal" href="#rejected-alternatives">Rejected Alternatives</a></li> <li><a class="reference internal" href="#reference-implementation">Reference Implementation</a></li> <li><a class="reference internal" href="#to-do">TO-DO</a></li> <li><a class="reference internal" href="#references">References</a></li> <li><a class="reference internal" href="#copyright">Copyright</a></li> </ul> </details></section> <section id="abstract"> <h2><a class="toc-backref" href="#abstract" role="doc-backlink">Abstract</a></h2> This PEP proposes the addition of an optional <code class="docutils literal notranslate">given</code> clause to several Python statements that do not currently have an associated code suite. This clause will create a statement local namespace for additional names that are accessible in the associated statement, but do not become part of the containing namespace. Adoption of a new symbol, <code class="docutils literal notranslate">?</code>, is proposed to denote a forward reference to the namespace created by running the associated code suite. It will be a reference to a <code class="docutils literal notranslate">types.SimpleNamespace</code> object. The primary motivation is to enable a more declarative style of programming, where the operation to be performed is presented to the reader first, and the details of the necessary subcalculations are presented in the following indented suite. As a key example, this would elevate ordinary assignment statements to be on par with <code class="docutils literal notranslate">class</code> and <code class="docutils literal notranslate">def</code> statements where the name of the item to be defined is presented to the reader in advance of the details of how the value of that item is calculated. It also allows named functions to be used in a “multi-line lambda” fashion, where the name is used solely as a placeholder in the current expression and then defined in the following suite. A secondary motivation is to simplify interim calculations in module and class level code without polluting the resulting namespaces. The intent is that the relationship between a given clause and a separate function definition that performs the specified operation will be similar to the existing relationship between an explicit while loop and a generator that produces the same sequence of operations as that while loop. The specific proposal in this PEP has been informed by various explorations of this and related concepts over the years (e.g. <a class="footnote-reference brackets" href="#id9" id="id1">[1]</a>, <a class="footnote-reference brackets" href="#id10" id="id2">[2]</a>, <a class="footnote-reference brackets" href="#id11" id="id3">[3]</a>, <a class="footnote-reference brackets" href="#id13" id="id4">[6]</a>, <a class="footnote-reference brackets" href="#id15" id="id5">[8]</a>), and is inspired to some degree by the <code class="docutils literal notranslate">where</code> and <code class="docutils literal notranslate">let</code> clauses in Haskell. It avoids some problems that have been identified in past proposals, but has not yet itself been subject to the test of implementation. </section> <section id="proposal"> <h2><a class="toc-backref" href="#proposal" role="doc-backlink">Proposal</a></h2> This PEP proposes the addition of an optional <code class="docutils literal notranslate">given</code> clause to the syntax for simple statements which may contain an expression, or may substitute for such a statement for purely syntactic purposes. The current list of simple statements that would be affected by this addition is as follows: <ul class="simple"> <li>expression statement</li> <li>assignment statement</li> <li>augmented assignment statement</li> <li>del statement</li> <li>return statement</li> <li>yield statement</li> <li>raise statement</li> <li>assert statement</li> <li>pass statement</li> </ul> The <code class="docutils literal notranslate">given</code> clause would allow subexpressions to be referenced by name in the header line, with the actual definitions following in the indented clause. As a simple example: <div class="highlight-default notranslate"><div class="highlight"><pre>sorted_data = sorted(data, key=?.sort_key) given: def sort_key(item): return item.attr1, item.attr2 </pre></div> </div> The new symbol <code class="docutils literal notranslate">?</code> is used to refer to the given namespace. It would be a <code class="docutils literal notranslate">types.SimpleNamespace</code> instance, so <code class="docutils literal notranslate">?.sort_key</code> functions as a forward reference to a name defined in the <code class="docutils literal notranslate">given</code> clause. A docstring would be permitted in the given clause, and would be attached to the result namespace as its <code class="docutils literal notranslate">__doc__</code> attribute. The <code class="docutils literal notranslate">pass</code> statement is included to provide a consistent way to skip inclusion of a meaningful expression in the header line. While this is not an intended use case, it isn’t one that can be prevented as multiple alternatives (such as <code class="docutils literal notranslate">...</code> and <code class="docutils literal notranslate">()</code>) remain available even if <code class="docutils literal notranslate">pass</code> itself is disallowed. The body of the given clause will execute in a new scope, using normal function closure semantics. To support early binding of loop variables and global references, as well as to allow access to other names defined at class scope, the <code class="docutils literal notranslate">given</code> clause will also allow explicit binding operations in the header line: <div class="highlight-default notranslate"><div class="highlight"><pre># Explicit early binding via given clause seq = [] for i in range(10): seq.append(?.f) given i=i in: def f(): return i assert [f() for f in seq] == list(range(10)) </pre></div> </div> <section id="semantics"> <h3><a class="toc-backref" href="#semantics" role="doc-backlink">Semantics</a></h3> The following statement: <div class="highlight-default notranslate"><div class="highlight"><pre>op(?.f, ?.g) given bound_a=a, bound_b=b in: def f(): return bound_a + bound_b def g(): return bound_a - bound_b </pre></div> </div> Would be roughly equivalent to the following code (<code class="docutils literal notranslate">__var</code> denotes a hidden compiler variable or simply an entry on the interpreter stack): <div class="highlight-default notranslate"><div class="highlight"><pre>__arg1 = a __arg2 = b def __scope(bound_a, bound_b): def f(): return bound_a + bound_b def g(): return bound_a - bound_b return types.SimpleNamespace(**locals()) __ref = __scope(__arg1, __arg2) __ref.__doc__ = __scope.__doc__ op(__ref.f, __ref.g) </pre></div> </div> A <code class="docutils literal notranslate">given</code> clause is essentially a nested function which is created and then immediately executed. Unless explicitly passed in, names are looked up using normal scoping rules, and thus names defined at class scope will not be visible. Names declared as forward references are returned and used in the header statement, without being bound locally in the surrounding namespace. </section> <section id="syntax-change"> <h3><a class="toc-backref" href="#syntax-change" role="doc-backlink">Syntax Change</a></h3> Current: <div class="highlight-default notranslate"><div class="highlight"><pre>expr_stmt: testlist_star_expr (augassign (yield_expr|testlist) | ('=' (yield_expr|testlist_star_expr))*) del_stmt: 'del' exprlist pass_stmt: 'pass' return_stmt: 'return' [testlist] yield_stmt: yield_expr raise_stmt: 'raise' [test ['from' test]] assert_stmt: 'assert' test [',' test] </pre></div> </div> New: <div class="highlight-default notranslate"><div class="highlight"><pre>expr_stmt: testlist_star_expr (augassign (yield_expr|testlist) | ('=' (yield_expr|testlist_star_expr))*) [given_clause] del_stmt: 'del' exprlist [given_clause] pass_stmt: 'pass' [given_clause] return_stmt: 'return' [testlist] [given_clause] yield_stmt: yield_expr [given_clause] raise_stmt: 'raise' [test ['from' test]] [given_clause] assert_stmt: 'assert' test [',' test] [given_clause] given_clause: "given" [(NAME '=' test)+ "in"]":" suite </pre></div> </div> (Note that <code class="docutils literal notranslate">expr_stmt</code> in the grammar is a slight misnomer, as it covers assignment and augmented assignment in addition to simple expression statements) <div class="admonition note"> Note These proposed grammar changes don’t yet cover the forward reference expression syntax for accessing names defined in the statement local namespace. </div> The new clause is added as an optional element of the existing statements rather than as a new kind of compound statement in order to avoid creating an ambiguity in the grammar. It is applied only to the specific elements listed so that nonsense like the following is disallowed: <div class="highlight-default notranslate"><div class="highlight"><pre>break given: a = b = 1 import sys given: a = b = 1 </pre></div> </div> However, the precise Grammar change described above is inadequate, as it creates problems for the definition of simple_stmt (which allows chaining of multiple single line statements with “;” rather than “\n”). So the above syntax change should instead be taken as a statement of intent. Any actual proposal would need to resolve the simple_stmt parsing problem before it could be seriously considered. This would likely require a non-trivial restructuring of the grammar, breaking up small_stmt and flow_stmt to separate the statements that potentially contain arbitrary subexpressions and then allowing a single one of those statements with a <code class="docutils literal notranslate">given</code> clause at the simple_stmt level. Something along the lines of: <div class="highlight-default notranslate"><div class="highlight"><pre>stmt: simple_stmt | given_stmt | compound_stmt simple_stmt: small_stmt (';' (small_stmt | subexpr_stmt))* [';'] NEWLINE small_stmt: (pass_stmt | flow_stmt | import_stmt | global_stmt | nonlocal_stmt) flow_stmt: break_stmt | continue_stmt given_stmt: subexpr_stmt (given_clause | (';' (small_stmt | subexpr_stmt))* [';']) NEWLINE subexpr_stmt: expr_stmt | del_stmt | flow_subexpr_stmt | assert_stmt flow_subexpr_stmt: return_stmt | raise_stmt | yield_stmt given_clause: "given" (NAME '=' test)* ":" suite </pre></div> </div> For reference, here are the current definitions at that level: <div class="highlight-default notranslate"><div class="highlight"><pre>stmt: simple_stmt | compound_stmt simple_stmt: small_stmt (';' small_stmt)* [';'] NEWLINE small_stmt: (expr_stmt | del_stmt | pass_stmt | flow_stmt | import_stmt | global_stmt | nonlocal_stmt | assert_stmt) flow_stmt: break_stmt | continue_stmt | return_stmt | raise_stmt | yield_stmt </pre></div> </div> In addition to the above changes, the definition of <code class="docutils literal notranslate">atom</code> would be changed to also allow <code class="docutils literal notranslate">?</code>. The restriction of this usage to statements with an associated <code class="docutils literal notranslate">given</code> clause would be handled by a later stage of the compilation process (likely AST construction, which already enforces other restrictions where the grammar is overly permissive in order to simplify the initial parsing step). </section> <section id="new-pep-8-guidelines"> <h3><a class="toc-backref" href="#new-pep-8-guidelines" role="doc-backlink">New PEP 8 Guidelines</a></h3> As discussed on python-ideas (<a class="footnote-reference brackets" href="#id14" id="id6">[7]</a>, <a class="footnote-reference brackets" href="#id16" id="id7">[9]</a>) new <a class="pep reference internal" href="../pep-0008/" title="PEP 8 – Style Guide for Python Code">PEP 8</a> guidelines would also need to be developed to provide appropriate direction on when to use the <code class="docutils literal notranslate">given</code> clause over ordinary variable assignments. Based on the similar guidelines already present for <code class="docutils literal notranslate">try</code> statements, this PEP proposes the following additions for <code class="docutils literal notranslate">given</code> statements to the “Programming Conventions” section of <a class="pep reference internal" href="../pep-0008/" title="PEP 8 – Style Guide for Python Code">PEP 8</a>: <ul class="simple"> <li>for code that could reasonably be factored out into a separate function, but is not currently reused anywhere, consider using a <code class="docutils literal notranslate">given</code> clause. This clearly indicates which variables are being used only to define subcomponents of another statement rather than to hold algorithm or application state. This is an especially useful technique when passing multi-line functions to operations which take callable arguments.</li> <li>keep <code class="docutils literal notranslate">given</code> clauses concise. If they become unwieldy, either break them up into multiple steps or else move the details into a separate function.</li> </ul> </section> </section> <section id="rationale"> <h2><a class="toc-backref" href="#rationale" role="doc-backlink">Rationale</a></h2> Function and class statements in Python have a unique property relative to ordinary assignment statements: to some degree, they are declarative. They present the reader of the code with some critical information about a name that is about to be defined, before proceeding on with the details of the actual definition in the function or class body. The name of the object being declared is the first thing stated after the keyword. Other important information is also given the honour of preceding the implementation details: <ul class="simple"> <li>decorators (which can greatly affect the behaviour of the created object, and were placed ahead of even the keyword and name as a matter of practicality more so than aesthetics)</li> <li>the docstring (on the first line immediately following the header line)</li> <li>parameters, default values and annotations for function definitions</li> <li>parent classes, metaclass and optionally other details (depending on the metaclass) for class definitions</li> </ul> This PEP proposes to make a similar declarative style available for arbitrary assignment operations, by permitting the inclusion of a “given” suite following any simple assignment statement: <div class="highlight-default notranslate"><div class="highlight"><pre>TARGET = [TARGET2 = ... TARGETN =] EXPR given: SUITE </pre></div> </div> By convention, code in the body of the suite should be oriented solely towards correctly defining the assignment operation carried out in the header line. The header line operation should also be adequately descriptive (e.g. through appropriate choices of variable names) to give a reader a reasonable idea of the purpose of the operation without reading the body of the suite. However, while they are the initial motivating use case, limiting this feature solely to simple assignments would be overly restrictive. Once the feature is defined at all, it would be quite arbitrary to prevent its use for augmented assignments, return statements, yield expressions, comprehensions and arbitrary expressions that may modify the application state. The <code class="docutils literal notranslate">given</code> clause may also function as a more readable alternative to some uses of lambda expressions and similar constructs when passing one-off functions to operations like <code class="docutils literal notranslate">sorted()</code> or in callback based event-driven programming. In module and class level code, the <code class="docutils literal notranslate">given</code> clause will serve as a clear and reliable replacement for usage of the <code class="docutils literal notranslate">del</code> statement to keep interim working variables from polluting the resulting namespace. One potentially useful way to think of the proposed clause is as a middle ground between conventional in-line code and separation of an operation out into a dedicated function, just as an inline while loop may eventually be factored out into a dedicated generator. </section> <section id="design-discussion"> <h2><a class="toc-backref" href="#design-discussion" role="doc-backlink">Design Discussion</a></h2> <section id="keyword-choice"> <h3><a class="toc-backref" href="#keyword-choice" role="doc-backlink">Keyword Choice</a></h3> This proposal initially used <code class="docutils literal notranslate">where</code> based on the name of a similar construct in Haskell. However, it has been pointed out that there are existing Python libraries (such as Numpy <a class="footnote-reference brackets" href="#id12" id="id8">[4]</a>) that already use <code class="docutils literal notranslate">where</code> in the SQL query condition sense, making that keyword choice potentially confusing. While <code class="docutils literal notranslate">given</code> may also be used as a variable name (and hence would be deprecated using the usual <code class="docutils literal notranslate">__future__</code> dance for introducing new keywords), it is associated much more strongly with the desired “here are some extra variables this expression may use” semantics for the new clause. Reusing the <code class="docutils literal notranslate">with</code> keyword has also been proposed. This has the advantage of avoiding the addition of a new keyword, but also has a high potential for confusion as the <code class="docutils literal notranslate">with</code> clause and <code class="docutils literal notranslate">with</code> statement would look similar but do completely different things. That way lies C++ and Perl :) </section> <section id="relation-to-pep-403"> <h3><a class="toc-backref" href="#relation-to-pep-403" role="doc-backlink">Relation to PEP 403</a></h3> <a class="pep reference internal" href="../pep-0403/" title="PEP 403 – General purpose decorator clause (aka “@in” clause)">PEP 403</a> (General Purpose Decorator Clause) attempts to achieve the main goals of this PEP using a less radical language change inspired by the existing decorator syntax. Despite having the same author, the two PEPs are in direct competition with each other. <a class="pep reference internal" href="../pep-0403/" title="PEP 403 – General purpose decorator clause (aka “@in” clause)">PEP 403</a> represents a minimalist approach that attempts to achieve useful functionality with a minimum of change from the status quo. This PEP instead aims for a more flexible standalone statement design, which requires a larger degree of change to the language. Note that where <a class="pep reference internal" href="../pep-0403/" title="PEP 403 – General purpose decorator clause (aka “@in” clause)">PEP 403</a> is better suited to explaining the behaviour of generator expressions correctly, this PEP is better able to explain the behaviour of decorator clauses in general. Both PEPs support adequate explanations for the semantics of container comprehensions. </section> <section id="explaining-container-comprehensions-and-generator-expressions"> <h3><a class="toc-backref" href="#explaining-container-comprehensions-and-generator-expressions" role="doc-backlink">Explaining Container Comprehensions and Generator Expressions</a></h3> One interesting feature of the proposed construct is that it can be used as a primitive to explain the scoping and execution order semantics of container comprehensions: <div class="highlight-default notranslate"><div class="highlight"><pre>seq2 = [x for x in y if q(x) for y in seq if p(y)] # would be equivalent to seq2 = ?.result given seq=seq: result = [] for y in seq: if p(y): for x in y: if q(x): result.append(x) </pre></div> </div> The important point in this expansion is that it explains why comprehensions appear to misbehave at class scope: only the outermost iterator is evaluated at class scope, while all predicates, nested iterators and value expressions are evaluated inside a nested scope. Not that, unlike <a class="pep reference internal" href="../pep-0403/" title="PEP 403 – General purpose decorator clause (aka “@in” clause)">PEP 403</a>, the current version of this PEP cannot provide a precisely equivalent expansion for a generator expression. The closest it can get is to define an additional level of scoping: <div class="highlight-default notranslate"><div class="highlight"><pre>seq2 = ?.g(seq) given: def g(seq): for y in seq: if p(y): for x in y: if q(x): yield x </pre></div> </div> This limitation could be remedied by permitting the given clause to be a generator function, in which case ? would refer to a generator-iterator object rather than a simple namespace: <div class="highlight-default notranslate"><div class="highlight"><pre>seq2 = ? given seq=seq in: for y in seq: if p(y): for x in y: if q(x): yield x </pre></div> </div> However, this would make the meaning of “?” quite ambiguous, even more so than is already the case for the meaning of <code class="docutils literal notranslate">def</code> statements (which will usually have a docstring indicating whether or not a function definition is actually a generator) </section> <section id="explaining-decorator-clause-evaluation-and-application"> <h3><a class="toc-backref" href="#explaining-decorator-clause-evaluation-and-application" role="doc-backlink">Explaining Decorator Clause Evaluation and Application</a></h3> The standard explanation of decorator clause evaluation and application has to deal with the idea of hidden compiler variables in order to show steps in their order of execution. The given statement allows a decorated function definition like: <div class="highlight-default notranslate"><div class="highlight"><pre>@classmethod def classname(cls): return cls.__name__ </pre></div> </div> To instead be explained as roughly equivalent to: <div class="highlight-default notranslate"><div class="highlight"><pre>classname = .d1(classname) given: d1 = classmethod def classname(cls): return cls.__name__ </pre></div> </div> </section> <section id="anticipated-objections"> <h3><a class="toc-backref" href="#anticipated-objections" role="doc-backlink">Anticipated Objections</a></h3> <section id="two-ways-to-do-it"> <h4><a class="toc-backref" href="#two-ways-to-do-it" role="doc-backlink">Two Ways To Do It</a></h4> A lot of code may now be written with values defined either before the expression where they are used or afterwards in a <code class="docutils literal notranslate">given</code> clause, creating two ways to do it, perhaps without an obvious way of choosing between them. On reflection, I feel this is a misapplication of the “one obvious way” aphorism. Python already offers lots of ways to write code. We can use a for loop or a while loop, a functional style or an imperative style or an object oriented style. The language, in general, is designed to let people write code that matches the way they think. Since different people think differently, the way they write their code will change accordingly. Such stylistic questions in a code base are rightly left to the development group responsible for that code. When does an expression get so complicated that the subexpressions should be taken out and assigned to variables, even though those variables are only going to be used once? When should an inline while loop be replaced with a generator that implements the same logic? Opinions differ, and that’s OK. However, explicit <a class="pep reference internal" href="../pep-0008/" title="PEP 8 – Style Guide for Python Code">PEP 8</a> guidance will be needed for CPython and the standard library, and that is discussed in the proposal above. </section> <section id="out-of-order-execution"> <h4><a class="toc-backref" href="#out-of-order-execution" role="doc-backlink">Out of Order Execution</a></h4> The <code class="docutils literal notranslate">given</code> clause makes execution jump around a little strangely, as the body of the <code class="docutils literal notranslate">given</code> clause is executed before the simple statement in the clause header. The closest any other part of Python comes to this is the out of order evaluation in list comprehensions, generator expressions and conditional expressions and the delayed application of decorator functions to the function they decorate (the decorator expressions themselves are executed in the order they are written). While this is true, the syntax is intended for cases where people are themselves thinking about a problem out of sequence (at least as far as the language is concerned). As an example of this, consider the following thought in the mind of a Python user: <blockquote> <div>I want to sort the items in this sequence according to the values of attr1 and attr2 on each item.</div></blockquote> If they’re comfortable with Python’s <code class="docutils literal notranslate">lambda</code> expressions, then they might choose to write it like this: <div class="highlight-default notranslate"><div class="highlight"><pre>sorted_list = sorted(original, key=(lambda v: v.attr1, v.attr2)) </pre></div> </div> That gets the job done, but it hardly reaches the standard of <code class="docutils literal notranslate">executable pseudocode</code> that fits Python’s reputation. If they don’t like <code class="docutils literal notranslate">lambda</code> specifically, the <code class="docutils literal notranslate">operator</code> module offers an alternative that still allows the key function to be defined inline: <div class="highlight-default notranslate"><div class="highlight"><pre>sorted_list = sorted(original, key=operator.attrgetter(v. 'attr1', 'attr2')) </pre></div> </div> Again, it gets the job done, but even the most generous of readers would not consider that to be “executable pseudocode”. If they think both of the above options are ugly and confusing, or they need logic in their key function that can’t be expressed as an expression (such as catching an exception), then Python currently forces them to reverse the order of their original thought and define the sorting criteria first: <div class="highlight-default notranslate"><div class="highlight"><pre>def sort_key(item): return item.attr1, item.attr2 sorted_list = sorted(original, key=sort_key) </pre></div> </div> “Just define a function” has been the rote response to requests for multi-line lambda support for years. As with the above options, it gets the job done, but it really does represent a break between what the user is thinking and what the language allows them to express. I believe the proposal in this PEP would finally let Python get close to the “executable pseudocode” bar for the kind of thought expressed above: <div class="highlight-default notranslate"><div class="highlight"><pre>sorted_list = sorted(original, key=?.key) given: def key(item): return item.attr1, item.attr2 </pre></div> </div> Everything is in the same order as it was in the user’s original thought, and they don’t even need to come up with a name for the sorting criteria: it is possible to reuse the keyword argument name directly. A possible enhancement to those proposal would be to provide a convenient shorthand syntax to say “use the given clause contents as keyword arguments”. Even without dedicated syntax, that can be written simply as <code class="docutils literal notranslate">**vars(?)</code>. </section> <section id="harmful-to-introspection"> <h4><a class="toc-backref" href="#harmful-to-introspection" role="doc-backlink">Harmful to Introspection</a></h4> Poking around in module and class internals is an invaluable tool for white-box testing and interactive debugging. The <code class="docutils literal notranslate">given</code> clause will be quite effective at preventing access to temporary state used during calculations (although no more so than current usage of <code class="docutils literal notranslate">del</code> statements in that regard). While this is a valid concern, design for testability is an issue that cuts across many aspects of programming. If a component needs to be tested independently, then a <code class="docutils literal notranslate">given</code> statement should be refactored in to separate statements so that information is exposed to the test suite. This isn’t significantly different from refactoring an operation hidden inside a function or generator out into its own function purely to allow it to be tested in isolation. </section> <section id="lack-of-real-world-impact-assessment"> <h4><a class="toc-backref" href="#lack-of-real-world-impact-assessment" role="doc-backlink">Lack of Real World Impact Assessment</a></h4> The examples in the current PEP are almost all relatively small “toy” examples. The proposal in this PEP needs to be subjected to the test of application to a large code base (such as the standard library or a large Twisted application) in a search for examples where the readability of real world code is genuinely enhanced. This is more of a deficiency in the PEP rather than the idea, though. If it wasn’t a real world problem, we wouldn’t get so many complaints about the lack of multi-line lambda support and Ruby’s block construct probably wouldn’t be quite so popular. </section> </section> </section> <section id="open-questions"> <h2><a class="toc-backref" href="#open-questions" role="doc-backlink">Open Questions</a></h2> <section id="syntax-for-forward-references"> <h3><a class="toc-backref" href="#syntax-for-forward-references" role="doc-backlink">Syntax for Forward References</a></h3> The <code class="docutils literal notranslate">?</code> symbol is proposed for forward references to the given namespace as it is short, currently unused and suggests “there’s something missing here that will be filled in later”. The proposal in the PEP doesn’t neatly parallel any existing Python feature, so reusing an already used symbol has been deliberately avoided. </section> <section id="handling-of-nonlocal-and-global"> <h3><a class="toc-backref" href="#handling-of-nonlocal-and-global" role="doc-backlink">Handling of <code class="docutils literal notranslate">nonlocal</code> and <code class="docutils literal notranslate">global</code></a></h3> <code class="docutils literal notranslate">nonlocal</code> and <code class="docutils literal notranslate">global</code> are explicitly disallowed in the <code class="docutils literal notranslate">given</code> clause suite and will be syntax errors if they occur. They will work normally if they appear within a <code class="docutils literal notranslate">def</code> statement within that suite. Alternatively, they could be defined as operating as if the anonymous functions were defined as in the expansion above. </section> <section id="handling-of-break-and-continue"> <h3><a class="toc-backref" href="#handling-of-break-and-continue" role="doc-backlink">Handling of <code class="docutils literal notranslate">break</code> and <code class="docutils literal notranslate">continue</code></a></h3> <code class="docutils literal notranslate">break</code> and <code class="docutils literal notranslate">continue</code> will operate as if the anonymous functions were defined as in the expansion above. They will be syntax errors if they occur in the <code class="docutils literal notranslate">given</code> clause suite but will work normally if they appear within a <code class="docutils literal notranslate">for</code> or <code class="docutils literal notranslate">while</code> loop as part of that suite. </section> <section id="handling-of-return-and-yield"> <h3><a class="toc-backref" href="#handling-of-return-and-yield" role="doc-backlink">Handling of <code class="docutils literal notranslate">return</code> and <code class="docutils literal notranslate">yield</code></a></h3> <code class="docutils literal notranslate">return</code> and <code class="docutils literal notranslate">yield</code> are explicitly disallowed in the <code class="docutils literal notranslate">given</code> clause suite and will be syntax errors if they occur. They will work normally if they appear within a <code class="docutils literal notranslate">def</code> statement within that suite. </section> </section> <section id="examples"> <h2><a class="toc-backref" href="#examples" role="doc-backlink">Examples</a></h2> Defining callbacks for event driven programming: <div class="highlight-default notranslate"><div class="highlight"><pre># Current Python (definition before use) def cb(sock): # Do something with socket def eb(exc): logging.exception( "Failed connecting to %s:%s", host, port) loop.create_connection((host, port), cb, eb) given: # Becomes: loop.create_connection((host, port), ?.cb, ?.eb) given: def cb(sock): # Do something with socket def eb(exc): logging.exception( "Failed connecting to %s:%s", host, port) </pre></div> </div> Defining “one-off” classes which typically only have a single instance: <div class="highlight-default notranslate"><div class="highlight"><pre># Current Python (instantiation after definition) class public_name(): ... # However many lines public_name = public_name(*params) # Current Python (custom decorator) def singleton(*args, **kwds): def decorator(cls): return cls(*args, **kwds) return decorator @singleton(*params) class public_name(): ... # However many lines # Becomes: public_name = ?.MeaningfulClassName(*params) given: class MeaningfulClassName(): ... # Should trawl the stdlib for an example of doing this </pre></div> </div> Calculating attributes without polluting the local namespace (from os.py): <div class="highlight-default notranslate"><div class="highlight"><pre># Current Python (manual namespace cleanup) def _createenviron(): ... # 27 line function environ = _createenviron() del _createenviron # Becomes: environ = ?._createenviron() given: def _createenviron(): ... # 27 line function </pre></div> </div> Replacing default argument hack (from functools.lru_cache): <div class="highlight-default notranslate"><div class="highlight"><pre># Current Python (default argument hack) def decorating_function(user_function, tuple=tuple, sorted=sorted, len=len, KeyError=KeyError): ... # 60 line function return decorating_function # Becomes: return ?.decorating_function given: # Cell variables rather than locals, but should give similar speedup tuple, sorted, len, KeyError = tuple, sorted, len, KeyError def decorating_function(user_function): ... # 60 line function # This example also nicely makes it clear that there is nothing in the # function after the nested function definition. Due to additional # nested functions, that isn't entirely clear in the current code. </pre></div> </div> </section> <section id="possible-additions"> <h2><a class="toc-backref" href="#possible-additions" role="doc-backlink">Possible Additions</a></h2> <ul class="simple"> <li>The current proposal allows the addition of a <code class="docutils literal notranslate">given</code> clause only for simple statements. Extending the idea to allow the use of compound statements would be quite possible (by appending the given clause as an independent suite at the end), but doing so raises serious readability concerns (as values defined in the <code class="docutils literal notranslate">given</code> clause may be used well before they are defined, exactly the kind of readability trap that other features like decorators and <code class="docutils literal notranslate">with</code> statements are designed to eliminate)</li> <li>The “explicit early binding” variant may be applicable to the discussions on python-ideas on how to eliminate the default argument hack. A <code class="docutils literal notranslate">given</code> clause in the header line for functions (after the return type annotation) may be the answer to that question.</li> </ul> </section> <section id="rejected-alternatives"> <h2><a class="toc-backref" href="#rejected-alternatives" role="doc-backlink">Rejected Alternatives</a></h2> <ul class="simple"> <li>An earlier version of this PEP allowed implicit forward references to the names in the trailing suite, and also used implicit early binding semantics. Both of these ideas substantially complicated the proposal without providing a sufficient increase in expressive power. The current proposing with explicit forward references and early binding brings the new construct into line with existing scoping semantics, greatly improving the chances the idea can actually be implemented.</li> <li>In addition to the proposals made here, there have also been suggestions of two suite “in-order” variants which provide the limited scoping of names without supporting out-of-order execution. I believe these suggestions largely miss the point of what people are complaining about when they ask for multi-line lambda support - it isn’t that coming up with a name for the subexpression is especially difficult, it’s that naming the function before the statement that uses it means the code no longer matches the way the developer thinks about the problem at hand.</li> <li>I’ve made some unpublished attempts to allow direct references to the closure implicitly created by the <code class="docutils literal notranslate">given</code> clause, while still retaining the general structure of the syntax as defined in this PEP (For example, allowing a subexpression like <code class="docutils literal notranslate">?given</code> or <code class="docutils literal notranslate">:given</code> to be used in expressions to indicate a direct reference to the implied closure, thus preventing it from being called automatically to create the local namespace). All such attempts have appeared unattractive and confusing compared to the simpler decorator-inspired proposal in <a class="pep reference internal" href="../pep-0403/" title="PEP 403 – General purpose decorator clause (aka “@in” clause)">PEP 403</a>.</li> </ul> </section> <section id="reference-implementation"> <h2><a class="toc-backref" href="#reference-implementation" role="doc-backlink">Reference Implementation</a></h2> None as yet. If you want a crash course in Python namespace semantics and code compilation, feel free to try ;) </section> <section id="to-do"> <h2><a class="toc-backref" href="#to-do" role="doc-backlink">TO-DO</a></h2> <ul class="simple"> <li>Mention <a class="pep reference internal" href="../pep-0359/" title="PEP 359 – The “make” Statement">PEP 359</a> and possible uses for locals() in the <code class="docutils literal notranslate">given</code> clause</li> <li>Figure out if this can be used internally to make the implementation of zero-argument super() calls less awful</li> </ul> </section> <section id="references"> <h2><a class="toc-backref" href="#references" role="doc-backlink">References</a></h2> <aside class="footnote-list brackets"> <aside class="footnote brackets" id="id9" role="doc-footnote"> <dt class="label" id="id9">[<a href="#id1">1</a>]</dt> <dd><a class="reference external" href="https://mail.python.org/pipermail/python-ideas/2010-June/007476.html">Explicitation lines in Python</a></aside> <aside class="footnote brackets" id="id10" role="doc-footnote"> <dt class="label" id="id10">[<a href="#id2">2</a>]</dt> <dd><a class="reference external" href="https://mail.python.org/pipermail/python-ideas/2010-July/007584.html">‘where’ statement in Python</a></aside> <aside class="footnote brackets" id="id11" role="doc-footnote"> <dt class="label" id="id11">[<a href="#id3">3</a>]</dt> <dd><a class="reference external" href="https://mail.python.org/pipermail/python-ideas/2009-July/005132.html">Where-statement (Proposal for function expressions)</a></aside> <aside class="footnote brackets" id="id12" role="doc-footnote"> <dt class="label" id="id12">[<a href="#id8">4</a>]</dt> <dd><a class="reference external" href="https://mail.python.org/pipermail/python-ideas/2010-July/007596.html">Name conflict with NumPy for ‘where’ keyword choice</a></aside> <aside class="footnote brackets" id="id13" role="doc-footnote"> <dt class="label" id="id13">[<a href="#id4">6</a>]</dt> <dd><a class="reference external" href="https://mail.python.org/pipermail/python-ideas/2011-April/009863.html">Assignments in list/generator expressions</a></aside> <aside class="footnote brackets" id="id14" role="doc-footnote"> <dt class="label" id="id14">[<a href="#id6">7</a>]</dt> <dd><a class="reference external" href="https://mail.python.org/pipermail/python-ideas/2011-April/009869.html">Possible PEP 3150 style guidelines (#1)</a></aside> <aside class="footnote brackets" id="id15" role="doc-footnote"> <dt class="label" id="id15">[<a href="#id5">8</a>]</dt> <dd><a class="reference external" href="https://mail.python.org/pipermail/python-ideas/2011-October/012276.html">Discussion of PEP 403 (statement local function definition)</a></aside> <aside class="footnote brackets" id="id16" role="doc-footnote"> <dt class="label" id="id16">[<a href="#id7">9</a>]</dt> <dd><a class="reference external" href="https://mail.python.org/pipermail/python-ideas/2011-October/012341.html">Possible PEP 3150 style guidelines (#2)</a></aside> </aside> <ul class="simple"> <li><a class="reference external" href="https://www.curiousefficiency.org/posts/2011/02/status-quo-wins-stalemate.html">The “Status quo wins a stalemate” design principle</a></li> <li><a class="reference external" href="https://mail.python.org/pipermail/python-ideas/2013-August/022526.html">Multi-line lambdas (again!)</a></li> </ul> </section> <section id="copyright"> <h2><a class="toc-backref" href="#copyright" role="doc-backlink">Copyright</a></h2> This document has been placed in the public domain. </section> </section> <hr class="docutils" /> Source: <a class="reference external" href="https://github.com/python/peps/blob/main/peps/pep-3150.rst">https://github.com/python/peps/blob/main/peps/pep-3150.rst</a> Last modified: <a class="reference external" href="https://github.com/python/peps/commits/main/peps/pep-3150.rst">2025-02-01 08:59:27 GMT</a> </article> <nav id="pep-sidebar"> <h2>Contents</h2> <ul> <li><a class="reference internal" href="#abstract">Abstract</a></li> <li><a class="reference internal" href="#proposal">Proposal</a><ul> <li><a class="reference internal" href="#semantics">Semantics</a></li> <li><a class="reference internal" href="#syntax-change">Syntax Change</a></li> <li><a class="reference internal" href="#new-pep-8-guidelines">New PEP 8 Guidelines</a></li> </ul> </li> <li><a class="reference internal" href="#rationale">Rationale</a></li> <li><a class="reference internal" href="#design-discussion">Design Discussion</a><ul> <li><a class="reference internal" href="#keyword-choice">Keyword Choice</a></li> <li><a class="reference internal" href="#relation-to-pep-403">Relation to PEP 403</a></li> <li><a class="reference internal" href="#explaining-container-comprehensions-and-generator-expressions">Explaining Container Comprehensions and Generator Expressions</a></li> <li><a class="reference internal" href="#explaining-decorator-clause-evaluation-and-application">Explaining Decorator Clause Evaluation and Application</a></li> <li><a class="reference internal" href="#anticipated-objections">Anticipated Objections</a><ul> <li><a class="reference internal" href="#two-ways-to-do-it">Two Ways To Do It</a></li> <li><a class="reference internal" href="#out-of-order-execution">Out of Order Execution</a></li> <li><a class="reference internal" href="#harmful-to-introspection">Harmful to Introspection</a></li> <li><a class="reference internal" href="#lack-of-real-world-impact-assessment">Lack of Real World Impact Assessment</a></li> </ul> </li> </ul> </li> <li><a class="reference internal" href="#open-questions">Open Questions</a><ul> <li><a class="reference internal" href="#syntax-for-forward-references">Syntax for Forward References</a></li> <li><a class="reference internal" href="#handling-of-nonlocal-and-global">Handling of <code class="docutils literal notranslate">nonlocal</code> and <code class="docutils literal notranslate">global</code></a></li> <li><a class="reference internal" href="#handling-of-break-and-continue">Handling of <code class="docutils literal notranslate">break</code> and <code class="docutils literal notranslate">continue</code></a></li> <li><a class="reference internal" href="#handling-of-return-and-yield">Handling of <code class="docutils literal notranslate">return</code> and <code class="docutils literal notranslate">yield</code></a></li> </ul> </li> <li><a class="reference internal" href="#examples">Examples</a></li> <li><a class="reference internal" href="#possible-additions">Possible Additions</a></li> <li><a class="reference internal" href="#rejected-alternatives">Rejected Alternatives</a></li> <li><a class="reference internal" href="#reference-implementation">Reference Implementation</a></li> <li><a class="reference internal" href="#to-do">TO-DO</a></li> <li><a class="reference internal" href="#references">References</a></li> <li><a class="reference internal" href="#copyright">Copyright</a></li> </ul> <a id="source" href="https://github.com/python/peps/blob/main/peps/pep-3150.rst">Page Source (GitHub)</a> </nav> </section> <script src="../_static/colour_scheme.js"></script> <script src="../_static/wrap_tables.js"></script> <script src="../_static/sticky_banner.js"></script> </body> </html>