CINXE.COM
PEP 227 – Statically Nested Scopes | peps.python.org
<!DOCTYPE html> <html lang="en"> <head> <meta charset="utf-8"> <meta name="viewport" content="width=device-width, initial-scale=1.0"> <meta name="color-scheme" content="light dark"> <title>PEP 227 – Statically Nested Scopes | peps.python.org</title> <link rel="shortcut icon" href="../_static/py.png"> <link rel="canonical" href="https://peps.python.org/pep-0227/"> <link rel="stylesheet" href="../_static/style.css" type="text/css"> <link rel="stylesheet" href="../_static/mq.css" type="text/css"> <link rel="stylesheet" href="../_static/pygments.css" type="text/css" media="(prefers-color-scheme: light)" id="pyg-light"> <link rel="stylesheet" href="../_static/pygments_dark.css" type="text/css" media="(prefers-color-scheme: dark)" id="pyg-dark"> <link rel="alternate" type="application/rss+xml" title="Latest PEPs" href="https://peps.python.org/peps.rss"> <meta property="og:title" content='PEP 227 – Statically Nested Scopes | peps.python.org'> <meta property="og:description" content="This PEP describes the addition of statically nested scoping (lexical scoping) for Python 2.2, and as a source level option for python 2.1. In addition, Python 2.1 will issue warnings about constructs whose meaning may change when this feature is enabled."> <meta property="og:type" content="website"> <meta property="og:url" content="https://peps.python.org/pep-0227/"> <meta property="og:site_name" content="Python Enhancement Proposals (PEPs)"> <meta property="og:image" content="https://peps.python.org/_static/og-image.png"> <meta property="og:image:alt" content="Python PEPs"> <meta property="og:image:width" content="200"> <meta property="og:image:height" content="200"> <meta name="description" content="This PEP describes the addition of statically nested scoping (lexical scoping) for Python 2.2, and as a source level option for python 2.1. In addition, Python 2.1 will issue warnings about constructs whose meaning may change when this feature is enabled."> <meta name="theme-color" content="#3776ab"> </head> <body> <svg xmlns="http://www.w3.org/2000/svg" style="display: none;"> <symbol id="svg-sun-half" viewBox="0 0 24 24" pointer-events="all"> <title>Following system colour scheme</title> <svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"> <circle cx="12" cy="12" r="9"></circle> <path d="M12 3v18m0-12l4.65-4.65M12 14.3l7.37-7.37M12 19.6l8.85-8.85"></path> </svg> </symbol> <symbol id="svg-moon" viewBox="0 0 24 24" pointer-events="all"> <title>Selected dark colour scheme</title> <svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"> <path stroke="none" d="M0 0h24v24H0z" fill="none"></path> <path d="M12 3c.132 0 .263 0 .393 0a7.5 7.5 0 0 0 7.92 12.446a9 9 0 1 1 -8.313 -12.454z"></path> </svg> </symbol> <symbol id="svg-sun" viewBox="0 0 24 24" pointer-events="all"> <title>Selected light colour scheme</title> <svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"> <circle cx="12" cy="12" r="5"></circle> <line x1="12" y1="1" x2="12" y2="3"></line> <line x1="12" y1="21" x2="12" y2="23"></line> <line x1="4.22" y1="4.22" x2="5.64" y2="5.64"></line> <line x1="18.36" y1="18.36" x2="19.78" y2="19.78"></line> <line x1="1" y1="12" x2="3" y2="12"></line> <line x1="21" y1="12" x2="23" y2="12"></line> <line x1="4.22" y1="19.78" x2="5.64" y2="18.36"></line> <line x1="18.36" y1="5.64" x2="19.78" y2="4.22"></line> </svg> </symbol> </svg> <script> document.documentElement.dataset.colour_scheme = localStorage.getItem("colour_scheme") || "auto" </script> <section id="pep-page-section"> <header> <h1>Python Enhancement Proposals</h1> <ul class="breadcrumbs"> <li><a href="https://www.python.org/" title="The Python Programming Language">Python</a> » </li> <li><a href="../pep-0000/">PEP Index</a> » </li> <li>PEP 227</li> </ul> <button id="colour-scheme-cycler" onClick="setColourScheme(nextColourScheme())"> <svg aria-hidden="true" class="colour-scheme-icon-when-auto"><use href="#svg-sun-half"></use></svg> <svg aria-hidden="true" class="colour-scheme-icon-when-dark"><use href="#svg-moon"></use></svg> <svg aria-hidden="true" class="colour-scheme-icon-when-light"><use href="#svg-sun"></use></svg> <span class="visually-hidden">Toggle light / dark / auto colour theme</span> </button> </header> <article> <section id="pep-content"> <h1 class="page-title">PEP 227 – Statically Nested Scopes</h1> <dl class="rfc2822 field-list simple"> <dt class="field-odd">Author<span class="colon">:</span></dt> <dd class="field-odd">Jeremy Hylton <jeremy at alum.mit.edu></dd> <dt class="field-even">Status<span class="colon">:</span></dt> <dd class="field-even"><abbr title="Accepted and implementation complete, or no longer active">Final</abbr></dd> <dt class="field-odd">Type<span class="colon">:</span></dt> <dd class="field-odd"><abbr title="Normative PEP with a new feature for Python, implementation change for CPython or interoperability standard for the ecosystem">Standards Track</abbr></dd> <dt class="field-even">Created<span class="colon">:</span></dt> <dd class="field-even">01-Nov-2000</dd> <dt class="field-odd">Python-Version<span class="colon">:</span></dt> <dd class="field-odd">2.1</dd> <dt class="field-even">Post-History<span class="colon">:</span></dt> <dd class="field-even"><p></p></dd> </dl> <hr class="docutils" /> <section id="contents"> <details><summary>Table of Contents</summary><ul class="simple"> <li><a class="reference internal" href="#abstract">Abstract</a></li> <li><a class="reference internal" href="#introduction">Introduction</a></li> <li><a class="reference internal" href="#specification">Specification</a></li> <li><a class="reference internal" href="#discussion">Discussion</a></li> <li><a class="reference internal" href="#examples">Examples</a></li> <li><a class="reference internal" href="#backwards-compatibility">Backwards compatibility</a></li> <li><a class="reference internal" href="#compatibility-of-c-api">Compatibility of C API</a></li> <li><a class="reference internal" href="#locals-vars">locals() / vars()</a></li> <li><a class="reference internal" href="#warnings-and-errors">Warnings and Errors</a><ul> <li><a class="reference internal" href="#import-used-in-function-scope">import * used in function scope</a></li> <li><a class="reference internal" href="#bare-exec-in-function-scope">bare exec in function scope</a></li> <li><a class="reference internal" href="#local-shadows-global">local shadows global</a></li> <li><a class="reference internal" href="#rebinding-names-in-enclosing-scopes">Rebinding names in enclosing scopes</a></li> </ul> </li> <li><a class="reference internal" href="#implementation">Implementation</a></li> <li><a class="reference internal" href="#references">References</a></li> <li><a class="reference internal" href="#copyright">Copyright</a></li> </ul> </details></section> <section id="abstract"> <h2><a class="toc-backref" href="#abstract" role="doc-backlink">Abstract</a></h2> <p>This PEP describes the addition of statically nested scoping (lexical scoping) for Python 2.2, and as a source level option for python 2.1. In addition, Python 2.1 will issue warnings about constructs whose meaning may change when this feature is enabled.</p> <p>The old language definition (2.0 and before) defines exactly three namespaces that are used to resolve names – the local, global, and built-in namespaces. The addition of nested scopes allows resolution of unbound local names in enclosing functions’ namespaces.</p> <p>The most visible consequence of this change is that lambdas (and other nested functions) can reference variables defined in the surrounding namespace. Currently, lambdas must often use default arguments to explicitly creating bindings in the lambda’s namespace.</p> </section> <section id="introduction"> <h2><a class="toc-backref" href="#introduction" role="doc-backlink">Introduction</a></h2> <p>This proposal changes the rules for resolving free variables in Python functions. The new name resolution semantics will take effect with Python 2.2. These semantics will also be available in Python 2.1 by adding “from __future__ import nested_scopes” to the top of a module. (See <a class="pep reference internal" href="../pep-0236/" title="PEP 236 – Back to the __future__">PEP 236</a>.)</p> <p>The Python 2.0 definition specifies exactly three namespaces to check for each name – the local namespace, the global namespace, and the builtin namespace. According to this definition, if a function A is defined within a function B, the names bound in B are not visible in A. The proposal changes the rules so that names bound in B are visible in A (unless A contains a name binding that hides the binding in B).</p> <p>This specification introduces rules for lexical scoping that are common in Algol-like languages. The combination of lexical scoping and existing support for first-class functions is reminiscent of Scheme.</p> <p>The changed scoping rules address two problems – the limited utility of lambda expressions (and nested functions in general), and the frequent confusion of new users familiar with other languages that support nested lexical scopes, e.g. the inability to define recursive functions except at the module level.</p> <p>The lambda expression yields an unnamed function that evaluates a single expression. It is often used for callback functions. In the example below (written using the Python 2.0 rules), any name used in the body of the lambda must be explicitly passed as a default argument to the lambda.</p> <div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="kn">from</span><span class="w"> </span><span class="nn">Tkinter</span><span class="w"> </span><span class="kn">import</span> <span class="o">*</span> <span class="n">root</span> <span class="o">=</span> <span class="n">Tk</span><span class="p">()</span> <span class="n">Button</span><span class="p">(</span><span class="n">root</span><span class="p">,</span> <span class="n">text</span><span class="o">=</span><span class="s2">"Click here"</span><span class="p">,</span> <span class="n">command</span><span class="o">=</span><span class="k">lambda</span> <span class="n">root</span><span class="o">=</span><span class="n">root</span><span class="p">:</span> <span class="n">root</span><span class="o">.</span><span class="n">test</span><span class="o">.</span><span class="n">configure</span><span class="p">(</span><span class="n">text</span><span class="o">=</span><span class="s2">"..."</span><span class="p">))</span> </pre></div> </div> <p>This approach is cumbersome, particularly when there are several names used in the body of the lambda. The long list of default arguments obscures the purpose of the code. The proposed solution, in crude terms, implements the default argument approach automatically. The “root=root” argument can be omitted.</p> <p>The new name resolution semantics will cause some programs to behave differently than they did under Python 2.0. In some cases, programs will fail to compile. In other cases, names that were previously resolved using the global namespace will be resolved using the local namespace of an enclosing function. In Python 2.1, warnings will be issued for all statements that will behave differently.</p> </section> <section id="specification"> <h2><a class="toc-backref" href="#specification" role="doc-backlink">Specification</a></h2> <p>Python is a statically scoped language with block structure, in the traditional of Algol. A code block or region, such as a module, class definition, or function body, is the basic unit of a program.</p> <p>Names refer to objects. Names are introduced by name binding operations. Each occurrence of a name in the program text refers to the binding of that name established in the innermost function block containing the use.</p> <p>The name binding operations are argument declaration, assignment, class and function definition, import statements, for statements, and except clauses. Each name binding occurs within a block defined by a class or function definition or at the module level (the top-level code block).</p> <p>If a name is bound anywhere within a code block, all uses of the name within the block are treated as references to the current block. (Note: This can lead to errors when a name is used within a block before it is bound.)</p> <p>If the global statement occurs within a block, all uses of the name specified in the statement refer to the binding of that name in the top-level namespace. Names are resolved in the top-level namespace by searching the global namespace, i.e. the namespace of the module containing the code block, and in the builtin namespace, i.e. the namespace of the <code class="docutils literal notranslate"><span class="pre">__builtin__</span></code> module. The global namespace is searched first. If the name is not found there, the builtin namespace is searched. The global statement must precede all uses of the name.</p> <p>If a name is used within a code block, but it is not bound there and is not declared global, the use is treated as a reference to the nearest enclosing function region. (Note: If a region is contained within a class definition, the name bindings that occur in the class block are not visible to enclosed functions.)</p> <p>A class definition is an executable statement that may contain uses and definitions of names. These references follow the normal rules for name resolution. The namespace of the class definition becomes the attribute dictionary of the class.</p> <p>The following operations are name binding operations. If they occur within a block, they introduce new local names in the current block unless there is also a global declaration.</p> <div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">Function</span> <span class="n">definition</span><span class="p">:</span> <span class="k">def</span><span class="w"> </span><span class="nf">name</span> <span class="o">...</span> <span class="n">Argument</span> <span class="n">declaration</span><span class="p">:</span> <span class="k">def</span><span class="w"> </span><span class="nf">f</span><span class="p">(</span><span class="o">...</span><span class="n">name</span><span class="o">...</span><span class="p">),</span> <span class="k">lambda</span> <span class="o">...</span><span class="n">name</span><span class="o">...</span> <span class="n">Class</span> <span class="n">definition</span><span class="p">:</span> <span class="k">class</span><span class="w"> </span><span class="nc">name</span> <span class="o">...</span> <span class="n">Assignment</span> <span class="n">statement</span><span class="p">:</span> <span class="n">name</span> <span class="o">=</span> <span class="o">...</span> <span class="n">Import</span> <span class="n">statement</span><span class="p">:</span> <span class="kn">import</span><span class="w"> </span><span class="nn">name</span><span class="o">,</span><span class="w"> </span><span class="nn">import</span> <span class="n">module</span> <span class="k">as</span> <span class="n">name</span><span class="p">,</span> <span class="kn">from</span><span class="w"> </span><span class="nn">module</span><span class="w"> </span><span class="kn">import</span> <span class="n">name</span> <span class="n">Implicit</span> <span class="n">assignment</span><span class="p">:</span> <span class="n">names</span> <span class="n">are</span> <span class="n">bound</span> <span class="n">by</span> <span class="k">for</span> <span class="n">statements</span> <span class="ow">and</span> <span class="k">except</span> <span class="n">clauses</span> </pre></div> </div> <p>There are several cases where Python statements are illegal when used in conjunction with nested scopes that contain free variables.</p> <p>If a variable is referenced in an enclosed scope, it is an error to delete the name. The compiler will raise a <code class="docutils literal notranslate"><span class="pre">SyntaxError</span></code> for ‘del name’.</p> <p>If the wild card form of import (<code class="docutils literal notranslate"><span class="pre">import</span> <span class="pre">*</span></code>) is used in a function and the function contains a nested block with free variables, the compiler will raise a <code class="docutils literal notranslate"><span class="pre">SyntaxError</span></code>.</p> <p>If exec is used in a function and the function contains a nested block with free variables, the compiler will raise a <code class="docutils literal notranslate"><span class="pre">SyntaxError</span></code> unless the exec explicitly specifies the local namespace for the exec. (In other words, “exec obj” would be illegal, but “exec obj in ns” would be legal.)</p> <p>If a name bound in a function scope is also the name of a module global name or a standard builtin name, and the function contains a nested function scope that references the name, the compiler will issue a warning. The name resolution rules will result in different bindings under Python 2.0 than under Python 2.2. The warning indicates that the program may not run correctly with all versions of Python.</p> </section> <section id="discussion"> <h2><a class="toc-backref" href="#discussion" role="doc-backlink">Discussion</a></h2> <p>The specified rules allow names defined in a function to be referenced in any nested function defined with that function. The name resolution rules are typical for statically scoped languages, with three primary exceptions:</p> <ul class="simple"> <li>Names in class scope are not accessible.</li> <li>The global statement short-circuits the normal rules.</li> <li>Variables are not declared.</li> </ul> <p>Names in class scope are not accessible. Names are resolved in the innermost enclosing function scope. If a class definition occurs in a chain of nested scopes, the resolution process skips class definitions. This rule prevents odd interactions between class attributes and local variable access. If a name binding operation occurs in a class definition, it creates an attribute on the resulting class object. To access this variable in a method, or in a function nested within a method, an attribute reference must be used, either via self or via the class name.</p> <p>An alternative would have been to allow name binding in class scope to behave exactly like name binding in function scope. This rule would allow class attributes to be referenced either via attribute reference or simple name. This option was ruled out because it would have been inconsistent with all other forms of class and instance attribute access, which always use attribute references. Code that used simple names would have been obscure.</p> <p>The global statement short-circuits the normal rules. Under the proposal, the global statement has exactly the same effect that it does for Python 2.0. It is also noteworthy because it allows name binding operations performed in one block to change bindings in another block (the module).</p> <p>Variables are not declared. If a name binding operation occurs anywhere in a function, then that name is treated as local to the function and all references refer to the local binding. If a reference occurs before the name is bound, a NameError is raised. The only kind of declaration is the global statement, which allows programs to be written using mutable global variables. As a consequence, it is not possible to rebind a name defined in an enclosing scope. An assignment operation can only bind a name in the current scope or in the global scope. The lack of declarations and the inability to rebind names in enclosing scopes are unusual for lexically scoped languages; there is typically a mechanism to create name bindings (e.g. lambda and let in Scheme) and a mechanism to change the bindings (set! in Scheme).</p> </section> <section id="examples"> <h2><a class="toc-backref" href="#examples" role="doc-backlink">Examples</a></h2> <p>A few examples are included to illustrate the way the rules work.</p> <div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="gp">>>> </span><span class="k">def</span><span class="w"> </span><span class="nf">make_adder</span><span class="p">(</span><span class="n">base</span><span class="p">):</span> <span class="gp">... </span> <span class="k">def</span><span class="w"> </span><span class="nf">adder</span><span class="p">(</span><span class="n">x</span><span class="p">):</span> <span class="gp">... </span> <span class="k">return</span> <span class="n">base</span> <span class="o">+</span> <span class="n">x</span> <span class="gp">... </span> <span class="k">return</span> <span class="n">adder</span> <span class="gp">>>> </span><span class="n">add5</span> <span class="o">=</span> <span class="n">make_adder</span><span class="p">(</span><span class="mi">5</span><span class="p">)</span> <span class="gp">>>> </span><span class="n">add5</span><span class="p">(</span><span class="mi">6</span><span class="p">)</span> <span class="go">11</span> <span class="gp">>>> </span><span class="k">def</span><span class="w"> </span><span class="nf">make_fact</span><span class="p">():</span> <span class="gp">... </span> <span class="k">def</span><span class="w"> </span><span class="nf">fact</span><span class="p">(</span><span class="n">n</span><span class="p">):</span> <span class="gp">... </span> <span class="k">if</span> <span class="n">n</span> <span class="o">==</span> <span class="mi">1</span><span class="p">:</span> <span class="gp">... </span> <span class="k">return</span> <span class="mi">1</span><span class="n">L</span> <span class="gp">... </span> <span class="k">else</span><span class="p">:</span> <span class="gp">... </span> <span class="k">return</span> <span class="n">n</span> <span class="o">*</span> <span class="n">fact</span><span class="p">(</span><span class="n">n</span> <span class="o">-</span> <span class="mi">1</span><span class="p">)</span> <span class="gp">... </span> <span class="k">return</span> <span class="n">fact</span> <span class="gp">>>> </span><span class="n">fact</span> <span class="o">=</span> <span class="n">make_fact</span><span class="p">()</span> <span class="gp">>>> </span><span class="n">fact</span><span class="p">(</span><span class="mi">7</span><span class="p">)</span> <span class="go">5040L</span> <span class="gp">>>> </span><span class="k">def</span><span class="w"> </span><span class="nf">make_wrapper</span><span class="p">(</span><span class="n">obj</span><span class="p">):</span> <span class="gp">... </span> <span class="k">class</span><span class="w"> </span><span class="nc">Wrapper</span><span class="p">:</span> <span class="gp">... </span> <span class="k">def</span><span class="w"> </span><span class="fm">__getattr__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">attr</span><span class="p">):</span> <span class="gp">... </span> <span class="k">if</span> <span class="n">attr</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="o">!=</span> <span class="s1">'_'</span><span class="p">:</span> <span class="gp">... </span> <span class="k">return</span> <span class="nb">getattr</span><span class="p">(</span><span class="n">obj</span><span class="p">,</span> <span class="n">attr</span><span class="p">)</span> <span class="gp">... </span> <span class="k">else</span><span class="p">:</span> <span class="gp">... </span> <span class="k">raise</span> <span class="ne">AttributeError</span><span class="p">,</span> <span class="n">attr</span> <span class="gp">... </span> <span class="k">return</span> <span class="n">Wrapper</span><span class="p">()</span> <span class="gp">>>> </span><span class="k">class</span><span class="w"> </span><span class="nc">Test</span><span class="p">:</span> <span class="gp">... </span> <span class="n">public</span> <span class="o">=</span> <span class="mi">2</span> <span class="gp">... </span> <span class="n">_private</span> <span class="o">=</span> <span class="mi">3</span> <span class="gp">>>> </span><span class="n">w</span> <span class="o">=</span> <span class="n">make_wrapper</span><span class="p">(</span><span class="n">Test</span><span class="p">())</span> <span class="gp">>>> </span><span class="n">w</span><span class="o">.</span><span class="n">public</span> <span class="go">2</span> <span class="gp">>>> </span><span class="n">w</span><span class="o">.</span><span class="n">_private</span> <span class="gt">Traceback (most recent call last):</span> File <span class="nb">"<stdin>"</span>, line <span class="m">1</span>, in <span class="n">?</span> <span class="gr">AttributeError</span>: <span class="n">_private</span> </pre></div> </div> <p>An example from Tim Peters demonstrates the potential pitfalls of nested scopes in the absence of declarations:</p> <div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">i</span> <span class="o">=</span> <span class="mi">6</span> <span class="k">def</span><span class="w"> </span><span class="nf">f</span><span class="p">(</span><span class="n">x</span><span class="p">):</span> <span class="k">def</span><span class="w"> </span><span class="nf">g</span><span class="p">():</span> <span class="nb">print</span> <span class="n">i</span> <span class="c1"># ...</span> <span class="c1"># skip to the next page</span> <span class="c1"># ...</span> <span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="n">x</span><span class="p">:</span> <span class="c1"># ah, i *is* local to f, so this is what g sees</span> <span class="k">pass</span> <span class="n">g</span><span class="p">()</span> </pre></div> </div> <p>The call to <code class="docutils literal notranslate"><span class="pre">g()</span></code> will refer to the variable i bound in <code class="docutils literal notranslate"><span class="pre">f()</span></code> by the for loop. If <code class="docutils literal notranslate"><span class="pre">g()</span></code> is called before the loop is executed, a NameError will be raised.</p> </section> <section id="backwards-compatibility"> <h2><a class="toc-backref" href="#backwards-compatibility" role="doc-backlink">Backwards compatibility</a></h2> <p>There are two kinds of compatibility problems caused by nested scopes. In one case, code that behaved one way in earlier versions behaves differently because of nested scopes. In the other cases, certain constructs interact badly with nested scopes and will trigger SyntaxErrors at compile time.</p> <p>The following example from Skip Montanaro illustrates the first kind of problem:</p> <div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">x</span> <span class="o">=</span> <span class="mi">1</span> <span class="k">def</span><span class="w"> </span><span class="nf">f1</span><span class="p">():</span> <span class="n">x</span> <span class="o">=</span> <span class="mi">2</span> <span class="k">def</span><span class="w"> </span><span class="nf">inner</span><span class="p">():</span> <span class="nb">print</span> <span class="n">x</span> <span class="n">inner</span><span class="p">()</span> </pre></div> </div> <p>Under the Python 2.0 rules, the print statement inside <code class="docutils literal notranslate"><span class="pre">inner()</span></code> refers to the global variable x and will print 1 if <code class="docutils literal notranslate"><span class="pre">f1()</span></code> is called. Under the new rules, it refers to the <code class="docutils literal notranslate"><span class="pre">f1()</span></code>’s namespace, the nearest enclosing scope with a binding.</p> <p>The problem occurs only when a global variable and a local variable share the same name and a nested function uses that name to refer to the global variable. This is poor programming practice, because readers will easily confuse the two different variables. One example of this problem was found in the Python standard library during the implementation of nested scopes.</p> <p>To address this problem, which is unlikely to occur often, the Python 2.1 compiler (when nested scopes are not enabled) issues a warning.</p> <p>The other compatibility problem is caused by the use of <code class="docutils literal notranslate"><span class="pre">import</span> <span class="pre">*</span></code> and ‘exec’ in a function body, when that function contains a nested scope and the contained scope has free variables. For example:</p> <div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">y</span> <span class="o">=</span> <span class="mi">1</span> <span class="k">def</span><span class="w"> </span><span class="nf">f</span><span class="p">():</span> <span class="n">exec</span> <span class="s2">"y = 'gotcha'"</span> <span class="c1"># or from module import *</span> <span class="k">def</span><span class="w"> </span><span class="nf">g</span><span class="p">():</span> <span class="k">return</span> <span class="n">y</span> <span class="o">...</span> </pre></div> </div> <p>At compile-time, the compiler cannot tell whether an exec that operates on the local namespace or an <code class="docutils literal notranslate"><span class="pre">import</span> <span class="pre">*</span></code> will introduce name bindings that shadow the global y. Thus, it is not possible to tell whether the reference to y in <code class="docutils literal notranslate"><span class="pre">g()</span></code> should refer to the global or to a local name in <code class="docutils literal notranslate"><span class="pre">f()</span></code>.</p> <p>In discussion of the python-list, people argued for both possible interpretations. On the one hand, some thought that the reference in <code class="docutils literal notranslate"><span class="pre">g()</span></code> should be bound to a local y if one exists. One problem with this interpretation is that it is impossible for a human reader of the code to determine the binding of y by local inspection. It seems likely to introduce subtle bugs. The other interpretation is to treat exec and import * as dynamic features that do not effect static scoping. Under this interpretation, the exec and import * would introduce local names, but those names would never be visible to nested scopes. In the specific example above, the code would behave exactly as it did in earlier versions of Python.</p> <p>Since each interpretation is problematic and the exact meaning ambiguous, the compiler raises an exception. The Python 2.1 compiler issues a warning when nested scopes are not enabled.</p> <p>A brief review of three Python projects (the standard library, Zope, and a beta version of PyXPCOM) found four backwards compatibility issues in approximately 200,000 lines of code. There was one example of case #1 (subtle behavior change) and two examples of <code class="docutils literal notranslate"><span class="pre">import</span> <span class="pre">*</span></code> problems in the standard library.</p> <p>(The interpretation of the <code class="docutils literal notranslate"><span class="pre">import</span> <span class="pre">*</span></code> and exec restriction that was implemented in Python 2.1a2 was much more restrictive, based on language that in the reference manual that had never been enforced. These restrictions were relaxed following the release.)</p> </section> <section id="compatibility-of-c-api"> <h2><a class="toc-backref" href="#compatibility-of-c-api" role="doc-backlink">Compatibility of C API</a></h2> <p>The implementation causes several Python C API functions to change, including <code class="docutils literal notranslate"><span class="pre">PyCode_New()</span></code>. As a result, C extensions may need to be updated to work correctly with Python 2.1.</p> </section> <section id="locals-vars"> <h2><a class="toc-backref" href="#locals-vars" role="doc-backlink">locals() / vars()</a></h2> <p>These functions return a dictionary containing the current scope’s local variables. Modifications to the dictionary do not affect the values of variables. Under the current rules, the use of <code class="docutils literal notranslate"><span class="pre">locals()</span></code> and <code class="docutils literal notranslate"><span class="pre">globals()</span></code> allows the program to gain access to all the namespaces in which names are resolved.</p> <p>An analogous function will not be provided for nested scopes. Under this proposal, it will not be possible to gain dictionary-style access to all visible scopes.</p> </section> <section id="warnings-and-errors"> <h2><a class="toc-backref" href="#warnings-and-errors" role="doc-backlink">Warnings and Errors</a></h2> <p>The compiler will issue warnings in Python 2.1 to help identify programs that may not compile or run correctly under future versions of Python. Under Python 2.2 or Python 2.1 if the <code class="docutils literal notranslate"><span class="pre">nested_scopes</span></code> future statement is used, which are collectively referred to as “future semantics” in this section, the compiler will issue SyntaxErrors in some cases.</p> <p>The warnings typically apply when a function that contains a nested function that has free variables. For example, if function F contains a function G and G uses the builtin <code class="docutils literal notranslate"><span class="pre">len()</span></code>, then F is a function that contains a nested function (G) with a free variable (len). The label “free-in-nested” will be used to describe these functions.</p> <section id="import-used-in-function-scope"> <h3><a class="toc-backref" href="#import-used-in-function-scope" role="doc-backlink">import * used in function scope</a></h3> <p>The language reference specifies that <code class="docutils literal notranslate"><span class="pre">import</span> <span class="pre">*</span></code> may only occur in a module scope. (Sec. 6.11) The implementation of C Python has supported <code class="docutils literal notranslate"><span class="pre">import</span> <span class="pre">*</span></code> at the function scope.</p> <p>If <code class="docutils literal notranslate"><span class="pre">import</span> <span class="pre">*</span></code> is used in the body of a free-in-nested function, the compiler will issue a warning. Under future semantics, the compiler will raise a <code class="docutils literal notranslate"><span class="pre">SyntaxError</span></code>.</p> </section> <section id="bare-exec-in-function-scope"> <h3><a class="toc-backref" href="#bare-exec-in-function-scope" role="doc-backlink">bare exec in function scope</a></h3> <p>The exec statement allows two optional expressions following the keyword “in” that specify the namespaces used for locals and globals. An exec statement that omits both of these namespaces is a bare exec.</p> <p>If a bare exec is used in the body of a free-in-nested function, the compiler will issue a warning. Under future semantics, the compiler will raise a <code class="docutils literal notranslate"><span class="pre">SyntaxError</span></code>.</p> </section> <section id="local-shadows-global"> <h3><a class="toc-backref" href="#local-shadows-global" role="doc-backlink">local shadows global</a></h3> <p>If a free-in-nested function has a binding for a local variable that (1) is used in a nested function and (2) is the same as a global variable, the compiler will issue a warning.</p> </section> <section id="rebinding-names-in-enclosing-scopes"> <h3><a class="toc-backref" href="#rebinding-names-in-enclosing-scopes" role="doc-backlink">Rebinding names in enclosing scopes</a></h3> <p>There are technical issues that make it difficult to support rebinding of names in enclosing scopes, but the primary reason that it is not allowed in the current proposal is that Guido is opposed to it. His motivation: it is difficult to support, because it would require a new mechanism that would allow the programmer to specify that an assignment in a block is supposed to rebind the name in an enclosing block; presumably a keyword or special syntax (x := 3) would make this possible. Given that this would encourage the use of local variables to hold state that is better stored in a class instance, it’s not worth adding new syntax to make this possible (in Guido’s opinion).</p> <p>The proposed rules allow programmers to achieve the effect of rebinding, albeit awkwardly. The name that will be effectively rebound by enclosed functions is bound to a container object. In place of assignment, the program uses modification of the container to achieve the desired effect:</p> <div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="k">def</span><span class="w"> </span><span class="nf">bank_account</span><span class="p">(</span><span class="n">initial_balance</span><span class="p">):</span> <span class="n">balance</span> <span class="o">=</span> <span class="p">[</span><span class="n">initial_balance</span><span class="p">]</span> <span class="k">def</span><span class="w"> </span><span class="nf">deposit</span><span class="p">(</span><span class="n">amount</span><span class="p">):</span> <span class="n">balance</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="o">=</span> <span class="n">balance</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="o">+</span> <span class="n">amount</span> <span class="k">return</span> <span class="n">balance</span> <span class="k">def</span><span class="w"> </span><span class="nf">withdraw</span><span class="p">(</span><span class="n">amount</span><span class="p">):</span> <span class="n">balance</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="o">=</span> <span class="n">balance</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="o">-</span> <span class="n">amount</span> <span class="k">return</span> <span class="n">balance</span> <span class="k">return</span> <span class="n">deposit</span><span class="p">,</span> <span class="n">withdraw</span> </pre></div> </div> <p>Support for rebinding in nested scopes would make this code clearer. A class that defines <code class="docutils literal notranslate"><span class="pre">deposit()</span></code> and <code class="docutils literal notranslate"><span class="pre">withdraw()</span></code> methods and the balance as an instance variable would be clearer still. Since classes seem to achieve the same effect in a more straightforward manner, they are preferred.</p> </section> </section> <section id="implementation"> <h2><a class="toc-backref" href="#implementation" role="doc-backlink">Implementation</a></h2> <p>The implementation for C Python uses flat closures <a class="footnote-reference brackets" href="#id2" id="id1">[1]</a>. Each def or lambda expression that is executed will create a closure if the body of the function or any contained function has free variables. Using flat closures, the creation of closures is somewhat expensive but lookup is cheap.</p> <p>The implementation adds several new opcodes and two new kinds of names in code objects. A variable can be either a cell variable or a free variable for a particular code object. A cell variable is referenced by containing scopes; as a result, the function where it is defined must allocate separate storage for it on each invocation. A free variable is referenced via a function’s closure.</p> <p>The choice of free closures was made based on three factors. First, nested functions are presumed to be used infrequently, deeply nested (several levels of nesting) still less frequently. Second, lookup of names in a nested scope should be fast. Third, the use of nested scopes, particularly where a function that access an enclosing scope is returned, should not prevent unreferenced objects from being reclaimed by the garbage collector.</p> </section> <section id="references"> <h2><a class="toc-backref" href="#references" role="doc-backlink">References</a></h2> <aside class="footnote-list brackets"> <aside class="footnote brackets" id="id2" role="doc-footnote"> <dt class="label" id="id2">[<a href="#id1">1</a>]</dt> <dd>Luca Cardelli. Compiling a functional language. In Proc. of the 1984 ACM Conference on Lisp and Functional Programming, pp. 208-217, Aug. 1984 <a class="reference external" href="https://dl.acm.org/doi/10.1145/800055.802037">https://dl.acm.org/doi/10.1145/800055.802037</a></aside> </aside> </section> <section id="copyright"> <h2><a class="toc-backref" href="#copyright" role="doc-backlink">Copyright</a></h2> </section> </section> <hr class="docutils" /> <p>Source: <a class="reference external" href="https://github.com/python/peps/blob/main/peps/pep-0227.rst">https://github.com/python/peps/blob/main/peps/pep-0227.rst</a></p> <p>Last modified: <a class="reference external" href="https://github.com/python/peps/commits/main/peps/pep-0227.rst">2025-02-01 08:55:40 GMT</a></p> </article> <nav id="pep-sidebar"> <h2>Contents</h2> <ul> <li><a class="reference internal" href="#abstract">Abstract</a></li> <li><a class="reference internal" href="#introduction">Introduction</a></li> <li><a class="reference internal" href="#specification">Specification</a></li> <li><a class="reference internal" href="#discussion">Discussion</a></li> <li><a class="reference internal" href="#examples">Examples</a></li> <li><a class="reference internal" href="#backwards-compatibility">Backwards compatibility</a></li> <li><a class="reference internal" href="#compatibility-of-c-api">Compatibility of C API</a></li> <li><a class="reference internal" href="#locals-vars">locals() / vars()</a></li> <li><a class="reference internal" href="#warnings-and-errors">Warnings and Errors</a><ul> <li><a class="reference internal" href="#import-used-in-function-scope">import * used in function scope</a></li> <li><a class="reference internal" href="#bare-exec-in-function-scope">bare exec in function scope</a></li> <li><a class="reference internal" href="#local-shadows-global">local shadows global</a></li> <li><a class="reference internal" href="#rebinding-names-in-enclosing-scopes">Rebinding names in enclosing scopes</a></li> </ul> </li> <li><a class="reference internal" href="#implementation">Implementation</a></li> <li><a class="reference internal" href="#references">References</a></li> <li><a class="reference internal" href="#copyright">Copyright</a></li> </ul> <br> <a id="source" href="https://github.com/python/peps/blob/main/peps/pep-0227.rst">Page Source (GitHub)</a> </nav> </section> <script src="../_static/colour_scheme.js"></script> <script src="../_static/wrap_tables.js"></script> <script src="../_static/sticky_banner.js"></script> </body> </html>