CINXE.COM

<!DOCTYPE html> <html lang="en"> <head> <meta charset="utf-8"> <meta name="viewport" content="width=device-width, initial-scale=1.0"> <meta name="color-scheme" content="light dark"> <title>PEP 255 – Simple Generators | peps.python.org</title> <link rel="shortcut icon" href="../_static/py.png"> <link rel="canonical" href="https://peps.python.org/pep-0255/"> <link rel="stylesheet" href="../_static/style.css" type="text/css"> <link rel="stylesheet" href="../_static/mq.css" type="text/css"> <link rel="stylesheet" href="../_static/pygments.css" type="text/css" media="(prefers-color-scheme: light)" id="pyg-light"> <link rel="stylesheet" href="../_static/pygments_dark.css" type="text/css" media="(prefers-color-scheme: dark)" id="pyg-dark"> <link rel="alternate" type="application/rss+xml" title="Latest PEPs" href="https://peps.python.org/peps.rss"> <meta property="og:title" content='PEP 255 – Simple Generators | peps.python.org'> <meta property="og:description" content="This PEP introduces the concept of generators to Python, as well as a new statement used in conjunction with them, the yield statement."> <meta property="og:type" content="website"> <meta property="og:url" content="https://peps.python.org/pep-0255/"> <meta property="og:site_name" content="Python Enhancement Proposals (PEPs)"> <meta property="og:image" content="https://peps.python.org/_static/og-image.png"> <meta property="og:image:alt" content="Python PEPs"> <meta property="og:image:width" content="200"> <meta property="og:image:height" content="200"> <meta name="description" content="This PEP introduces the concept of generators to Python, as well as a new statement used in conjunction with them, the yield statement."> <meta name="theme-color" content="#3776ab"> </head> <body> <svg xmlns="http://www.w3.org/2000/svg" style="display: none;"> <symbol id="svg-sun-half" viewBox="0 0 24 24" pointer-events="all"> <title>Following system colour scheme</title> <svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"> <circle cx="12" cy="12" r="9"></circle> <path d="M12 3v18m0-12l4.65-4.65M12 14.3l7.37-7.37M12 19.6l8.85-8.85"></path> </svg> </symbol> <symbol id="svg-moon" viewBox="0 0 24 24" pointer-events="all"> <title>Selected dark colour scheme</title> <svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"> <path stroke="none" d="M0 0h24v24H0z" fill="none"></path> <path d="M12 3c.132 0 .263 0 .393 0a7.5 7.5 0 0 0 7.92 12.446a9 9 0 1 1 -8.313 -12.454z"></path> </svg> </symbol> <symbol id="svg-sun" viewBox="0 0 24 24" pointer-events="all"> <title>Selected light colour scheme</title> <svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"> <circle cx="12" cy="12" r="5"></circle> <line x1="12" y1="1" x2="12" y2="3"></line> <line x1="12" y1="21" x2="12" y2="23"></line> <line x1="4.22" y1="4.22" x2="5.64" y2="5.64"></line> <line x1="18.36" y1="18.36" x2="19.78" y2="19.78"></line> <line x1="1" y1="12" x2="3" y2="12"></line> <line x1="21" y1="12" x2="23" y2="12"></line> <line x1="4.22" y1="19.78" x2="5.64" y2="18.36"></line> <line x1="18.36" y1="5.64" x2="19.78" y2="4.22"></line> </svg> </symbol> </svg> <script> document.documentElement.dataset.colour_scheme = localStorage.getItem("colour_scheme") || "auto" </script> <section id="pep-page-section"> <header> <h1>Python Enhancement Proposals</h1> <ul class="breadcrumbs"> <li><a href="https://www.python.org/" title="The Python Programming Language">Python</a> » </li> <li><a href="../pep-0000/">PEP Index</a> » </li> <li>PEP 255</li> </ul> <button id="colour-scheme-cycler" onClick="setColourScheme(nextColourScheme())"> <svg aria-hidden="true" class="colour-scheme-icon-when-auto"><use href="#svg-sun-half"></use></svg> <svg aria-hidden="true" class="colour-scheme-icon-when-dark"><use href="#svg-moon"></use></svg> <svg aria-hidden="true" class="colour-scheme-icon-when-light"><use href="#svg-sun"></use></svg> Toggle light / dark / auto colour theme </button> </header> <article> <section id="pep-content"> <h1 class="page-title">PEP 255 – Simple Generators</h1> <dl class="rfc2822 field-list simple"> <dt class="field-odd">Author:</dt> <dd class="field-odd">Neil Schemenauer <nas at arctrix.com>, Tim Peters <tim.peters at gmail.com>, Magnus Lie Hetland <magnus at hetland.org></dd> <dt class="field-even">Status:</dt> <dd class="field-even"><abbr title="Accepted and implementation complete, or no longer active">Final</abbr></dd> <dt class="field-odd">Type:</dt> <dd class="field-odd"><abbr title="Normative PEP with a new feature for Python, implementation change for CPython or interoperability standard for the ecosystem">Standards Track</abbr></dd> <dt class="field-even">Requires:</dt> <dd class="field-even"><a class="reference external" href="../pep-0234/">234</a></dd> <dt class="field-odd">Created:</dt> <dd class="field-odd">18-May-2001</dd> <dt class="field-even">Python-Version:</dt> <dd class="field-even">2.2</dd> <dt class="field-odd">Post-History:</dt> <dd class="field-odd">14-Jun-2001, 23-Jun-2001</dd> </dl> <hr class="docutils" /> <section id="contents"> <details><summary>Table of Contents</summary><ul class="simple"> <li><a class="reference internal" href="#abstract">Abstract</a></li> <li><a class="reference internal" href="#motivation">Motivation</a></li> <li><a class="reference internal" href="#specification-yield">Specification: Yield</a></li> <li><a class="reference internal" href="#specification-return">Specification: Return</a></li> <li><a class="reference internal" href="#specification-generators-and-exception-propagation">Specification: Generators and Exception Propagation</a></li> <li><a class="reference internal" href="#specification-try-except-finally">Specification: Try/Except/Finally</a></li> <li><a class="reference internal" href="#example">Example</a></li> <li><a class="reference internal" href="#q-a">Q & A</a><ul> <li><a class="reference internal" href="#why-not-a-new-keyword-instead-of-reusing-def">Why not a new keyword instead of reusing <code class="docutils literal notranslate">def</code>?</a></li> <li><a class="reference internal" href="#why-a-new-keyword-for-yield-why-not-a-builtin-function-instead">Why a new keyword for <code class="docutils literal notranslate">yield</code>? Why not a builtin function instead?</a></li> <li><a class="reference internal" href="#then-why-not-some-other-special-syntax-without-a-new-keyword">Then why not some other special syntax without a new keyword?</a></li> <li><a class="reference internal" href="#why-allow-return-at-all-why-not-force-termination-to-be-spelled-raise-stopiteration">Why allow <code class="docutils literal notranslate">return</code> at all? Why not force termination to be spelled <code class="docutils literal notranslate">raise StopIteration</code>?</a></li> <li><a class="reference internal" href="#then-why-not-allow-an-expression-on-return-too">Then why not allow an expression on <code class="docutils literal notranslate">return</code> too?</a></li> </ul> </li> <li><a class="reference internal" href="#bdfl-pronouncements">BDFL Pronouncements</a><ul> <li><a class="reference internal" href="#issue">Issue</a></li> <li><a class="reference internal" href="#con">Con</a></li> <li><a class="reference internal" href="#pro">Pro</a></li> <li><a class="reference internal" href="#bdfl">BDFL</a></li> </ul> </li> <li><a class="reference internal" href="#reference-implementation">Reference Implementation</a></li> <li><a class="reference internal" href="#footnotes-and-references">Footnotes and References</a></li> <li><a class="reference internal" href="#copyright">Copyright</a></li> </ul> </details></section> <section id="abstract"> <h2><a class="toc-backref" href="#abstract" role="doc-backlink">Abstract</a></h2> This PEP introduces the concept of generators to Python, as well as a new statement used in conjunction with them, the <code class="docutils literal notranslate">yield</code> statement. </section> <section id="motivation"> <h2><a class="toc-backref" href="#motivation" role="doc-backlink">Motivation</a></h2> When a producer function has a hard enough job that it requires maintaining state between values produced, most programming languages offer no pleasant and efficient solution beyond adding a callback function to the producer’s argument list, to be called with each value produced. For example, <code class="docutils literal notranslate">tokenize.py</code> in the standard library takes this approach: the caller must pass a tokeneater function to <code class="docutils literal notranslate">tokenize()</code>, called whenever <code class="docutils literal notranslate">tokenize()</code> finds the next token. This allows tokenize to be coded in a natural way, but programs calling tokenize are typically convoluted by the need to remember between callbacks which token(s) were seen last. The tokeneater function in <code class="docutils literal notranslate">tabnanny.py</code> is a good example of that, maintaining a state machine in global variables, to remember across callbacks what it has already seen and what it hopes to see next. This was difficult to get working correctly, and is still difficult for people to understand. Unfortunately, that’s typical of this approach. An alternative would have been for tokenize to produce an entire parse of the Python program at once, in a large list. Then tokenize clients could be written in a natural way, using local variables and local control flow (such as loops and nested if statements) to keep track of their state. But this isn’t practical: programs can be very large, so no a priori bound can be placed on the memory needed to materialize the whole parse; and some tokenize clients only want to see whether something specific appears early in the program (e.g., a future statement, or, as is done in IDLE, just the first indented statement), and then parsing the whole program first is a severe waste of time. Another alternative would be to make tokenize an <a class="pep reference internal" href="../pep-0234/" title="PEP 234 – Iterators">iterator</a>, delivering the next token whenever its <code class="docutils literal notranslate">.next()</code> method is invoked. This is pleasant for the caller in the same way a large list of results would be, but without the memory and “what if I want to get out early?” drawbacks. However, this shifts the burden on tokenize to remember its state between <code class="docutils literal notranslate">.next()</code> invocations, and the reader need only glance at <code class="docutils literal notranslate">tokenize.tokenize_loop()</code> to realize what a horrid chore that would be. Or picture a recursive algorithm for producing the nodes of a general tree structure: to cast that into an iterator framework requires removing the recursion manually and maintaining the state of the traversal by hand. A fourth option is to run the producer and consumer in separate threads. This allows both to maintain their states in natural ways, and so is pleasant for both. Indeed, Demo/threads/Generator.py in the Python source distribution provides a usable synchronized-communication class for doing that in a general way. This doesn’t work on platforms without threads, though, and is very slow on platforms that do (compared to what is achievable without threads). A final option is to use the Stackless <a class="footnote-reference brackets" href="#id6" id="id1">[1]</a> (<a class="pep reference internal" href="../pep-0219/" title="PEP 219 – Stackless Python">PEP 219</a>) variant implementation of Python instead, which supports lightweight coroutines. This has much the same programmatic benefits as the thread option, but is much more efficient. However, Stackless is a controversial rethinking of the Python core, and it may not be possible for Jython to implement the same semantics. This PEP isn’t the place to debate that, so suffice it to say here that generators provide a useful subset of Stackless functionality in a way that fits easily into the current CPython implementation, and is believed to be relatively straightforward for other Python implementations. That exhausts the current alternatives. Some other high-level languages provide pleasant solutions, notably iterators in Sather <a class="footnote-reference brackets" href="#id7" id="id2">[2]</a>, which were inspired by iterators in CLU; and generators in Icon <a class="footnote-reference brackets" href="#id8" id="id3">[3]</a>, a novel language where every expression is a generator. There are differences among these, but the basic idea is the same: provide a kind of function that can return an intermediate result (“the next value”) to its caller, but maintaining the function’s local state so that the function can be resumed again right where it left off. A very simple example: <div class="highlight-default notranslate"><div class="highlight"><pre>def fib(): a, b = 0, 1 while 1: yield b a, b = b, a+b </pre></div> </div> When <code class="docutils literal notranslate">fib()</code> is first invoked, it sets a to 0 and b to 1, then yields b back to its caller. The caller sees 1. When <code class="docutils literal notranslate">fib</code> is resumed, from its point of view the <code class="docutils literal notranslate">yield</code> statement is really the same as, say, a <code class="docutils literal notranslate">print</code> statement: <code class="docutils literal notranslate">fib</code> continues after the yield with all local state intact. a and b then become 1 and 1, and <code class="docutils literal notranslate">fib</code> loops back to the <code class="docutils literal notranslate">yield</code>, yielding 1 to its invoker. And so on. From <code class="docutils literal notranslate">fib</code>’s point of view it’s just delivering a sequence of results, as if via callback. But from its caller’s point of view, the <code class="docutils literal notranslate">fib</code> invocation is an iterable object that can be resumed at will. As in the thread approach, this allows both sides to be coded in the most natural ways; but unlike the thread approach, this can be done efficiently and on all platforms. Indeed, resuming a generator should be no more expensive than a function call. The same kind of approach applies to many producer/consumer functions. For example, <code class="docutils literal notranslate">tokenize.py</code> could yield the next token instead of invoking a callback function with it as argument, and tokenize clients could iterate over the tokens in a natural way: a Python generator is a kind of Python <a class="pep reference internal" href="../pep-0234/" title="PEP 234 – Iterators">iterator</a>, but of an especially powerful kind. </section> <section id="specification-yield"> <h2><a class="toc-backref" href="#specification-yield" role="doc-backlink">Specification: Yield</a></h2> A new statement is introduced: <div class="highlight-default notranslate"><div class="highlight"><pre>yield_stmt: "yield" expression_list </pre></div> </div> <code class="docutils literal notranslate">yield</code> is a new keyword, so a <code class="docutils literal notranslate">future</code> statement (<a class="pep reference internal" href="../pep-0236/" title="PEP 236 – Back to the __future__">PEP 236</a>) is needed to phase this in: in the initial release, a module desiring to use generators must include the line: <div class="highlight-default notranslate"><div class="highlight"><pre>from __future__ import generators </pre></div> </div> near the top (see <a class="pep reference internal" href="../pep-0236/" title="PEP 236 – Back to the __future__">PEP 236</a>) for details). Modules using the identifier <code class="docutils literal notranslate">yield</code> without a <code class="docutils literal notranslate">future</code> statement will trigger warnings. In the following release, <code class="docutils literal notranslate">yield</code> will be a language keyword and the <code class="docutils literal notranslate">future</code> statement will no longer be needed. The <code class="docutils literal notranslate">yield</code> statement may only be used inside functions. A function that contains a <code class="docutils literal notranslate">yield</code> statement is called a generator function. A generator function is an ordinary function object in all respects, but has the new <code class="docutils literal notranslate">CO_GENERATOR</code> flag set in the code object’s co_flags member. When a generator function is called, the actual arguments are bound to function-local formal argument names in the usual way, but no code in the body of the function is executed. Instead a generator-iterator object is returned; this conforms to the <a class="pep reference internal" href="../pep-0234/" title="PEP 234 – Iterators">iterator protocol</a>, so in particular can be used in for-loops in a natural way. Note that when the intent is clear from context, the unqualified name “generator” may be used to refer either to a generator-function or a generator-iterator. Each time the <code class="docutils literal notranslate">.next()</code> method of a generator-iterator is invoked, the code in the body of the generator-function is executed until a <code class="docutils literal notranslate">yield</code> or <code class="docutils literal notranslate">return</code> statement (see below) is encountered, or until the end of the body is reached. If a <code class="docutils literal notranslate">yield</code> statement is encountered, the state of the function is frozen, and the value of expression_list is returned to <code class="docutils literal notranslate">.next()</code>’s caller. By “frozen” we mean that all local state is retained, including the current bindings of local variables, the instruction pointer, and the internal evaluation stack: enough information is saved so that the next time <code class="docutils literal notranslate">.next()</code> is invoked, the function can proceed exactly as if the <code class="docutils literal notranslate">yield</code> statement were just another external call. Restriction: A <code class="docutils literal notranslate">yield</code> statement is not allowed in the <code class="docutils literal notranslate">try</code> clause of a <code class="docutils literal notranslate">try/finally</code> construct. The difficulty is that there’s no guarantee the generator will ever be resumed, hence no guarantee that the finally block will ever get executed; that’s too much a violation of finally’s purpose to bear. Restriction: A generator cannot be resumed while it is actively running: <div class="highlight-default notranslate"><div class="highlight"><pre>>>> def g(): ... i = me.next() ... yield i >>> me = g() >>> me.next() Traceback (most recent call last): ... File "<string>", line 2, in g ValueError: generator already executing </pre></div> </div> </section> <section id="specification-return"> <h2><a class="toc-backref" href="#specification-return" role="doc-backlink">Specification: Return</a></h2> A generator function can also contain return statements of the form: <div class="highlight-default notranslate"><div class="highlight"><pre>return </pre></div> </div> Note that an expression_list is not allowed on return statements in the body of a generator (although, of course, they may appear in the bodies of non-generator functions nested within the generator). When a return statement is encountered, control proceeds as in any function return, executing the appropriate <code class="docutils literal notranslate">finally</code> clauses (if any exist). Then a <code class="docutils literal notranslate">StopIteration</code> exception is raised, signalling that the iterator is exhausted. A <code class="docutils literal notranslate">StopIteration</code> exception is also raised if control flows off the end of the generator without an explicit return. Note that return means “I’m done, and have nothing interesting to return”, for both generator functions and non-generator functions. Note that return isn’t always equivalent to raising <code class="docutils literal notranslate">StopIteration</code>: the difference lies in how enclosing <code class="docutils literal notranslate">try/except</code> constructs are treated. For example,: <div class="highlight-default notranslate"><div class="highlight"><pre>>>> def f1(): ... try: ... return ... except: ... yield 1 >>> print list(f1()) [] </pre></div> </div> because, as in any function, <code class="docutils literal notranslate">return</code> simply exits, but: <div class="highlight-default notranslate"><div class="highlight"><pre>>>> def f2(): ... try: ... raise StopIteration ... except: ... yield 42 >>> print list(f2()) [42] </pre></div> </div> because <code class="docutils literal notranslate">StopIteration</code> is captured by a bare <code class="docutils literal notranslate">except</code>, as is any exception. </section> <section id="specification-generators-and-exception-propagation"> <h2><a class="toc-backref" href="#specification-generators-and-exception-propagation" role="doc-backlink">Specification: Generators and Exception Propagation</a></h2> If an unhandled exception– including, but not limited to, <code class="docutils literal notranslate">StopIteration</code> –is raised by, or passes through, a generator function, then the exception is passed on to the caller in the usual way, and subsequent attempts to resume the generator function raise <code class="docutils literal notranslate">StopIteration</code>. In other words, an unhandled exception terminates a generator’s useful life. Example (not idiomatic but to illustrate the point): <div class="highlight-default notranslate"><div class="highlight"><pre>>>> def f(): ... return 1/0 >>> def g(): ... yield f() # the zero division exception propagates ... yield 42 # and we'll never get here >>> k = g() >>> k.next() Traceback (most recent call last): File "<stdin>", line 1, in ? File "<stdin>", line 2, in g File "<stdin>", line 2, in f ZeroDivisionError: integer division or modulo by zero >>> k.next() # and the generator cannot be resumed Traceback (most recent call last): File "<stdin>", line 1, in ? StopIteration >>> </pre></div> </div> </section> <section id="specification-try-except-finally"> <h2><a class="toc-backref" href="#specification-try-except-finally" role="doc-backlink">Specification: Try/Except/Finally</a></h2> As noted earlier, <code class="docutils literal notranslate">yield</code> is not allowed in the <code class="docutils literal notranslate">try</code> clause of a <code class="docutils literal notranslate">try/finally</code> construct. A consequence is that generators should allocate critical resources with great care. There is no restriction on <code class="docutils literal notranslate">yield</code> otherwise appearing in <code class="docutils literal notranslate">finally</code> clauses, <code class="docutils literal notranslate">except</code> clauses, or in the <code class="docutils literal notranslate">try</code> clause of a <code class="docutils literal notranslate">try/except</code> construct: <div class="highlight-default notranslate"><div class="highlight"><pre>>>> def f(): ... try: ... yield 1 ... try: ... yield 2 ... 1/0 ... yield 3 # never get here ... except ZeroDivisionError: ... yield 4 ... yield 5 ... raise ... except: ... yield 6 ... yield 7 # the "raise" above stops this ... except: ... yield 8 ... yield 9 ... try: ... x = 12 ... finally: ... yield 10 ... yield 11 >>> print list(f()) [1, 2, 4, 5, 8, 9, 10, 11] >>> </pre></div> </div> </section> <section id="example"> <h2><a class="toc-backref" href="#example" role="doc-backlink">Example</a></h2> <div class="highlight-python2 notranslate"><div class="highlight"><pre># A binary tree class. class Tree: def __init__(self, label, left=None, right=None): self.label = label self.left = left self.right = right def __repr__(self, level=0, indent=" "): s = level*indent + `self.label` if self.left: s = s + "\n" + self.left.__repr__(level+1, indent) if self.right: s = s + "\n" + self.right.__repr__(level+1, indent) return s def __iter__(self): return inorder(self) # Create a Tree from a list. def tree(list): n = len(list) if n == 0: return [] i = n / 2 return Tree(list[i], tree(list[:i]), tree(list[i+1:])) # A recursive generator that generates Tree labels in in-order. def inorder(t): if t: for x in inorder(t.left): yield x yield t.label for x in inorder(t.right): yield x # Show it off: create a tree. t = tree("ABCDEFGHIJKLMNOPQRSTUVWXYZ") # Print the nodes of the tree in in-order. for x in t: print x, print # A non-recursive generator. def inorder(node): stack = [] while node: while node.left: stack.append(node) node = node.left yield node.label while not node.right: try: node = stack.pop() except IndexError: return yield node.label node = node.right # Exercise the non-recursive generator. for x in t: print x, print </pre></div> </div> Both output blocks display: <div class="highlight-default notranslate"><div class="highlight"><pre>A B C D E F G H I J K L M N O P Q R S T U V W X Y Z </pre></div> </div> </section> <section id="q-a"> <h2><a class="toc-backref" href="#q-a" role="doc-backlink">Q & A</a></h2> <section id="why-not-a-new-keyword-instead-of-reusing-def"> <h3><a class="toc-backref" href="#why-not-a-new-keyword-instead-of-reusing-def" role="doc-backlink">Why not a new keyword instead of reusing <code class="docutils literal notranslate">def</code>?</a></h3> See BDFL Pronouncements section below. </section> <section id="why-a-new-keyword-for-yield-why-not-a-builtin-function-instead"> <h3><a class="toc-backref" href="#why-a-new-keyword-for-yield-why-not-a-builtin-function-instead" role="doc-backlink">Why a new keyword for <code class="docutils literal notranslate">yield</code>? Why not a builtin function instead?</a></h3> Control flow is much better expressed via keyword in Python, and yield is a control construct. It’s also believed that efficient implementation in Jython requires that the compiler be able to determine potential suspension points at compile-time, and a new keyword makes that easy. The CPython reference implementation also exploits it heavily, to detect which functions are generator-functions (although a new keyword in place of <code class="docutils literal notranslate">def</code> would solve that for CPython – but people asking the “why a new keyword?” question don’t want any new keyword). </section> <section id="then-why-not-some-other-special-syntax-without-a-new-keyword"> <h3><a class="toc-backref" href="#then-why-not-some-other-special-syntax-without-a-new-keyword" role="doc-backlink">Then why not some other special syntax without a new keyword?</a></h3> For example, one of these instead of <code class="docutils literal notranslate">yield 3</code>: <div class="highlight-default notranslate"><div class="highlight"><pre>return 3 and continue return and continue 3 return generating 3 continue return 3 return >> , 3 from generator return 3 return >> 3 return << 3 >> 3 << 3 * 3 </pre></div> </div> Did I miss one <wink>? Out of hundreds of messages, I counted three suggesting such an alternative, and extracted the above from them. It would be nice not to need a new keyword, but nicer to make <code class="docutils literal notranslate">yield</code> very clear – I don’t want to have to deduce that a yield is occurring from making sense of a previously senseless sequence of keywords or operators. Still, if this attracts enough interest, proponents should settle on a single consensus suggestion, and Guido will Pronounce on it. </section> <section id="why-allow-return-at-all-why-not-force-termination-to-be-spelled-raise-stopiteration"> <h3><a class="toc-backref" href="#why-allow-return-at-all-why-not-force-termination-to-be-spelled-raise-stopiteration" role="doc-backlink">Why allow <code class="docutils literal notranslate">return</code> at all? Why not force termination to be spelled <code class="docutils literal notranslate">raise StopIteration</code>?</a></h3> The mechanics of <code class="docutils literal notranslate">StopIteration</code> are low-level details, much like the mechanics of <code class="docutils literal notranslate">IndexError</code> in Python 2.1: the implementation needs to do something well-defined under the covers, and Python exposes these mechanisms for advanced users. That’s not an argument for forcing everyone to work at that level, though. <code class="docutils literal notranslate">return</code> means “I’m done” in any kind of function, and that’s easy to explain and to use. Note that <code class="docutils literal notranslate">return</code> isn’t always equivalent to <code class="docutils literal notranslate">raise StopIteration</code> in try/except construct, either (see the “Specification: Return” section). </section> <section id="then-why-not-allow-an-expression-on-return-too"> <h3><a class="toc-backref" href="#then-why-not-allow-an-expression-on-return-too" role="doc-backlink">Then why not allow an expression on <code class="docutils literal notranslate">return</code> too?</a></h3> Perhaps we will someday. In Icon, <code class="docutils literal notranslate">return expr</code> means both “I’m done”, and “but I have one final useful value to return too, and this is it”. At the start, and in the absence of compelling uses for <code class="docutils literal notranslate">return expr</code>, it’s simply cleaner to use <code class="docutils literal notranslate">yield</code> exclusively for delivering values. </section> </section> <section id="bdfl-pronouncements"> <h2><a class="toc-backref" href="#bdfl-pronouncements" role="doc-backlink">BDFL Pronouncements</a></h2> <section id="issue"> <h3><a class="toc-backref" href="#issue" role="doc-backlink">Issue</a></h3> Introduce another new keyword (say, <code class="docutils literal notranslate">gen</code> or <code class="docutils literal notranslate">generator</code>) in place of <code class="docutils literal notranslate">def</code>, or otherwise alter the syntax, to distinguish generator-functions from non-generator functions. </section> <section id="con"> <h3><a class="toc-backref" href="#con" role="doc-backlink">Con</a></h3> In practice (how you think about them), generators are functions, but with the twist that they’re resumable. The mechanics of how they’re set up is a comparatively minor technical issue, and introducing a new keyword would unhelpfully overemphasize the mechanics of how generators get started (a vital but tiny part of a generator’s life). </section> <section id="pro"> <h3><a class="toc-backref" href="#pro" role="doc-backlink">Pro</a></h3> In reality (how you think about them), generator-functions are actually factory functions that produce generator-iterators as if by magic. In this respect they’re radically different from non-generator functions, acting more like a constructor than a function, so reusing <code class="docutils literal notranslate">def</code> is at best confusing. A <code class="docutils literal notranslate">yield</code> statement buried in the body is not enough warning that the semantics are so different. </section> <section id="bdfl"> <h3><a class="toc-backref" href="#bdfl" role="doc-backlink">BDFL</a></h3> <code class="docutils literal notranslate">def</code> it stays. No argument on either side is totally convincing, so I have consulted my language designer’s intuition. It tells me that the syntax proposed in the PEP is exactly right - not too hot, not too cold. But, like the Oracle at Delphi in Greek mythology, it doesn’t tell me why, so I don’t have a rebuttal for the arguments against the PEP syntax. The best I can come up with (apart from agreeing with the rebuttals … already made) is “FUD”. If this had been part of the language from day one, I very much doubt it would have made Andrew Kuchling’s “Python Warts” page. </section> </section> <section id="reference-implementation"> <h2><a class="toc-backref" href="#reference-implementation" role="doc-backlink">Reference Implementation</a></h2> The current implementation, in a preliminary state (no docs, but well tested and solid), is part of Python’s CVS development tree <a class="footnote-reference brackets" href="#id10" id="id4">[5]</a>. Using this requires that you build Python from source. This was derived from an earlier patch by Neil Schemenauer <a class="footnote-reference brackets" href="#id9" id="id5">[4]</a>. </section> <section id="footnotes-and-references"> <h2><a class="toc-backref" href="#footnotes-and-references" role="doc-backlink">Footnotes and References</a></h2> <aside class="footnote-list brackets"> <aside class="footnote brackets" id="id6" role="doc-footnote"> <dt class="label" id="id6">[<a href="#id1">1</a>]</dt> <dd><a class="reference external" href="http://www.stackless.com/">http://www.stackless.com/</a></aside> <aside class="footnote brackets" id="id7" role="doc-footnote"> <dt class="label" id="id7">[<a href="#id2">2</a>]</dt> <dd>“Iteration Abstraction in Sather” Murer, Omohundro, Stoutamire and Szyperski <a class="reference external" href="http://www.icsi.berkeley.edu/~sather/Publications/toplas.html">http://www.icsi.berkeley.edu/~sather/Publications/toplas.html</a></aside> <aside class="footnote brackets" id="id8" role="doc-footnote"> <dt class="label" id="id8">[<a href="#id3">3</a>]</dt> <dd><a class="reference external" href="http://www.cs.arizona.edu/icon/">http://www.cs.arizona.edu/icon/</a></aside> <aside class="footnote brackets" id="id9" role="doc-footnote"> <dt class="label" id="id9">[<a href="#id5">4</a>]</dt> <dd><a class="reference external" href="http://python.ca/nas/python/generator.diff">http://python.ca/nas/python/generator.diff</a></aside> <aside class="footnote brackets" id="id10" role="doc-footnote"> <dt class="label" id="id10">[<a href="#id4">5</a>]</dt> <dd>To experiment with this implementation, check out Python from CVS according to the instructions at <a class="reference external" href="http://sf.net/cvs/?group_id=5470">http://sf.net/cvs/?group_id=5470</a> Note that the std test <code class="docutils literal notranslate">Lib/test/test_generators.py</code> contains many examples, including all those in this PEP.</aside> </aside> </section> <section id="copyright"> <h2><a class="toc-backref" href="#copyright" role="doc-backlink">Copyright</a></h2> This document has been placed in the public domain. </section> </section> <hr class="docutils" /> Source: <a class="reference external" href="https://github.com/python/peps/blob/main/peps/pep-0255.rst">https://github.com/python/peps/blob/main/peps/pep-0255.rst</a> Last modified: <a class="reference external" href="https://github.com/python/peps/commits/main/peps/pep-0255.rst">2025-01-31 10:51:19 GMT</a> </article> <nav id="pep-sidebar"> <h2>Contents</h2> <ul> <li><a class="reference internal" href="#abstract">Abstract</a></li> <li><a class="reference internal" href="#motivation">Motivation</a></li> <li><a class="reference internal" href="#specification-yield">Specification: Yield</a></li> <li><a class="reference internal" href="#specification-return">Specification: Return</a></li> <li><a class="reference internal" href="#specification-generators-and-exception-propagation">Specification: Generators and Exception Propagation</a></li> <li><a class="reference internal" href="#specification-try-except-finally">Specification: Try/Except/Finally</a></li> <li><a class="reference internal" href="#example">Example</a></li> <li><a class="reference internal" href="#q-a">Q & A</a><ul> <li><a class="reference internal" href="#why-not-a-new-keyword-instead-of-reusing-def">Why not a new keyword instead of reusing <code class="docutils literal notranslate">def</code>?</a></li> <li><a class="reference internal" href="#why-a-new-keyword-for-yield-why-not-a-builtin-function-instead">Why a new keyword for <code class="docutils literal notranslate">yield</code>? Why not a builtin function instead?</a></li> <li><a class="reference internal" href="#then-why-not-some-other-special-syntax-without-a-new-keyword">Then why not some other special syntax without a new keyword?</a></li> <li><a class="reference internal" href="#why-allow-return-at-all-why-not-force-termination-to-be-spelled-raise-stopiteration">Why allow <code class="docutils literal notranslate">return</code> at all? Why not force termination to be spelled <code class="docutils literal notranslate">raise StopIteration</code>?</a></li> <li><a class="reference internal" href="#then-why-not-allow-an-expression-on-return-too">Then why not allow an expression on <code class="docutils literal notranslate">return</code> too?</a></li> </ul> </li> <li><a class="reference internal" href="#bdfl-pronouncements">BDFL Pronouncements</a><ul> <li><a class="reference internal" href="#issue">Issue</a></li> <li><a class="reference internal" href="#con">Con</a></li> <li><a class="reference internal" href="#pro">Pro</a></li> <li><a class="reference internal" href="#bdfl">BDFL</a></li> </ul> </li> <li><a class="reference internal" href="#reference-implementation">Reference Implementation</a></li> <li><a class="reference internal" href="#footnotes-and-references">Footnotes and References</a></li> <li><a class="reference internal" href="#copyright">Copyright</a></li> </ul> <a id="source" href="https://github.com/python/peps/blob/main/peps/pep-0255.rst">Page Source (GitHub)</a> </nav> </section> <script src="../_static/colour_scheme.js"></script> <script src="../_static/wrap_tables.js"></script> <script src="../_static/sticky_banner.js"></script> </body> </html>