CINXE.COM

<!DOCTYPE html> <html lang="en"> <head> <meta charset="utf-8"> <meta name="viewport" content="width=device-width, initial-scale=1.0"> <meta name="color-scheme" content="light dark"> <title>PEP 789 – Preventing task-cancellation bugs by limiting yield in async generators | peps.python.org</title> <link rel="shortcut icon" href="../_static/py.png"> <link rel="canonical" href="https://peps.python.org/pep-0789/"> <link rel="stylesheet" href="../_static/style.css" type="text/css"> <link rel="stylesheet" href="../_static/mq.css" type="text/css"> <link rel="stylesheet" href="../_static/pygments.css" type="text/css" media="(prefers-color-scheme: light)" id="pyg-light"> <link rel="stylesheet" href="../_static/pygments_dark.css" type="text/css" media="(prefers-color-scheme: dark)" id="pyg-dark"> <link rel="alternate" type="application/rss+xml" title="Latest PEPs" href="https://peps.python.org/peps.rss"> <meta property="og:title" content='PEP 789 – Preventing task-cancellation bugs by limiting yield in async generators | peps.python.org'> <meta property="og:description" content="Structured concurrency is increasingly popular in Python. Interfaces such as the asyncio.TaskGroup and asyncio.timeout context managers support compositional reasoning, and allow developers to clearly scope the lifetimes of concurrent tasks. However, u..."> <meta property="og:type" content="website"> <meta property="og:url" content="https://peps.python.org/pep-0789/"> <meta property="og:site_name" content="Python Enhancement Proposals (PEPs)"> <meta property="og:image" content="https://peps.python.org/_static/og-image.png"> <meta property="og:image:alt" content="Python PEPs"> <meta property="og:image:width" content="200"> <meta property="og:image:height" content="200"> <meta name="description" content="Structured concurrency is increasingly popular in Python. Interfaces such as the asyncio.TaskGroup and asyncio.timeout context managers support compositional reasoning, and allow developers to clearly scope the lifetimes of concurrent tasks. However, u..."> <meta name="theme-color" content="#3776ab"> </head> <body> <svg xmlns="http://www.w3.org/2000/svg" style="display: none;"> <symbol id="svg-sun-half" viewBox="0 0 24 24" pointer-events="all"> <title>Following system colour scheme</title> <svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"> <circle cx="12" cy="12" r="9"></circle> <path d="M12 3v18m0-12l4.65-4.65M12 14.3l7.37-7.37M12 19.6l8.85-8.85"></path> </svg> </symbol> <symbol id="svg-moon" viewBox="0 0 24 24" pointer-events="all"> <title>Selected dark colour scheme</title> <svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"> <path stroke="none" d="M0 0h24v24H0z" fill="none"></path> <path d="M12 3c.132 0 .263 0 .393 0a7.5 7.5 0 0 0 7.92 12.446a9 9 0 1 1 -8.313 -12.454z"></path> </svg> </symbol> <symbol id="svg-sun" viewBox="0 0 24 24" pointer-events="all"> <title>Selected light colour scheme</title> <svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"> <circle cx="12" cy="12" r="5"></circle> <line x1="12" y1="1" x2="12" y2="3"></line> <line x1="12" y1="21" x2="12" y2="23"></line> <line x1="4.22" y1="4.22" x2="5.64" y2="5.64"></line> <line x1="18.36" y1="18.36" x2="19.78" y2="19.78"></line> <line x1="1" y1="12" x2="3" y2="12"></line> <line x1="21" y1="12" x2="23" y2="12"></line> <line x1="4.22" y1="19.78" x2="5.64" y2="18.36"></line> <line x1="18.36" y1="5.64" x2="19.78" y2="4.22"></line> </svg> </symbol> </svg> <script> document.documentElement.dataset.colour_scheme = localStorage.getItem("colour_scheme") || "auto" </script> <section id="pep-page-section"> <header> <h1>Python Enhancement Proposals</h1> <ul class="breadcrumbs"> <li><a href="https://www.python.org/" title="The Python Programming Language">Python</a> » </li> <li><a href="../pep-0000/">PEP Index</a> » </li> <li>PEP 789</li> </ul> <button id="colour-scheme-cycler" onClick="setColourScheme(nextColourScheme())"> <svg aria-hidden="true" class="colour-scheme-icon-when-auto"><use href="#svg-sun-half"></use></svg> <svg aria-hidden="true" class="colour-scheme-icon-when-dark"><use href="#svg-moon"></use></svg> <svg aria-hidden="true" class="colour-scheme-icon-when-light"><use href="#svg-sun"></use></svg> Toggle light / dark / auto colour theme </button> </header> <article> <section id="pep-content"> <h1 class="page-title">PEP 789 – Preventing task-cancellation bugs by limiting yield in async generators</h1> <dl class="rfc2822 field-list simple"> <dt class="field-odd">Author:</dt> <dd class="field-odd">Zac Hatfield-Dodds <zac at zhd.dev>, Nathaniel J. Smith <njs at pobox.com></dd> <dt class="field-even">PEP-Delegate:</dt> <dd class="field-even"></dd> <dt class="field-odd">Discussions-To:</dt> <dd class="field-odd"><a class="reference external" href="https://discuss.python.org/t/preventing-yield-inside-certain-context-managers/1091">Discourse thread</a></dd> <dt class="field-even">Status:</dt> <dd class="field-even"><abbr title="Proposal under active discussion and revision">Draft</abbr></dd> <dt class="field-odd">Type:</dt> <dd class="field-odd"><abbr title="Normative PEP with a new feature for Python, implementation change for CPython or interoperability standard for the ecosystem">Standards Track</abbr></dd> <dt class="field-even">Created:</dt> <dd class="field-even">14-May-2024</dd> <dt class="field-odd">Python-Version:</dt> <dd class="field-odd">3.14</dd> </dl> <hr class="docutils" /> <section id="contents"> <details><summary>Table of Contents</summary><ul class="simple"> <li><a class="reference internal" href="#abstract">Abstract</a></li> <li><a class="reference internal" href="#background">Background</a></li> <li><a class="reference internal" href="#problem-statement">Problem statement</a></li> <li><a class="reference internal" href="#motivating-examples">Motivating examples</a><ul> <li><a class="reference internal" href="#leaking-a-timeout-to-the-outer-scope">Leaking a timeout to the outer scope</a></li> <li><a class="reference internal" href="#leaking-background-tasks-breaks-cancellation-and-exception-handling">Leaking background tasks (breaks cancellation and exception handling)</a></li> <li><a class="reference internal" href="#in-a-user-defined-context-manager">In a user-defined context manager</a></li> </ul> </li> <li><a class="reference internal" href="#specification">Specification</a><ul> <li><a class="reference internal" href="#implementation-tracking-frames">Implementation - tracking frames</a></li> <li><a class="reference internal" href="#worked-examples">Worked examples</a><ul> <li><a class="reference internal" href="#no-yield-example">No-yield example</a></li> <li><a class="reference internal" href="#attempts-to-yield-example">Attempts-to-yield example</a></li> <li><a class="reference internal" href="#allowed-to-yield-example">Allowed-to-yield example</a></li> <li><a class="reference internal" href="#allowing-yield-for-context-managers">Allowing yield for context managers</a></li> </ul> </li> <li><a class="reference internal" href="#behavior-if-sys-prevent-yields-is-misused">Behavior if <code class="docutils literal notranslate">sys.prevent_yields</code> is misused</a></li> </ul> </li> <li><a class="reference internal" href="#anticipated-uses">Anticipated uses</a></li> <li><a class="reference internal" href="#backwards-compatibility">Backwards Compatibility</a><ul> <li><a class="reference internal" href="#how-widespread-is-this-bug">How widespread is this bug?</a></li> </ul> </li> <li><a class="reference internal" href="#how-to-teach-this">How to Teach This</a></li> <li><a class="reference internal" href="#rejected-alternatives">Rejected alternatives</a><ul> <li><a class="reference internal" href="#pep-533-deterministic-cleanup-for-iterators">PEP 533, deterministic cleanup for iterators</a></li> <li><a class="reference internal" href="#deprecate-async-generators-entirely">Deprecate async generators entirely</a></li> <li><a class="reference internal" href="#can-t-we-just-deliver-exceptions-to-the-right-place">Can’t we just deliver exceptions to the right place?</a></li> <li><a class="reference internal" href="#alternative-implementation-inspecting-bytecode">Alternative implementation - inspecting bytecode</a></li> </ul> </li> <li><a class="reference internal" href="#footnotes">Footnotes</a></li> <li><a class="reference internal" href="#copyright">Copyright</a></li> </ul> </details></section> <section id="abstract"> <h2><a class="toc-backref" href="#abstract" role="doc-backlink">Abstract</a></h2> <a class="reference external" href="https://vorpus.org/blog/notes-on-structured-concurrency-or-go-statement-considered-harmful/">Structured concurrency</a> is increasingly popular in Python. Interfaces such as the <code class="docutils literal notranslate">asyncio.TaskGroup</code> and <code class="docutils literal notranslate">asyncio.timeout</code> context managers support compositional reasoning, and allow developers to clearly scope the lifetimes of concurrent tasks. However, using <code class="docutils literal notranslate">yield</code> to suspend a frame inside such a context leads to situations where the wrong task is canceled, timeouts are ignored, and exceptions are mishandled. More fundamentally, suspending a frame inside a <code class="docutils literal notranslate">TaskGroup</code> violates the structured concurrency design principle that child tasks are encapsulated within their parent frame. To address these issues, this PEP proposes a new <code class="docutils literal notranslate">sys.prevent_yields()</code> context manager. When syntactically inside this context, attempting to <code class="docutils literal notranslate">yield</code> will raise a RuntimeError, preventing the task from yielding. Additionally, a mechanism will be provided for decorators such as <code class="docutils literal notranslate">@contextmanager</code> to allow yields inside the decorated function. <code class="docutils literal notranslate">sys.prevent_yields()</code> will be used by asyncio and downstream libraries to implement task groups, timeouts, and cancellation; and a related mechanism by <code class="docutils literal notranslate">contextlib</code> etc. to convert generators into context managers which allow safe yields. </section> <section id="background"> <h2><a class="toc-backref" href="#background" role="doc-backlink">Background</a></h2> Structured concurrency is increasingly popular in Python, in the form of newer <a class="reference external" href="https://docs.python.org/3/library/asyncio.html#module-asyncio" title="(in Python v3.13)"><code class="xref py py-mod docutils literal notranslate">asyncio</code></a> interfaces and third-party libraries such as Trio and anyio. These interfaces support compositional reasoning, so long as users never write a <code class="docutils literal notranslate">yield</code> which suspends a frame while inside a cancel scope. A cancel scope is a context manager which can… cancel… whatever work occurs within that context (…scope). In asyncio, this is implicit in the design of <code class="docutils literal notranslate">with asyncio.timeout():</code> or <code class="docutils literal notranslate">async with asyncio.TaskGroup() as tg:</code>, which respectively cancel the contained work after the specified duration, or cancel sibling tasks when one of them raises an exception. The core functionality of a cancel scope is synchronous, but the user-facing context managers may be either sync or async. <a class="footnote-reference brackets" href="#trio-cancel-scope" id="id1">[1]</a> <a class="footnote-reference brackets" href="#tg-cs" id="id2">[2]</a> This structured approach works beautifully, unless you hit one specific sharp edge: breaking the nesting structure by <code class="docutils literal notranslate">yield</code>ing inside a cancel scope. This has much the same effect on structured control flow as adding just a few cross-function <code class="docutils literal notranslate">goto</code>s, and the effects are truly dire: <ul class="simple"> <li>The wrong task can be canceled, whether due to a timeout, an error in a sibling task, or an explicit request to cancel some other task</li> <li>Exceptions, including <code class="docutils literal notranslate">CancelledError</code>, can be delivered to the wrong task</li> <li>Exceptions can go missing entirely, being dropped instead of added to an <code class="docutils literal notranslate">ExceptionGroup</code></li> </ul> </section> <section id="problem-statement"> <h2><a class="toc-backref" href="#problem-statement" role="doc-backlink">Problem statement</a></h2> Here’s the fundamental issue: yield suspends a call frame. It only makes sense to yield in a leaf frame – i.e., if your call stack goes like A -> B -> C, then you can suspend C, but you can’t suspend B while leaving C running. But, TaskGroup is a kind of “concurrent call” primitive, where a single frame can have multiple child frames that run concurrently. This means that if we allow people to mix yield and TaskGroup, then we can end up in exactly this situation, where B gets suspended but C is actively running. This is nonsensical, and causes serious practical problems (e.g., if C raises an exception and A has returned, we have no way to propagate it). This is a fundamental incompatibility between generator control flow and structured concurrency control flow, not something we can fix by tweaking our APIs. The only solution seems to be to forbid yield inside a TaskGroup. Although timeouts don’t leave a child task running, the close analogy and related problems lead us to conclude that yield should be forbidden inside all cancel scopes, not only TaskGroups. See <a class="reference internal" href="#just-deliver">Can’t we just deliver exceptions to the right place?</a> for discussion. </section> <section id="motivating-examples"> <h2><a class="toc-backref" href="#motivating-examples" role="doc-backlink">Motivating examples</a></h2> Let’s consider three examples, to see what this might look like in practice. <section id="leaking-a-timeout-to-the-outer-scope"> <h3><a class="toc-backref" href="#leaking-a-timeout-to-the-outer-scope" role="doc-backlink">Leaking a timeout to the outer scope</a></h3> Suppose that we want to iterate over an async iterator, but wait for at most <code class="docutils literal notranslate">max_time</code> seconds for each element. We might naturally encapsulate the logic for doing so in an async generator, so that the call site can continue to use a straightforward <code class="docutils literal notranslate">async for</code> loop: <div class="highlight-python notranslate"><div class="highlight"><pre>async def iter_with_timeout(ait, max_time): try: while True: with timeout(max_time): yield await anext(ait) except StopAsyncIteration: return async def fn(): async for elem in iter_with_timeout(ait, max_time=1.0): await do_something_with(elem) </pre></div> </div> Unfortunately, there’s a bug in this version: the timeout might expire after the generator yields but before it is resumed! In this case, we’ll see a <code class="docutils literal notranslate">CancelledError</code> raised in the outer task, where it cannot be caught by the <code class="docutils literal notranslate">with timeout(max_time):</code> statement. The fix is fairly simple: get the next element inside the timeout context, and then yield outside that context. <div class="highlight-python notranslate"><div class="highlight"><pre>async def correct_iter_with_timeout(ait, max_time): try: while True: with timeout(max_time): tmp = await anext(ait) yield tmp except StopAsyncIteration: return </pre></div> </div> </section> <section id="leaking-background-tasks-breaks-cancellation-and-exception-handling"> <h3><a class="toc-backref" href="#leaking-background-tasks-breaks-cancellation-and-exception-handling" role="doc-backlink">Leaking background tasks (breaks cancellation and exception handling)</a></h3> Timeouts are not the only interface which wrap a cancel scope - and if you need some background worker tasks, you can’t simply close the <code class="docutils literal notranslate">TaskGroup</code> before yielding. As an example, let’s look at a fan-in generator, which we’ll use to merge the feeds from several “sensors”. We’ll also set up our mock sensors with a small buffer, so that we’ll raise an error in the background task while control flow is outside the <code class="docutils literal notranslate">combined_iterators</code> generator. <div class="highlight-python notranslate"><div class="highlight"><pre>import asyncio, itertools async def mock_sensor(name): for n in itertools.count(): await asyncio.sleep(0.1) if n == 1 and name == "b": # 'presence detection' yield "PRESENT" elif n == 3 and name == "a": # inject a simple bug print("oops, raising RuntimeError") raise RuntimeError else: yield f"{name}-{n}" # non-presence sensor data async def move_elements_to_queue(ait, queue): async for obj in ait: await queue.put(obj) async def combined_iterators(*aits): """Combine async iterators by starting N tasks, each of which move elements from one iterable to a shared queue.""" q = asyncio.Queue(maxsize=2) async with asyncio.TaskGroup() as tg: for ait in aits: tg.create_task(move_elements_to_queue(ait, q)) while True: yield await q.get() async def turn_on_lights_when_someone_gets_home(): combined = combined_iterators(mock_sensor("a"), mock_sensor("b")) async for event in combined: print(event) if event == "PRESENT": break print("main task sleeping for a bit") await asyncio.sleep(1) # do some other operation asyncio.run(turn_on_lights_when_someone_gets_home()) </pre></div> </div> When we run this code, we see the expected sequence of observations, then a ‘detection’, and then while the main task is sleeping we trigger that <code class="docutils literal notranslate">RuntimeError</code> in the background. But… we don’t actually observe the <code class="docutils literal notranslate">RuntimeError</code>, not even as the <code class="docutils literal notranslate">__context__</code> of another exception! <div class="highlight-pycon notranslate"><div class="highlight"><pre>>> python3.11 demo.py a-0 b-0 a-1 PRESENT main task sleeping for a bit oops, raising RuntimeError Traceback (most recent call last): File "demo.py", line 39, in <module> asyncio.run(turn_on_lights_when_someone_gets_home()) ... File "demo.py", line 37, in turn_on_lights_when_someone_gets_home await asyncio.sleep(1) # do some other operation File ".../python3.11/asyncio/tasks.py", line 649, in sleep return await future asyncio.exceptions.CancelledError </pre></div> </div> Here, again, the problem is that we’ve <code class="docutils literal notranslate">yield</code>ed inside a cancel scope; this time the scope which a <code class="docutils literal notranslate">TaskGroup</code> uses to cancel sibling tasks when one of the child tasks raises an exception. However, the <code class="docutils literal notranslate">CancelledError</code> which was intended for the sibling task was instead injected into the outer task, and so we never got a chance to create and raise an <code class="docutils literal notranslate">ExceptionGroup(..., [RuntimeError()])</code>. To fix this, we need to turn our async generator into an async context manager, which yields an async iterable - in this case a generator wrapping the queue; in future <a class="reference external" href="https://github.com/python/cpython/issues/119154">perhaps the queue itself</a>: <div class="highlight-python notranslate"><div class="highlight"><pre>async def queue_as_aiterable(queue): # async generators that don't `yield` inside a cancel scope are fine! while True: try: yield await queue.get() except asyncio.QueueShutDown: return @asynccontextmanager # yield-in-cancel-scope is OK in a context manager async def combined_iterators(*aits): q = asyncio.Queue(maxsize=2) async with asyncio.TaskGroup() as tg: for ait in aits: tg.create_task(move_elements_to_queue(ait, q)) yield queue_as_aiterable(q) async def turn_on_lights_when_someone_gets_home(): ... async with combined_iterators(...) as ait: async for event in ait: ... </pre></div> </div> </section> <section id="in-a-user-defined-context-manager"> <h3><a class="toc-backref" href="#in-a-user-defined-context-manager" role="doc-backlink">In a user-defined context manager</a></h3> Yielding inside a cancel scope can be safe, if and only if you’re using the generator to implement a context manager <a class="footnote-reference brackets" href="#redirected" id="id3">[3]</a> - in this case any propagating exceptions will be redirected to the expected task. We’ve also implemented the <code class="docutils literal notranslate">ASYNC101</code> linter rule in <a class="reference external" href="https://pypi.org/project/flake8-async/">flake8-async</a>, which warns against yielding in known cancel scopes. Could user education be sufficient to avoid these problems? Unfortunately not: user-defined context managers can also wrap a cancel scope, and it’s infeasible to recognize or lint for all such cases. This regularly arises in practice, because ‘run some background tasks for the duration of this context’ is a very common pattern in structured concurrency. We saw that in <code class="docutils literal notranslate">combined_iterators()</code> above; and have seen this bug in multiple implementations of the websocket protocol: <div class="highlight-python notranslate"><div class="highlight"><pre>async def get_messages(websocket_url): # The websocket protocol requires background tasks to manage the socket heartbeat async with open_websocket(websocket_url) as ws: # contains a TaskGroup! while True: yield await ws.get_message() async with open_websocket(websocket_url) as ws: async for message in get_messages(ws): ... </pre></div> </div> </section> </section> <section id="specification"> <h2><a class="toc-backref" href="#specification" role="doc-backlink">Specification</a></h2> To prevent these problems, we propose: <ol class="arabic simple"> <li>a new context manager, <code class="docutils literal notranslate">with sys.prevent_yields(reason): ...</code> which will raise a RuntimeError if you attempt to yield while inside it. <a class="footnote-reference brackets" href="#also-sync" id="id4">[4]</a> Cancel-scope-like context managers in asyncio and downstream code can then wrap this to prevent yielding inside their with-block.</li> <li>a mechanism by which generator-to-context-manager decorators can allow yields across one call. We’re not yet sure what this should look like; the leading candidates are:<ol class="loweralpha simple"> <li>a code-object attribute, <code class="docutils literal notranslate">fn.__code__.co_allow_yields = True</code>, or</li> <li>some sort of invocation flag, e.g. <code class="docutils literal notranslate">fn.__invoke_with_yields__</code>, to avoid mutating a code object that might be shared between decorated and undecorated functions</li> </ol> </li> </ol> <section id="implementation-tracking-frames"> <h3><a class="toc-backref" href="#implementation-tracking-frames" role="doc-backlink">Implementation - tracking frames</a></h3> The new <code class="docutils literal notranslate">sys.prevent_yields</code> context manager will require interpreter support. For each frame, we track the entries and exits of this context manager. We’re not particularly attached to the exact representation; we’ll discuss it as a stack (which would support clear error messages), but more compact representations such as pair-of-integers would also work. <ul class="simple"> <li>When entering a newly-created or resumed frame, initialize empty stacks of entries and exits.</li> <li>When returning from a frame, merge these stacks into that of the parent frame.</li> <li>When yielding:<ul> <li>if <code class="docutils literal notranslate">entries != [] and not frame.allow_yield_flag</code>, raise a <code class="docutils literal notranslate">RuntimeError</code> instead of yielding (the new behavior this PEP proposes)</li> <li>otherwise, merge stacks into the parent frame as for a return.</li> </ul> </li> </ul> Because this is about yielding frames within a task, not switching between tasks, syntactic <code class="docutils literal notranslate">yield</code> and <code class="docutils literal notranslate">yield from</code> should be affected, but <code class="docutils literal notranslate">await</code> expressions should not. We can reduce the overhead by storing this metadata in a single stack per thread for all stack frames which are not generators. </section> <section id="worked-examples"> <h3><a class="toc-backref" href="#worked-examples" role="doc-backlink">Worked examples</a></h3> <section id="no-yield-example"> <h4><a class="toc-backref" href="#no-yield-example" role="doc-backlink">No-yield example</a></h4> In this example, we see multiple rounds of the stack merging as we unwind from <code class="docutils literal notranslate">sys.prevent_yields</code>, through the user-defined ContextManager, back to the original Frame. For brevity, the reason for preventing yields is not shown; it is part of the “1 enter” state. <a class="reference internal image-reference" href="../_images/pep-789-example-no-yield.png"><img alt="../_images/pep-789-example-no-yield.png" class="align-center" src="../_images/pep-789-example-no-yield.png" style="width: 600px;" /> </a> With no <code class="docutils literal notranslate">yield</code> we don’t raise any errors, and because the number of enters and exits balance the frame returns as usual with no further tracking. </section> <section id="attempts-to-yield-example"> <h4><a class="toc-backref" href="#attempts-to-yield-example" role="doc-backlink">Attempts-to-yield example</a></h4> In this example, the Frame attempts to <code class="docutils literal notranslate">yield</code> while inside the <code class="docutils literal notranslate">sys.prevent_yields</code> context. This is detected by the interpreter, which raises a <code class="docutils literal notranslate">RuntimeError</code> instead of suspending the frame. <a class="reference internal image-reference" href="../_images/pep-789-example-yield-errors.png"><img alt="../_images/pep-789-example-yield-errors.png" class="align-center" src="../_images/pep-789-example-yield-errors.png" style="width: 500px;" /> </a> </section> <section id="allowed-to-yield-example"> <h4><a class="toc-backref" href="#allowed-to-yield-example" role="doc-backlink">Allowed-to-yield example</a></h4> In this example, a decorator has marked the Frame as allowing yields. This could be <code class="docutils literal notranslate">@contextlib.contextmanager</code> or a related decorator. <a class="reference internal image-reference" href="../_images/pep-789-example-yield-allowed.png"><img alt="../_images/pep-789-example-yield-allowed.png" class="align-center" src="../_images/pep-789-example-yield-allowed.png" style="width: 600px;" /> </a> When the Frame is allowed to yield, the entry/exit stack is merged into the parent frame’s stack before suspending. When the Frame resumes, its stack is empty. Finally, when the Frame exits, the exit is merged into the parent frame’s stack, rebalancing it. This ensures that the parent frame correctly inherits any remaining <code class="docutils literal notranslate">sys.prevent_yields</code> state, while allowing the Frame to safely suspend and resume. </section> <section id="allowing-yield-for-context-managers"> <h4><a class="toc-backref" href="#allowing-yield-for-context-managers" role="doc-backlink">Allowing yield for context managers</a></h4> TODO: this section is a placeholder, pending a decision on the mechanism for ``@contextmanager`` to re-enable yields in the wrapped function. <ul class="simple"> <li>Explain and show a code sample of how <code class="docutils literal notranslate">@asynccontextmanager</code> sets the flag</li> </ul> Note that third-party decorators such as <code class="docutils literal notranslate">@pytest.fixture</code> demonstrate that we can’t just have the interpreter special-case contextlib. </section> </section> <section id="behavior-if-sys-prevent-yields-is-misused"> <h3><a class="toc-backref" href="#behavior-if-sys-prevent-yields-is-misused" role="doc-backlink">Behavior if <code class="docutils literal notranslate">sys.prevent_yields</code> is misused</a></h3> While unwise, it’s possible to call <code class="docutils literal notranslate">sys.prevent_yields.__enter__</code> and <code class="docutils literal notranslate">.__exit__</code> in an order that does not correspond to any valid nesting, or get an invalid frame state in some other way. There are two ways <code class="docutils literal notranslate">sys.prevent_yields.__exit__</code> could detect an invalid state. First, if yields are not prevented, we can simply raise an exception without changing the state. Second, if an unexpected entry is at the top of the stack, we suggest popping that entry and raising an exception – this ensures that out-of-order calls will still clear the stack, while still making it clear that something is wrong. (and if we choose e.g. an integer- rather than stack-based representation, such states may not be distinguishable from correct nesting at all, in which case the question will not arise) </section> </section> <section id="anticipated-uses"> <h2><a class="toc-backref" href="#anticipated-uses" role="doc-backlink">Anticipated uses</a></h2> In the standard library, <code class="docutils literal notranslate">sys.prevent_yields</code> could be used by <code class="docutils literal notranslate">asyncio.TaskGroup</code>, <code class="docutils literal notranslate">asyncio.timeout</code>, and <code class="docutils literal notranslate">asyncio.timeout_at</code>. Downstream, we expect to use it in <code class="docutils literal notranslate">trio.CancelScope</code>, async fixtures (in pytest-trio, anyio, etc.), and perhaps other places. We consider use-cases unrelated to async correctness, such as preventing <code class="docutils literal notranslate">decimal.localcontext</code> from leaking out of a generator, out of scope for this PEP. The generator-to-context-manager support would be used by <code class="docutils literal notranslate">@contextlib.(async)contextmanager</code>, and if necessary in <code class="docutils literal notranslate">(Async)ExitStack</code>. </section> <section id="backwards-compatibility"> <h2><a class="toc-backref" href="#backwards-compatibility" role="doc-backlink">Backwards Compatibility</a></h2> The addition of the <code class="docutils literal notranslate">sys.prevent_yields</code> context manager, changes to <code class="docutils literal notranslate">@contextlib.(async)contextmanager</code>, and corresponding interpreter support are all fully backwards-compatible. Preventing yields inside <code class="docutils literal notranslate">asyncio.TaskGroup</code>, <code class="docutils literal notranslate">asycio.timeout</code>, and <code class="docutils literal notranslate">asyncio.timeout_at</code> would be a breaking change to at least some code in the wild, which (however unsafe and prone to the motivating problems above) may work often enough to make it into production. We will seek community feedback on appropriate deprecation pathways for standard-library code, including the suggested length of any deprecation period. As an initial suggestion, we could make suspending stdlib contexts emit a DeprecationWarning only under asyncio debug mode in 3.14; then transition to warn-by-default and error under debug mode in 3.15; and finally a hard error in 3.16. Irrespective of stdlib usage, downstream frameworks would adopt this functionality immediately. <section id="how-widespread-is-this-bug"> <h3><a class="toc-backref" href="#how-widespread-is-this-bug" role="doc-backlink">How widespread is this bug?</a></h3> We don’t have solid numbers here, but believe that many projects are affected in the wild. Since hitting a moderate and a critical bug attributed to suspending a cancel scope in the same week at work, we’ve <a class="reference external" href="https://flake8-async.readthedocs.io/en/latest/">used static analysis</a> with some success. Three people Zac spoke to at PyCon recognized the symptoms and concluded that they had likely been affected. TODO: run the ASYNC101 lint rule across ecosystem projects, e.g. the aio-libs packages, and get some sense of frequency in widely-used PyPI packages? This would help inform the break/deprecation pathways for stdlib code. </section> </section> <section id="how-to-teach-this"> <h2><a class="toc-backref" href="#how-to-teach-this" role="doc-backlink">How to Teach This</a></h2> Async generators are very rarely taught to novice programmers. Most intermediate and advanced Python programmers will only interact with this PEP as users of <code class="docutils literal notranslate">TaskGroup</code>, <code class="docutils literal notranslate">timeout</code>, and <code class="docutils literal notranslate">@contextmanager</code>. For this group, we expect a clear exception message and documentation to be sufficient. <ul class="simple"> <li>A new section will be added to the <a class="reference external" href="https://docs.python.org/3/library/asyncio-dev.html">developing with asyncio</a> page, which briefly states that async generators are not permitted to <code class="docutils literal notranslate">yield</code> when inside a “cancel scope” context, i.e. <code class="docutils literal notranslate">TaskGroup</code> or <code class="docutils literal notranslate">timeout</code> context manager. We anticipate that the problem-restatement and some parts of the motivation section will provide a basis for these docs.<ul> <li>When working in codebases which avoid async generators entirely <a class="footnote-reference brackets" href="#exp-report" id="id5">[5]</a>, we’ve found that an async context manager yielding an async iterable is a safe and ergonomic replacement for async generators – and avoids the delayed-cleanup problems described in <a class="pep reference internal" href="../pep-0533/" title="PEP 533 – Deterministic cleanup for iterators">PEP 533</a>, which this proposal does not address.</li> </ul> </li> <li>In the docs for each context manager which wraps a cancel scope, and thus now <code class="docutils literal notranslate">sys.prevent_yields</code>, include a standard sentence such as “If used within an async generator, [it is an error to <code class="docutils literal notranslate">yield</code> inside this context manager].” with a hyperlink to the explanation above.</li> </ul> For asyncio, Trio, curio, or other-framework maintainers who implement cancel scope semantics, we will ensure that the documentation of <code class="docutils literal notranslate">sys.prevent_yields</code> gives a full explanation distilled from the solution and implementation sections of this PEP. We anticipate consulting most such maintainers for their feedback on the draft PEP. </section> <section id="rejected-alternatives"> <h2><a class="toc-backref" href="#rejected-alternatives" role="doc-backlink">Rejected alternatives</a></h2> <section id="pep-533-deterministic-cleanup-for-iterators"> <h3><a class="toc-backref" href="#pep-533-deterministic-cleanup-for-iterators" role="doc-backlink">PEP 533, deterministic cleanup for iterators</a></h3> <a class="pep reference internal" href="../pep-0533/" title="PEP 533 – Deterministic cleanup for iterators">PEP 533</a> proposes adding <code class="docutils literal notranslate">__[a]iterclose__</code> to the iterator protocol, essentially wrapping a <code class="docutils literal notranslate">with [a]closing(ait)</code> around each (async) for loop. While this would be useful for ensuring timely and deterministic cleanup of resources held by iterators, the problem it aims to solve, it does not fully address the issues that motivate this PEP. Even with PEP 533, misfired cancellations would still be delivered to the wrong task and could wreak havoc before the iterator is closed. Moreover, it does not address the fundamental structured concurrency problem with <code class="docutils literal notranslate">TaskGroup</code>, where suspending a frame that owns a TaskGroup is incompatible with the model of child tasks being fully encapsulated within their parent frame. </section> <section id="deprecate-async-generators-entirely"> <h3><a class="toc-backref" href="#deprecate-async-generators-entirely" role="doc-backlink">Deprecate async generators entirely</a></h3> At the 2024 language summit, several attendees suggested instead deprecating async generators in toto. Unfortunately, while the common-in-practice cases all use async generators, Trio code can trigger the same problem with standard generators: <div class="highlight-python notranslate"><div class="highlight"><pre># We use Trio for this example, because while `asyncio.timeout()` is async, # Trio's CancelScope type and timeout context managers are synchronous. import trio def abandon_each_iteration_after(max_seconds): # This is of course broken, but I can imagine someone trying it... while True: with trio.move_on_after(max_seconds): yield @trio.run async def main(): for _ in abandon_each_iteration_after(max_seconds=1): await trio.sleep(3) </pre></div> </div> If it wasn’t for the bug in question, this code would look pretty idiomatic - but after about a second, instead of moving on to the next iteration it raises: <div class="highlight-pycon notranslate"><div class="highlight"><pre>Traceback (most recent call last): File "demo.py", line 10, in <module> async def main(): File "trio/_core/_run.py", line 2297, in run raise runner.main_task_outcome.error File "demo.py", line 12, in main await trio.sleep(3) File "trio/_timeouts.py", line 87, in sleep await sleep_until(trio.current_time() + seconds) ... File "trio/_core/_run.py", line 1450, in raise_cancel raise Cancelled._create() trio.Cancelled: Cancelled </pre></div> </div> Furthermore, there are some non-cancel-scope synchronous context managers which exhibit related problems, such as the abovementioned <code class="docutils literal notranslate">decimal.localcontext</code>. While fixing the example below is not a goal of this PEP, it demonstrates that yield-within-with problems are not exclusive to async generators: <div class="highlight-python notranslate"><div class="highlight"><pre>import decimal def why_would_you_do_this(): with decimal.localcontext(decimal.Context(prec=1)): yield one = decimal.Decimal(1) print(one / 3) # 0.3333333333333333333333333333 next(gen := why_would_you_do_this()) print(one / 3) # 0.3 </pre></div> </div> While I’ve had good experiences in async Python without async generators <a class="footnote-reference brackets" href="#exp-report" id="id6">[5]</a>, I’d prefer to fix the problem than remove them from the language. </section> <section id="can-t-we-just-deliver-exceptions-to-the-right-place"> <h3><a class="toc-backref" href="#can-t-we-just-deliver-exceptions-to-the-right-place" role="doc-backlink">Can’t we just deliver exceptions to the right place?</a></h3> If we implemented <a class="pep reference internal" href="../pep-0568/" title="PEP 568 – Generator-sensitivity for Context Variables">PEP 568</a> (Generator-sensitivity for Context Variables; see also <a class="pep reference internal" href="../pep-0550/" title="PEP 550 – Execution Context">PEP 550</a>), it would be possible to handle exceptions from timeouts: the event loop could avoid firing a <code class="docutils literal notranslate">CancelledError</code> until the generator frame which contains the context manager is on the stack - either when the generator is resumed, or when it is finalized. This can take arbitrarily long; even if we implemented <a class="pep reference internal" href="../pep-0533/" title="PEP 533 – Deterministic cleanup for iterators">PEP 533</a> to ensure timely cleanup on exiting (async) for-loops it’s still possible to drive a generator manually with next/send. However, this doesn’t address the other problem with <code class="docutils literal notranslate">TaskGroup</code>. The model for generators is that you put a stack frame in suspended animation and can then treat it as an inert value which can be stored, moved around, and maybe discarded or revived in some arbitrary place. The model for structured concurrency is that your stack becomes a tree, with child tasks encapsulated within some parent frame. They’re extending the basic structured programming model in different, and unfortunately incompatible, directions. Suppose for example that suspending a frame containing an open <code class="docutils literal notranslate">TaskGroup</code> also suspended all child tasks. This would preserve the ‘downward’ structured concurrency, in that children remain encapsulated - albeit at the cost of deadlocking both of our motivating examples, and much real-world code. However, it would still be possible to resume the generator in a different task, violating the ‘upwards’ invariant of structured concurrency. We don’t think it’s worth adding this much machinery to handle cancel scopes, while still leaving task groups broken. </section> <section id="alternative-implementation-inspecting-bytecode"> <h3><a class="toc-backref" href="#alternative-implementation-inspecting-bytecode" role="doc-backlink">Alternative implementation - inspecting bytecode</a></h3> Jelle Zijlstra has <a class="reference external" href="https://gist.github.com/JelleZijlstra/a53b17417c5189b487316628acc5555f">sketched an alternative</a>, where <code class="docutils literal notranslate">sys.prevent_yields</code> inspects the bytecode of callers until satisfied that there is no yield between the calling instruction pointer and the next context exit. We expect that support for syntatically-nested context managers could be added fairly easily. However, it’s not yet clear how this would work when user-defined context managers wrap <code class="docutils literal notranslate">sys.prevent_yields</code>. Worse, this approach ignores explicit calls to <code class="docutils literal notranslate">__enter__()</code> and <code class="docutils literal notranslate">__exit__()</code>, meaning that the context management protocol would vary depending on whether the <code class="docutils literal notranslate">with</code> statement was used. The ‘only pay if you use it’ performance cost is very attractive. However, inspecting frame objects is prohibitively expensive for core control-flow constructs, and causes whole-program slowdowns via de-optimization. On the other hand, adding interpreter support for better performance leads back to the same pay-regardless semantics as our preferred solution above. </section> </section> <section id="footnotes"> <h2><a class="toc-backref" href="#footnotes" role="doc-backlink">Footnotes</a></h2> <aside class="footnote-list brackets"> <aside class="footnote brackets" id="trio-cancel-scope" role="doc-footnote"> <dt class="label" id="trio-cancel-scope">[<a href="#id1">1</a>]</dt> <dd>While cancel scopes are implicit in asyncio, the analogous <a class="reference external" href="https://trio.readthedocs.io/en/latest/reference-core.html#trio.fail_after" title="(in Trio v0.29.0+dev)"><code class="docutils literal notranslate">trio.fail_after()</code></a> (sync) and <a class="reference external" href="https://trio.readthedocs.io/en/latest/reference-core.html#trio.open_nursery" title="(in Trio v0.29.0+dev)"><code class="docutils literal notranslate">trio.open_nursery()</code></a> (async) context managers literally wrap an instance of <a class="reference external" href="https://trio.readthedocs.io/en/latest/reference-core.html#trio.CancelScope" title="(in Trio v0.29.0+dev)"><code class="docutils literal notranslate">trio.CancelScope</code></a>. We’ll stick with asyncio for examples here, but say “cancel scope” when referring to the framework-independent concept.</aside> <aside class="footnote brackets" id="tg-cs" role="doc-footnote"> <dt class="label" id="tg-cs">[<a href="#id2">2</a>]</dt> <dd>A <code class="docutils literal notranslate">TaskGroup</code> is not _only_ a cancel scope, but preventing yields would resolve their further problem too. See <a class="reference internal" href="#just-deliver">Can’t we just deliver exceptions to the right place?</a>.</aside> <aside class="footnote brackets" id="redirected" role="doc-footnote"> <dt class="label" id="redirected">[<a href="#id3">3</a>]</dt> <dd>via e.g. <code class="docutils literal notranslate">contextlib.[async]contextmanager</code>, or moral equivalents such as <code class="docutils literal notranslate">@pytest.fixture</code></aside> <aside class="footnote brackets" id="also-sync" role="doc-footnote"> <dt class="label" id="also-sync">[<a href="#id4">4</a>]</dt> <dd>Note that this prevents yields in both sync and async generators, so that downstream frameworks can safely define sync cancel scope countexts such as <a class="reference external" href="https://trio.readthedocs.io/en/latest/reference-core.html#trio.fail_after" title="(in Trio v0.29.0+dev)"><code class="docutils literal notranslate">trio.fail_after()</code></a>.</aside> <aside class="footnote brackets" id="exp-report" role="doc-footnote"> <dt class="label" id="exp-report">[5] (<a href='#id5'>1</a>, <a href='#id6'>2</a>) </dt> <dd>see <a class="reference external" href="https://discuss.python.org/t/using-exceptiongroup-at-anthropic-experience-report/20888">Zac’s experience report here</a></aside> </aside> </section> <section id="copyright"> <h2><a class="toc-backref" href="#copyright" role="doc-backlink">Copyright</a></h2> This document is placed in the public domain or under the CC0-1.0-Universal license, whichever is more permissive. </section> </section> <hr class="docutils" /> Source: <a class="reference external" href="https://github.com/python/peps/blob/main/peps/pep-0789.rst">https://github.com/python/peps/blob/main/peps/pep-0789.rst</a> Last modified: <a class="reference external" href="https://github.com/python/peps/commits/main/peps/pep-0789.rst">2024-06-04 01:45:13 GMT</a> </article> <nav id="pep-sidebar"> <h2>Contents</h2> <ul> <li><a class="reference internal" href="#abstract">Abstract</a></li> <li><a class="reference internal" href="#background">Background</a></li> <li><a class="reference internal" href="#problem-statement">Problem statement</a></li> <li><a class="reference internal" href="#motivating-examples">Motivating examples</a><ul> <li><a class="reference internal" href="#leaking-a-timeout-to-the-outer-scope">Leaking a timeout to the outer scope</a></li> <li><a class="reference internal" href="#leaking-background-tasks-breaks-cancellation-and-exception-handling">Leaking background tasks (breaks cancellation and exception handling)</a></li> <li><a class="reference internal" href="#in-a-user-defined-context-manager">In a user-defined context manager</a></li> </ul> </li> <li><a class="reference internal" href="#specification">Specification</a><ul> <li><a class="reference internal" href="#implementation-tracking-frames">Implementation - tracking frames</a></li> <li><a class="reference internal" href="#worked-examples">Worked examples</a><ul> <li><a class="reference internal" href="#no-yield-example">No-yield example</a></li> <li><a class="reference internal" href="#attempts-to-yield-example">Attempts-to-yield example</a></li> <li><a class="reference internal" href="#allowed-to-yield-example">Allowed-to-yield example</a></li> <li><a class="reference internal" href="#allowing-yield-for-context-managers">Allowing yield for context managers</a></li> </ul> </li> <li><a class="reference internal" href="#behavior-if-sys-prevent-yields-is-misused">Behavior if <code class="docutils literal notranslate">sys.prevent_yields</code> is misused</a></li> </ul> </li> <li><a class="reference internal" href="#anticipated-uses">Anticipated uses</a></li> <li><a class="reference internal" href="#backwards-compatibility">Backwards Compatibility</a><ul> <li><a class="reference internal" href="#how-widespread-is-this-bug">How widespread is this bug?</a></li> </ul> </li> <li><a class="reference internal" href="#how-to-teach-this">How to Teach This</a></li> <li><a class="reference internal" href="#rejected-alternatives">Rejected alternatives</a><ul> <li><a class="reference internal" href="#pep-533-deterministic-cleanup-for-iterators">PEP 533, deterministic cleanup for iterators</a></li> <li><a class="reference internal" href="#deprecate-async-generators-entirely">Deprecate async generators entirely</a></li> <li><a class="reference internal" href="#can-t-we-just-deliver-exceptions-to-the-right-place">Can’t we just deliver exceptions to the right place?</a></li> <li><a class="reference internal" href="#alternative-implementation-inspecting-bytecode">Alternative implementation - inspecting bytecode</a></li> </ul> </li> <li><a class="reference internal" href="#footnotes">Footnotes</a></li> <li><a class="reference internal" href="#copyright">Copyright</a></li> </ul> <a id="source" href="https://github.com/python/peps/blob/main/peps/pep-0789.rst">Page Source (GitHub)</a> </nav> </section> <script src="../_static/colour_scheme.js"></script> <script src="../_static/wrap_tables.js"></script> <script src="../_static/sticky_banner.js"></script> </body> </html>