CINXE.COM

<!DOCTYPE html> <html lang="en"> <head> <meta charset="utf-8"> <meta name="viewport" content="width=device-width, initial-scale=1.0"> <meta name="color-scheme" content="light dark"> <title>PEP 531 – Existence checking operators | peps.python.org</title> <link rel="shortcut icon" href="../_static/py.png"> <link rel="canonical" href="https://peps.python.org/pep-0531/"> <link rel="stylesheet" href="../_static/style.css" type="text/css"> <link rel="stylesheet" href="../_static/mq.css" type="text/css"> <link rel="stylesheet" href="../_static/pygments.css" type="text/css" media="(prefers-color-scheme: light)" id="pyg-light"> <link rel="stylesheet" href="../_static/pygments_dark.css" type="text/css" media="(prefers-color-scheme: dark)" id="pyg-dark"> <link rel="alternate" type="application/rss+xml" title="Latest PEPs" href="https://peps.python.org/peps.rss"> <meta property="og:title" content='PEP 531 – Existence checking operators | peps.python.org'> <meta property="og:description" content="Inspired by PEP 505 and the related discussions, this PEP proposes the addition of two new control flow operators to Python:"> <meta property="og:type" content="website"> <meta property="og:url" content="https://peps.python.org/pep-0531/"> <meta property="og:site_name" content="Python Enhancement Proposals (PEPs)"> <meta property="og:image" content="https://peps.python.org/_static/og-image.png"> <meta property="og:image:alt" content="Python PEPs"> <meta property="og:image:width" content="200"> <meta property="og:image:height" content="200"> <meta name="description" content="Inspired by PEP 505 and the related discussions, this PEP proposes the addition of two new control flow operators to Python:"> <meta name="theme-color" content="#3776ab"> </head> <body> <svg xmlns="http://www.w3.org/2000/svg" style="display: none;"> <symbol id="svg-sun-half" viewBox="0 0 24 24" pointer-events="all"> <title>Following system colour scheme</title> <svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"> <circle cx="12" cy="12" r="9"></circle> <path d="M12 3v18m0-12l4.65-4.65M12 14.3l7.37-7.37M12 19.6l8.85-8.85"></path> </svg> </symbol> <symbol id="svg-moon" viewBox="0 0 24 24" pointer-events="all"> <title>Selected dark colour scheme</title> <svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"> <path stroke="none" d="M0 0h24v24H0z" fill="none"></path> <path d="M12 3c.132 0 .263 0 .393 0a7.5 7.5 0 0 0 7.92 12.446a9 9 0 1 1 -8.313 -12.454z"></path> </svg> </symbol> <symbol id="svg-sun" viewBox="0 0 24 24" pointer-events="all"> <title>Selected light colour scheme</title> <svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"> <circle cx="12" cy="12" r="5"></circle> <line x1="12" y1="1" x2="12" y2="3"></line> <line x1="12" y1="21" x2="12" y2="23"></line> <line x1="4.22" y1="4.22" x2="5.64" y2="5.64"></line> <line x1="18.36" y1="18.36" x2="19.78" y2="19.78"></line> <line x1="1" y1="12" x2="3" y2="12"></line> <line x1="21" y1="12" x2="23" y2="12"></line> <line x1="4.22" y1="19.78" x2="5.64" y2="18.36"></line> <line x1="18.36" y1="5.64" x2="19.78" y2="4.22"></line> </svg> </symbol> </svg> <script> document.documentElement.dataset.colour_scheme = localStorage.getItem("colour_scheme") || "auto" </script> <section id="pep-page-section"> <header> <h1>Python Enhancement Proposals</h1> <ul class="breadcrumbs"> <li><a href="https://www.python.org/" title="The Python Programming Language">Python</a> » </li> <li><a href="../pep-0000/">PEP Index</a> » </li> <li>PEP 531</li> </ul> <button id="colour-scheme-cycler" onClick="setColourScheme(nextColourScheme())"> <svg aria-hidden="true" class="colour-scheme-icon-when-auto"><use href="#svg-sun-half"></use></svg> <svg aria-hidden="true" class="colour-scheme-icon-when-dark"><use href="#svg-moon"></use></svg> <svg aria-hidden="true" class="colour-scheme-icon-when-light"><use href="#svg-sun"></use></svg> Toggle light / dark / auto colour theme </button> </header> <article> <section id="pep-content"> <h1 class="page-title">PEP 531 – Existence checking operators</h1> <dl class="rfc2822 field-list simple"> <dt class="field-odd">Author:</dt> <dd class="field-odd">Alyssa Coghlan <ncoghlan at gmail.com></dd> <dt class="field-even">Status:</dt> <dd class="field-even"><abbr title="Removed from consideration by sponsor or authors">Withdrawn</abbr></dd> <dt class="field-odd">Type:</dt> <dd class="field-odd"><abbr title="Normative PEP with a new feature for Python, implementation change for CPython or interoperability standard for the ecosystem">Standards Track</abbr></dd> <dt class="field-even">Created:</dt> <dd class="field-even">25-Oct-2016</dd> <dt class="field-odd">Python-Version:</dt> <dd class="field-odd">3.7</dd> <dt class="field-even">Post-History:</dt> <dd class="field-even">28-Oct-2016</dd> </dl> <hr class="docutils" /> <section id="contents"> <details><summary>Table of Contents</summary><ul class="simple"> <li><a class="reference internal" href="#abstract">Abstract</a></li> <li><a class="reference internal" href="#pep-withdrawal">PEP Withdrawal</a></li> <li><a class="reference internal" href="#relationship-with-other-peps">Relationship with other PEPs</a></li> <li><a class="reference internal" href="#rationale">Rationale</a><ul> <li><a class="reference internal" href="#existence-checking-expressions">Existence checking expressions</a></li> <li><a class="reference internal" href="#existence-checking-assignment">Existence checking assignment</a></li> <li><a class="reference internal" href="#existence-checking-protocol">Existence checking protocol</a></li> <li><a class="reference internal" href="#proposed-symbolic-notation">Proposed symbolic notation</a></li> <li><a class="reference internal" href="#proposed-keywords">Proposed keywords</a></li> </ul> </li> <li><a class="reference internal" href="#risks-and-concerns">Risks and concerns</a><ul> <li><a class="reference internal" href="#readability">Readability</a></li> <li><a class="reference internal" href="#magic-syntax">Magic syntax</a></li> <li><a class="reference internal" href="#conceptual-complexity">Conceptual complexity</a></li> </ul> </li> <li><a class="reference internal" href="#design-discussion">Design Discussion</a><ul> <li><a class="reference internal" href="#subtleties-in-chaining-existence-checking-expressions">Subtleties in chaining existence checking expressions</a></li> <li><a class="reference internal" href="#ambiguous-interaction-with-conditional-expressions">Ambiguous interaction with conditional expressions</a></li> <li><a class="reference internal" href="#existence-checking-in-other-truth-checking-contexts">Existence checking in other truth-checking contexts</a></li> <li><a class="reference internal" href="#defining-expected-invariant-relations-between-bool-and-exists">Defining expected invariant relations between <code class="docutils literal notranslate">__bool__</code> and <code class="docutils literal notranslate">__exists__</code></a></li> </ul> </li> <li><a class="reference internal" href="#limitations">Limitations</a><ul> <li><a class="reference internal" href="#arbitrary-sentinel-objects">Arbitrary sentinel objects</a></li> </ul> </li> <li><a class="reference internal" href="#specification">Specification</a></li> <li><a class="reference internal" href="#implementation">Implementation</a></li> <li><a class="reference internal" href="#references">References</a></li> <li><a class="reference internal" href="#copyright">Copyright</a></li> </ul> </details></section> <section id="abstract"> <h2><a class="toc-backref" href="#abstract" role="doc-backlink">Abstract</a></h2> Inspired by <a class="pep reference internal" href="../pep-0505/" title="PEP 505 – None-aware operators">PEP 505</a> and the related discussions, this PEP proposes the addition of two new control flow operators to Python: <ul class="simple"> <li>Existence-checking precondition (“exists-then”): <code class="docutils literal notranslate">expr1 ?then expr2</code></li> <li>Existence-checking fallback (“exists-else”): <code class="docutils literal notranslate">expr1 ?else expr2</code></li> </ul> as well as the following abbreviations for common existence checking expressions and statements: <ul class="simple"> <li>Existence-checking attribute access: <code class="docutils literal notranslate">obj?.attr</code> (for <code class="docutils literal notranslate">obj ?then obj.attr</code>)</li> <li>Existence-checking subscripting: <code class="docutils literal notranslate">obj?[expr]</code> (for <code class="docutils literal notranslate">obj ?then obj[expr]</code>)</li> <li>Existence-checking assignment: <code class="docutils literal notranslate">value ?= expr</code> (for <code class="docutils literal notranslate">value = value ?else expr</code>)</li> </ul> The common <code class="docutils literal notranslate">?</code> symbol in these new operator definitions indicates that they use a new “existence checking” protocol rather than the established truth-checking protocol used by if statements, while loops, comprehensions, generator expressions, conditional expressions, logical conjunction, and logical disjunction. This new protocol would be made available as <code class="docutils literal notranslate">operator.exists</code>, with the following characteristics: <ul class="simple"> <li>types can define a new <code class="docutils literal notranslate">__exists__</code> magic method (Python) or <code class="docutils literal notranslate">tp_exists</code> slot (C) to override the default behaviour. This optional method has the same signature and possible return values as <code class="docutils literal notranslate">__bool__</code>.</li> <li><code class="docutils literal notranslate">operator.exists(None)</code> returns <code class="docutils literal notranslate">False</code></li> <li><code class="docutils literal notranslate">operator.exists(NotImplemented)</code> returns <code class="docutils literal notranslate">False</code></li> <li><code class="docutils literal notranslate">operator.exists(Ellipsis)</code> returns <code class="docutils literal notranslate">False</code></li> <li><code class="docutils literal notranslate">float</code>, <code class="docutils literal notranslate">complex</code> and <code class="docutils literal notranslate">decimal.Decimal</code> will override the existence check such that <code class="docutils literal notranslate">NaN</code> values return <code class="docutils literal notranslate">False</code> and other values (including zero values) return <code class="docutils literal notranslate">True</code></li> <li>for any other type, <code class="docutils literal notranslate">operator.exists(obj)</code> returns True by default. Most importantly, values that evaluate to False in a truth checking context (zeroes, empty containers) will still evaluate to True in an existence checking context</li> </ul> </section> <section id="pep-withdrawal"> <h2><a class="toc-backref" href="#pep-withdrawal" role="doc-backlink">PEP Withdrawal</a></h2> When posting this PEP for discussion on python-ideas <a class="footnote-reference brackets" href="#id10" id="id1">[4]</a>, I asked reviewers to consider 3 high level design questions before moving on to considering the specifics of this particular syntactic proposal: 1. Do we collectively agree that “existence checking” is a useful general concept that exists in software development and is distinct from the concept of “truth checking”? 2. Do we collectively agree that the Python ecosystem would benefit from an existence checking protocol that permits generalisation of algorithms (especially short circuiting ones) across different “data missing” indicators, including those defined in the language definition, the standard library, and custom user code? 3. Do we collectively agree that it would be easier to use such a protocol effectively if existence-checking equivalents to the truth-checking “and” and “or” control flow operators were available? While the answers to the first question were generally positive, it quickly became clear that the answer to the second question is “No”. Steven D’Aprano articulated the counter-argument well in <a class="footnote-reference brackets" href="#id11" id="id2">[5]</a>, but the general idea is that when checking for “missing data” sentinels, we’re almost always looking for a specific sentinel value, rather than any sentinel value. <code class="docutils literal notranslate">NotImplemented</code> exists, for example, due to <code class="docutils literal notranslate">None</code> being a potentially legitimate result from overloaded arithmetic operators and exception handling imposing too much runtime overhead to be useful for operand coercion. Similarly, <code class="docutils literal notranslate">Ellipsis</code> exists for multi-dimensional slicing support due to <code class="docutils literal notranslate">None</code> already have another meaning in a slicing context (indicating the use of the default start or stop indices, or the default step size). In mathematics, the value of <code class="docutils literal notranslate">NaN</code> is that programmatically it behaves like a normal value of its type (e.g. exposing all the usual attributes and methods), while arithmetically it behaves according to the mathematical rules for handling <code class="docutils literal notranslate">NaN</code> values. With that core design concept invalidated, the proposal as a whole doesn’t make sense, and it is accordingly withdrawn. However, the discussion of the proposal did prompt consideration of a potential protocol based approach to make the existing <code class="docutils literal notranslate">and</code>, <code class="docutils literal notranslate">or</code> and <code class="docutils literal notranslate">if-else</code> operators more flexible <a class="footnote-reference brackets" href="#id12" id="id3">[6]</a> without introducing any new syntax, so I’ll be writing that up as another possible alternative to <a class="pep reference internal" href="../pep-0505/" title="PEP 505 – None-aware operators">PEP 505</a>. </section> <section id="relationship-with-other-peps"> <h2><a class="toc-backref" href="#relationship-with-other-peps" role="doc-backlink">Relationship with other PEPs</a></h2> While this PEP was inspired by and builds on Mark Haase’s excellent work in putting together <a class="pep reference internal" href="../pep-0505/" title="PEP 505 – None-aware operators">PEP 505</a>, it ultimately competes with that PEP due to significant differences in the specifics of the proposed syntax and semantics for the feature. It also presents a different perspective on the rationale for the change by focusing on the benefits to existing Python users as the typical demands of application and service development activities are genuinely changing. It isn’t an accident that similar features are now appearing in multiple programming languages, and while it’s a good idea for us to learn from how other language designers are handling the problem, precedents being set elsewhere are more relevant to how we would go about tackling this problem than they are to whether or not we think it’s a problem we should address in the first place. </section> <section id="rationale"> <h2><a class="toc-backref" href="#rationale" role="doc-backlink">Rationale</a></h2> <section id="existence-checking-expressions"> <h3><a class="toc-backref" href="#existence-checking-expressions" role="doc-backlink">Existence checking expressions</a></h3> An increasingly common requirement in modern software development is the need to work with “semi-structured data”: data where the structure of the data is known in advance, but pieces of it may be missing at runtime, and the software manipulating that data is expected to degrade gracefully (e.g. by omitting results that depend on the missing data) rather than failing outright. Some particularly common cases where this issue arises are: <ul class="simple"> <li>handling optional application configuration settings and function parameters</li> <li>handling external service failures in distributed systems</li> <li>handling data sets that include some partial records</li> </ul> It is the latter two cases that are the primary motivation for this PEP - while needing to deal with optional configuration settings and parameters is a design requirement at least as old as Python itself, the rise of public cloud infrastructure, the development of software systems as collaborative networks of distributed services, and the availability of large public and private data sets for analysis means that the ability to degrade operations gracefully in the face of partial service failures or partial data availability is becoming an essential feature of modern programming environments. At the moment, writing such software in Python can be genuinely awkward, as your code ends up littered with expressions like: <ul class="simple"> <li><code class="docutils literal notranslate">value1 = expr1.field.of.interest if expr1 is not None else None</code></li> <li><code class="docutils literal notranslate">value2 = expr2["field"]["of"]["interest"] if expr2 is not None else None</code></li> <li><code class="docutils literal notranslate">value3 = expr3 if expr3 is not None else expr4 if expr4 is not None else expr5</code></li> </ul> If these are only occasional, then expanding out to full statement forms may help improve readability, but if you have 4 or 5 of them in a row (which is a fairly common situation in data transformation pipelines), then replacing them with 16 or 20 lines of conditional logic really doesn’t help matters. Expanding the three examples above that way hopefully helps illustrate that: <div class="highlight-default notranslate"><div class="highlight"><pre>if expr1 is not None: value1 = expr1.field.of.interest else: value1 = None if expr2 is not None: value2 = expr2["field"]["of"]["interest"] else: value2 = None if expr3 is not None: value3 = expr3 else: if expr4 is not None: value3 = expr4 else: value3 = expr5 </pre></div> </div> The combined impact of the proposals in this PEP is to allow the above sample expressions to instead be written as: <ul class="simple"> <li><code class="docutils literal notranslate">value1 = expr1?.field.of.interest</code></li> <li><code class="docutils literal notranslate">value2 = expr2?["field"]["of"]["interest"]</code></li> <li><code class="docutils literal notranslate">value3 = expr3 ?else expr4 ?else expr5</code></li> </ul> In these forms, almost all of the information presented to the reader is immediately relevant to the question “What does this code do?”, while the boilerplate code to handle missing data by passing it through to the output or falling back to an alternative input, has shrunk to two uses of the <code class="docutils literal notranslate">?</code> symbol and two uses of the <code class="docutils literal notranslate">?else</code> keyword. In the first two examples, the 31 character boilerplate clause <code class="docutils literal notranslate">if exprN is not None else None</code> (minimally 27 characters for a single letter variable name) has been replaced by a single <code class="docutils literal notranslate">?</code> character, substantially improving the signal-to-pattern-noise ratio of the lines (especially if it encourages the use of more meaningful variable and field names rather than making them shorter purely for the sake of expression brevity). In the last example, two instances of the 21 character boilerplate, <code class="docutils literal notranslate">if exprN is not None</code> (minimally 17 characters) are replaced with single characters, again substantially improving the signal-to-pattern-noise ratio. Furthermore, each of our 5 “subexpressions of potential interest” is included exactly once, rather than 4 of them needing to be duplicated or pulled out to a named variable in order to first check if they exist. The existence checking precondition operator is mainly defined to provide a clear conceptual basis for the existence checking attribute access and subscripting operators: <ul class="simple"> <li><code class="docutils literal notranslate">obj?.attr</code> is roughly equivalent to <code class="docutils literal notranslate">obj ?then obj.attr</code></li> <li><code class="docutils literal notranslate">obj?[expr]</code> is roughly equivalent to <code class="docutils literal notranslate">obj ?then obj[expr]</code></li> </ul> The main semantic difference between the shorthand forms and their expanded equivalents is that the common subexpression to the left of the existence checking operator is evaluated only once in the shorthand form (similar to the benefit offered by augmented assignment statements). </section> <section id="existence-checking-assignment"> <h3><a class="toc-backref" href="#existence-checking-assignment" role="doc-backlink">Existence checking assignment</a></h3> Existence-checking assignment is proposed as a relatively straightforward expansion of the concepts in this PEP to also cover the common configuration handling idiom: <ul class="simple"> <li><code class="docutils literal notranslate">value = value if value is not None else expensive_default()</code></li> </ul> by allowing that to instead be abbreviated as: <ul class="simple"> <li><code class="docutils literal notranslate">value ?= expensive_default()</code></li> </ul> This is mainly beneficial when the target is a subscript operation or subattribute, as even without this specific change, the PEP would still permit this idiom to be updated to: <ul class="simple"> <li><code class="docutils literal notranslate">value = value ?else expensive_default()</code></li> </ul> The main argument against adding this form is that it’s arguably ambiguous and could mean either: <ul class="simple"> <li><code class="docutils literal notranslate">value = value ?else expensive_default()</code>; or</li> <li><code class="docutils literal notranslate">value = value ?then value.subfield.of.interest</code></li> </ul> The second form isn’t at all useful, but if this concern was deemed significant enough to address while still keeping the augmented assignment feature, the full keyword could be included in the syntax: <ul class="simple"> <li><code class="docutils literal notranslate">value ?else= expensive_default()</code></li> </ul> Alternatively, augmented assignment could just be dropped from the current proposal entirely and potentially reconsidered at a later date. </section> <section id="existence-checking-protocol"> <h3><a class="toc-backref" href="#existence-checking-protocol" role="doc-backlink">Existence checking protocol</a></h3> The existence checking protocol is including in this proposal primarily to allow for proxy objects (e.g. local representations of remote resources) and mock objects used in testing to correctly indicate non-existence of target resources, even though the proxy or mock object itself is not None. However, with that protocol defined, it then seems natural to expand it to provide a type independent way of checking for <code class="docutils literal notranslate">NaN</code> values in numeric types - at the moment you need to be aware of the exact data type you’re working with (e.g. builtin floats, builtin complex numbers, the decimal module) and use the appropriate operation (e.g. <code class="docutils literal notranslate">math.isnan</code>, <code class="docutils literal notranslate">cmath.isnan</code>, <code class="docutils literal notranslate">decimal.getcontext().is_nan()</code>, respectively) Similarly, it seems reasonable to declare that the other placeholder builtin singletons, <code class="docutils literal notranslate">Ellipsis</code> and <code class="docutils literal notranslate">NotImplemented</code>, also qualify as objects that represent the absence of data more so than they represent data. </section> <section id="proposed-symbolic-notation"> <h3><a class="toc-backref" href="#proposed-symbolic-notation" role="doc-backlink">Proposed symbolic notation</a></h3> Python has historically only had one kind of implied boolean context: truth checking, which can be invoked directly via the <code class="docutils literal notranslate">bool()</code> builtin. As this PEP proposes a new kind of control flow operation based on existence checking rather than truth checking, it is considered valuable to have a reminder directly in the code when existence checking is being used rather than truth checking. The mathematical symbol for existence assertions is U+2203 ‘THERE EXISTS’: <code class="docutils literal notranslate">∃</code> Accordingly, one possible approach to the syntactic additions proposed in this PEP would be to use that already defined mathematical notation: <ul class="simple"> <li><code class="docutils literal notranslate">expr1 ∃then expr2</code></li> <li><code class="docutils literal notranslate">expr1 ∃else expr2</code></li> <li><code class="docutils literal notranslate">obj∃.attr</code></li> <li><code class="docutils literal notranslate">obj∃[expr]</code></li> <li><code class="docutils literal notranslate">target ∃= expr</code></li> </ul> However, there are two major problems with that approach, one practical, and one pedagogical. The practical problem is the usual one that most keyboards don’t offer any easy way of entering mathematical symbols other than those used in basic arithmetic (even the symbols appearing in this PEP were ultimately copied & pasted from <a class="footnote-reference brackets" href="#id9" id="id4">[3]</a> rather than being entered directly). The pedagogical problem is that the symbols for existence assertions (<code class="docutils literal notranslate">∃</code>) and universal assertions (<code class="docutils literal notranslate">∀</code>) aren’t going to be familiar to most people the way basic arithmetic operators are, so we wouldn’t actually be making the proposed syntax easier to understand by adopting <code class="docutils literal notranslate">∃</code>. By contrast, <code class="docutils literal notranslate">?</code> is one of the few remaining unused ASCII punctuation characters in Python’s syntax, making it available as a candidate syntactic marker for “this control flow operation is based on an existence check, not a truth check”. Taking that path would also have the advantage of aligning Python’s syntax with corresponding syntax in other languages that offer similar features. Drawing from the existing summary in <a class="pep reference internal" href="../pep-0505/" title="PEP 505 – None-aware operators">PEP 505</a> and the Wikipedia articles on the “safe navigation operator <a class="footnote-reference brackets" href="#id7" id="id5">[1]</a> and the “null coalescing operator” <a class="footnote-reference brackets" href="#id8" id="id6">[2]</a>, we see: <ul class="simple"> <li>The <code class="docutils literal notranslate">?.</code> existence checking attribute access syntax precisely aligns with:<ul> <li>the “safe navigation” attribute access operator in C# (<code class="docutils literal notranslate">?.</code>)</li> <li>the “optional chaining” operator in Swift (<code class="docutils literal notranslate">?.</code>)</li> <li>the “safe navigation” attribute access operator in Groovy (<code class="docutils literal notranslate">?.</code>)</li> <li>the “conditional member access” operator in Dart (<code class="docutils literal notranslate">?.</code>)</li> </ul> </li> <li>The <code class="docutils literal notranslate">?[]</code> existence checking attribute access syntax precisely aligns with:<ul> <li>the “safe navigation” subscript operator in C# (<code class="docutils literal notranslate">?[]</code>)</li> <li>the “optional subscript” operator in Swift (<code class="docutils literal notranslate">?[].</code>)</li> </ul> </li> <li>The <code class="docutils literal notranslate">?else</code> existence checking fallback syntax semantically aligns with:<ul> <li>the “null-coalescing” operator in C# (<code class="docutils literal notranslate">??</code>)</li> <li>the “null-coalescing” operator in PHP (<code class="docutils literal notranslate">??</code>)</li> <li>the “nil-coalescing” operator in Swift (<code class="docutils literal notranslate">??</code>)</li> </ul> </li> </ul> To be clear, these aren’t the only spelling of these operators used in other languages, but they’re the most common ones, and the <code class="docutils literal notranslate">?</code> symbol is the most common syntactic marker by far (presumably prompted by the use of <code class="docutils literal notranslate">?</code> to introduce the “then” clause in C-style conditional expressions, which many of these languages also offer). </section> <section id="proposed-keywords"> <h3><a class="toc-backref" href="#proposed-keywords" role="doc-backlink">Proposed keywords</a></h3> Given the symbolic marker <code class="docutils literal notranslate">?</code>, it would be syntactically unambiguous to spell the existence checking precondition and fallback operations using the same keywords as their truth checking counterparts: <ul class="simple"> <li><code class="docutils literal notranslate">expr1 ?and expr2</code> (instead of <code class="docutils literal notranslate">expr1 ?then expr2</code>)</li> <li><code class="docutils literal notranslate">expr1 ?or expr2</code> (instead of <code class="docutils literal notranslate">expr1 ?else expr2</code>)</li> </ul> However, while syntactically unambiguous when written, this approach makes the code incredibly hard to pronounce (What’s the pronunciation of “?”?) and also hard to describe (given reused keywords, there’s no obvious shorthand terms for “existence checking precondition (?and)” and “existence checking fallback (?or)” that would distinguish them from “logical conjunction (and)” and “logical disjunction (or)”). We could try to encourage folks to pronounce the <code class="docutils literal notranslate">?</code> symbol as “exists”, making the shorthand names the “exists-and expression” and the “exists-or expression”, but there’d be no way of guessing those names purely from seeing them written in a piece of code. Instead, this PEP takes advantage of the proposed symbolic syntax to introduce a new keyword (<code class="docutils literal notranslate">?then</code>) and borrow an existing one (<code class="docutils literal notranslate">?else</code>) in a way that allows people to refer to “then expressions” and “else expressions” without ambiguity. These keywords also align well with the conditional expressions that are semantically equivalent to the proposed expressions. For <code class="docutils literal notranslate">?else</code> expressions, <code class="docutils literal notranslate">expr1 ?else expr2</code> is equivalent to: <div class="highlight-default notranslate"><div class="highlight"><pre>_lhs_result = expr1 _lhs_result if operator.exists(_lhs_result) else expr2 </pre></div> </div> Here the parallel is clear, since the <code class="docutils literal notranslate">else expr2</code> appears at the end of both the abbreviated and expanded forms. For <code class="docutils literal notranslate">?then</code> expressions, <code class="docutils literal notranslate">expr1 ?then expr2</code> is equivalent to: <div class="highlight-default notranslate"><div class="highlight"><pre>_lhs_result = expr1 expr2 if operator.exists(_lhs_result) else _lhs_result </pre></div> </div> Here the parallel isn’t as immediately obvious due to Python’s traditionally anonymous “then” clauses (introduced by <code class="docutils literal notranslate">:</code> in <code class="docutils literal notranslate">if</code> statements and suffixed by <code class="docutils literal notranslate">if</code> in conditional expressions), but it’s still reasonably clear as long as you’re already familiar with the “if-then-else” explanation of conditional control flow. </section> </section> <section id="risks-and-concerns"> <h2><a class="toc-backref" href="#risks-and-concerns" role="doc-backlink">Risks and concerns</a></h2> <section id="readability"> <h3><a class="toc-backref" href="#readability" role="doc-backlink">Readability</a></h3> Learning to read and write the new syntax effectively mainly requires internalising two concepts: <ul class="simple"> <li>expressions containing <code class="docutils literal notranslate">?</code> include an existence check and may short circuit</li> <li>if <code class="docutils literal notranslate">None</code> or another “non-existent” value is an expected input, and the correct handling is to propagate that to the result, then the existence checking operators are likely what you want</li> </ul> Currently, these concepts aren’t explicitly represented at the language level, so it’s a matter of learning to recognise and use the various idiomatic patterns based on conditional expressions and statements. </section> <section id="magic-syntax"> <h3><a class="toc-backref" href="#magic-syntax" role="doc-backlink">Magic syntax</a></h3> There’s nothing about <code class="docutils literal notranslate">?</code> as a syntactic element that inherently suggests <code class="docutils literal notranslate">is not None</code> or <code class="docutils literal notranslate">operator.exists</code>. The main current use of <code class="docutils literal notranslate">?</code> as a symbol in Python code is as a trailing suffix in IPython environments to request help information for the result of the preceding expression. However, the notion of existence checking really does benefit from a pervasive visual marker that distinguishes it from truth checking, and that calls for a single-character symbolic syntax if we’re going to do it at all. </section> <section id="conceptual-complexity"> <h3><a class="toc-backref" href="#conceptual-complexity" role="doc-backlink">Conceptual complexity</a></h3> This proposal takes the currently ad hoc and informal concept of “existence checking” and elevates it to the status of being a syntactic language feature with a clearly defined operator protocol. In many ways, this should actually reduce the overall conceptual complexity of the language, as many more expectations will map correctly between truth checking with <code class="docutils literal notranslate">bool(expr)</code> and existence checking with <code class="docutils literal notranslate">operator.exists(expr)</code> than currently map between truth checking and existence checking with <code class="docutils literal notranslate">expr is not None</code> (or <code class="docutils literal notranslate">expr is not NotImplemented</code> in the context of operand coercion, or the various NaN-checking operations in mathematical libraries). As a simple example of the new parallels introduced by this PEP, compare: <div class="highlight-default notranslate"><div class="highlight"><pre>all_are_true = all(map(bool, iterable)) at_least_one_is_true = any(map(bool, iterable)) all_exist = all(map(operator.exists, iterable)) at_least_one_exists = any(map(operator.exists, iterable)) </pre></div> </div> </section> </section> <section id="design-discussion"> <h2><a class="toc-backref" href="#design-discussion" role="doc-backlink">Design Discussion</a></h2> <section id="subtleties-in-chaining-existence-checking-expressions"> <h3><a class="toc-backref" href="#subtleties-in-chaining-existence-checking-expressions" role="doc-backlink">Subtleties in chaining existence checking expressions</a></h3> Similar subtleties arise in chaining existence checking expressions as already exist in chaining logical operators: the behaviour can be surprising if the right hand side of one of the expressions in the chain itself returns a value that doesn’t exist. As a result, <code class="docutils literal notranslate">value = arg1 ?then f(arg1) ?else default()</code> would be dubious for essentially the same reason that <code class="docutils literal notranslate">value = cond and expr1 or expr2</code> is dubious: the former will evaluate <code class="docutils literal notranslate">default()</code> if <code class="docutils literal notranslate">f(arg1)</code> returns <code class="docutils literal notranslate">None</code>, just as the latter will evaluate <code class="docutils literal notranslate">expr2</code> if <code class="docutils literal notranslate">expr1</code> evaluates to <code class="docutils literal notranslate">False</code> in a boolean context. </section> <section id="ambiguous-interaction-with-conditional-expressions"> <h3><a class="toc-backref" href="#ambiguous-interaction-with-conditional-expressions" role="doc-backlink">Ambiguous interaction with conditional expressions</a></h3> In the proposal as currently written, the following is a syntax error: <ul class="simple"> <li><code class="docutils literal notranslate">value = f(arg) if arg ?else default</code></li> </ul> While the following is a valid operation that checks a second condition if the first doesn’t exist rather than merely being false: <ul class="simple"> <li><code class="docutils literal notranslate">value = expr1 if cond1 ?else cond2 else expr2</code></li> </ul> The expression chaining problem described above means that the argument can be made that the first operation should instead be equivalent to: <ul class="simple"> <li><code class="docutils literal notranslate">value = f(arg) if operator.exists(arg) else default</code></li> </ul> requiring the second to be written in the arguably clearer form: <ul class="simple"> <li><code class="docutils literal notranslate">value = expr1 if (cond1 ?else cond2) else expr2</code></li> </ul> Alternatively, the first form could remain a syntax error, and the existence checking symbol could instead be attached to the <code class="docutils literal notranslate">if</code> keyword: <ul class="simple"> <li><code class="docutils literal notranslate">value = expr1 if? cond else expr2</code></li> </ul> </section> <section id="existence-checking-in-other-truth-checking-contexts"> <h3><a class="toc-backref" href="#existence-checking-in-other-truth-checking-contexts" role="doc-backlink">Existence checking in other truth-checking contexts</a></h3> The truth-checking protocol is currently used in the following syntactic constructs: <ul class="simple"> <li>logical conjunction (and-expressions)</li> <li>logical disjunction (or-expressions)</li> <li>conditional expressions (if-else expressions)</li> <li>if statements</li> <li>while loops</li> <li>filter clauses in comprehensions and generator expressions</li> </ul> In the current PEP, switching from truth-checking with <code class="docutils literal notranslate">and</code> and <code class="docutils literal notranslate">or</code> to existence-checking is a matter of substituting in the new keywords, <code class="docutils literal notranslate">?then</code> and <code class="docutils literal notranslate">?else</code> in the appropriate places. For other truth-checking contexts, it proposes either importing and using the <code class="docutils literal notranslate">operator.exists</code> API, or else continuing with the current idiom of checking specifically for <code class="docutils literal notranslate">expr is not None</code> (or the context appropriate equivalent). The simplest possible enhancement in that regard would be to elevate the proposed <code class="docutils literal notranslate">exists()</code> API from an operator module function to a new builtin function. Alternatively, the <code class="docutils literal notranslate">?</code> existence checking symbol could be supported as a modifier on the <code class="docutils literal notranslate">if</code> and <code class="docutils literal notranslate">while</code> keywords to indicate the use of an existence check rather than a truth check. However, it isn’t at all clear that the potential consistency benefits gained for either suggestion would justify the additional disruption, so they’ve currently been omitted from the proposal. </section> <section id="defining-expected-invariant-relations-between-bool-and-exists"> <h3><a class="toc-backref" href="#defining-expected-invariant-relations-between-bool-and-exists" role="doc-backlink">Defining expected invariant relations between <code class="docutils literal notranslate">__bool__</code> and <code class="docutils literal notranslate">__exists__</code></a></h3> The PEP currently leaves the definition of <code class="docutils literal notranslate">__bool__</code> on all existing types unmodified, which ensures the entire proposal remains backwards compatible, but results in the following cases where <code class="docutils literal notranslate">bool(obj)</code> returns <code class="docutils literal notranslate">True</code>, but the proposed <code class="docutils literal notranslate">operator.exists(obj)</code> would return <code class="docutils literal notranslate">False</code>: <ul class="simple"> <li><code class="docutils literal notranslate">NaN</code> values for <code class="docutils literal notranslate">float</code>, <code class="docutils literal notranslate">complex</code>, and <code class="docutils literal notranslate">decimal.Decimal</code></li> <li><code class="docutils literal notranslate">Ellipsis</code></li> <li><code class="docutils literal notranslate">NotImplemented</code></li> </ul> The main argument for potentially changing these is that it becomes easier to reason about potential code behaviour if we have a recommended invariant in place saying that values which indicate they don’t exist in an existence checking context should also report themselves as being <code class="docutils literal notranslate">False</code> in a truth checking context. Failing to define such an invariant would lead to arguably odd outcomes like <code class="docutils literal notranslate">float("NaN") ?else 0.0</code> returning <code class="docutils literal notranslate">0.0</code> while <code class="docutils literal notranslate">float("NaN") or 0.0</code> returns <code class="docutils literal notranslate">NaN</code>. </section> </section> <section id="limitations"> <h2><a class="toc-backref" href="#limitations" role="doc-backlink">Limitations</a></h2> <section id="arbitrary-sentinel-objects"> <h3><a class="toc-backref" href="#arbitrary-sentinel-objects" role="doc-backlink">Arbitrary sentinel objects</a></h3> This proposal doesn’t attempt to provide syntactic support for the “sentinel object” idiom, where <code class="docutils literal notranslate">None</code> is a permitted explicit value, so a separate sentinel object is defined to indicate missing values: <div class="highlight-default notranslate"><div class="highlight"><pre>_SENTINEL = object() def f(obj=_SENTINEL): return obj if obj is not _SENTINEL else default_value() </pre></div> </div> This could potentially be supported at the expense of making the existence protocol definition significantly more complex, both to define and to use: <ul class="simple"> <li>at the Python layer, <code class="docutils literal notranslate">operator.exists</code> and <code class="docutils literal notranslate">__exists__</code> implementations would return the empty tuple to indicate non-existence, and otherwise return a singleton tuple containing a reference to the object to be used as the result of the existence check</li> <li>at the C layer, <code class="docutils literal notranslate">tp_exists</code> implementations would return NULL to indicate non-existence, and otherwise return a <code class="docutils literal notranslate">PyObject *</code> pointer as the result of the existence check</li> </ul> Given that change, the sentinel object idiom could be rewritten as: <div class="highlight-default notranslate"><div class="highlight"><pre>class Maybe: SENTINEL = object() def __init__(self, value): self._result = (value,) is value is not self.SENTINEL else () def __exists__(self): return self._result def f(obj=Maybe.SENTINEL): return Maybe(obj) ?else default_value() </pre></div> </div> However, I don’t think cases where the 3 proposed standard sentinel values (i.e. <code class="docutils literal notranslate">None</code>, <code class="docutils literal notranslate">Ellipsis</code> and <code class="docutils literal notranslate">NotImplemented</code>) can’t be used are going to be anywhere near common enough for the additional protocol complexity and the loss of symmetry between <code class="docutils literal notranslate">__bool__</code> and <code class="docutils literal notranslate">__exists__</code> to be worth it. </section> </section> <section id="specification"> <h2><a class="toc-backref" href="#specification" role="doc-backlink">Specification</a></h2> The Abstract already gives the gist of the proposal and the Rationale gives some specific examples. If there’s enough interest in the basic idea, then a full specification will need to provide a precise correspondence between the proposed syntactic sugar and the underlying conditional expressions that is sufficient to guide the creation of a reference implementation. …TBD… </section> <section id="implementation"> <h2><a class="toc-backref" href="#implementation" role="doc-backlink">Implementation</a></h2> As with <a class="pep reference internal" href="../pep-0505/" title="PEP 505 – None-aware operators">PEP 505</a>, actual implementation has been deferred pending in-principle interest in the idea of adding these operators - the implementation isn’t the hard part of these proposals, the hard part is deciding whether or not this is a change where the long term benefits for new and existing Python users outweigh the short term costs involved in the wider ecosystem (including developers of other implementations, language curriculum developers, and authors of other Python related educational material) adjusting to the change. …TBD… </section> <section id="references"> <h2><a class="toc-backref" href="#references" role="doc-backlink">References</a></h2> <aside class="footnote-list brackets"> <aside class="footnote brackets" id="id7" role="doc-footnote"> <dt class="label" id="id7">[<a href="#id5">1</a>]</dt> <dd>Wikipedia: Safe navigation operator (<a class="reference external" href="https://en.wikipedia.org/wiki/Safe_navigation_operator">https://en.wikipedia.org/wiki/Safe_navigation_operator</a>)</aside> <aside class="footnote brackets" id="id8" role="doc-footnote"> <dt class="label" id="id8">[<a href="#id6">2</a>]</dt> <dd>Wikipedia: Null coalescing operator (<a class="reference external" href="https://en.wikipedia.org/wiki/Null_coalescing_operator">https://en.wikipedia.org/wiki/Null_coalescing_operator</a>)</aside> <aside class="footnote brackets" id="id9" role="doc-footnote"> <dt class="label" id="id9">[<a href="#id4">3</a>]</dt> <dd>FileFormat.info: Unicode Character ‘THERE EXISTS’ (U+2203) (<a class="reference external" href="http://www.fileformat.info/info/unicode/char/2203/index.htm">http://www.fileformat.info/info/unicode/char/2203/index.htm</a>)</aside> <aside class="footnote brackets" id="id10" role="doc-footnote"> <dt class="label" id="id10">[<a href="#id1">4</a>]</dt> <dd>python-ideas discussion thread (<a class="reference external" href="https://mail.python.org/pipermail/python-ideas/2016-October/043415.html">https://mail.python.org/pipermail/python-ideas/2016-October/043415.html</a>)</aside> <aside class="footnote brackets" id="id11" role="doc-footnote"> <dt class="label" id="id11">[<a href="#id2">5</a>]</dt> <dd>Steven D’Aprano’s critique of the proposal (<a class="reference external" href="https://mail.python.org/pipermail/python-ideas/2016-October/043453.html">https://mail.python.org/pipermail/python-ideas/2016-October/043453.html</a>)</aside> <aside class="footnote brackets" id="id12" role="doc-footnote"> <dt class="label" id="id12">[<a href="#id3">6</a>]</dt> <dd>Considering a link to the idea of overloadable Boolean operators (<a class="reference external" href="https://mail.python.org/pipermail/python-ideas/2016-October/043447.html">https://mail.python.org/pipermail/python-ideas/2016-October/043447.html</a>)</aside> </aside> </section> <section id="copyright"> <h2><a class="toc-backref" href="#copyright" role="doc-backlink">Copyright</a></h2> This document has been placed in the public domain under the terms of the CC0 1.0 license: <a class="reference external" href="https://creativecommons.org/publicdomain/zero/1.0/">https://creativecommons.org/publicdomain/zero/1.0/</a> </section> </section> <hr class="docutils" /> Source: <a class="reference external" href="https://github.com/python/peps/blob/main/peps/pep-0531.rst">https://github.com/python/peps/blob/main/peps/pep-0531.rst</a> Last modified: <a class="reference external" href="https://github.com/python/peps/commits/main/peps/pep-0531.rst">2023-10-11 12:05:51 GMT</a> </article> <nav id="pep-sidebar"> <h2>Contents</h2> <ul> <li><a class="reference internal" href="#abstract">Abstract</a></li> <li><a class="reference internal" href="#pep-withdrawal">PEP Withdrawal</a></li> <li><a class="reference internal" href="#relationship-with-other-peps">Relationship with other PEPs</a></li> <li><a class="reference internal" href="#rationale">Rationale</a><ul> <li><a class="reference internal" href="#existence-checking-expressions">Existence checking expressions</a></li> <li><a class="reference internal" href="#existence-checking-assignment">Existence checking assignment</a></li> <li><a class="reference internal" href="#existence-checking-protocol">Existence checking protocol</a></li> <li><a class="reference internal" href="#proposed-symbolic-notation">Proposed symbolic notation</a></li> <li><a class="reference internal" href="#proposed-keywords">Proposed keywords</a></li> </ul> </li> <li><a class="reference internal" href="#risks-and-concerns">Risks and concerns</a><ul> <li><a class="reference internal" href="#readability">Readability</a></li> <li><a class="reference internal" href="#magic-syntax">Magic syntax</a></li> <li><a class="reference internal" href="#conceptual-complexity">Conceptual complexity</a></li> </ul> </li> <li><a class="reference internal" href="#design-discussion">Design Discussion</a><ul> <li><a class="reference internal" href="#subtleties-in-chaining-existence-checking-expressions">Subtleties in chaining existence checking expressions</a></li> <li><a class="reference internal" href="#ambiguous-interaction-with-conditional-expressions">Ambiguous interaction with conditional expressions</a></li> <li><a class="reference internal" href="#existence-checking-in-other-truth-checking-contexts">Existence checking in other truth-checking contexts</a></li> <li><a class="reference internal" href="#defining-expected-invariant-relations-between-bool-and-exists">Defining expected invariant relations between <code class="docutils literal notranslate">__bool__</code> and <code class="docutils literal notranslate">__exists__</code></a></li> </ul> </li> <li><a class="reference internal" href="#limitations">Limitations</a><ul> <li><a class="reference internal" href="#arbitrary-sentinel-objects">Arbitrary sentinel objects</a></li> </ul> </li> <li><a class="reference internal" href="#specification">Specification</a></li> <li><a class="reference internal" href="#implementation">Implementation</a></li> <li><a class="reference internal" href="#references">References</a></li> <li><a class="reference internal" href="#copyright">Copyright</a></li> </ul> <a id="source" href="https://github.com/python/peps/blob/main/peps/pep-0531.rst">Page Source (GitHub)</a> </nav> </section> <script src="../_static/colour_scheme.js"></script> <script src="../_static/wrap_tables.js"></script> <script src="../_static/sticky_banner.js"></script> </body> </html>