CINXE.COM

<!DOCTYPE html> <html lang="en"> <head> <meta charset="utf-8"> <meta name="viewport" content="width=device-width, initial-scale=1.0"> <meta name="color-scheme" content="light dark"> <title>PEP 655 – Marking individual TypedDict items as required or potentially-missing | peps.python.org</title> <link rel="shortcut icon" href="../_static/py.png"> <link rel="canonical" href="https://peps.python.org/pep-0655/"> <link rel="stylesheet" href="../_static/style.css" type="text/css"> <link rel="stylesheet" href="../_static/mq.css" type="text/css"> <link rel="stylesheet" href="../_static/pygments.css" type="text/css" media="(prefers-color-scheme: light)" id="pyg-light"> <link rel="stylesheet" href="../_static/pygments_dark.css" type="text/css" media="(prefers-color-scheme: dark)" id="pyg-dark"> <link rel="alternate" type="application/rss+xml" title="Latest PEPs" href="https://peps.python.org/peps.rss"> <meta property="og:title" content='PEP 655 – Marking individual TypedDict items as required or potentially-missing | peps.python.org'> <meta property="og:description" content="PEP 589 defines notation for declaring a TypedDict with all required keys and notation for defining a TypedDict with all potentially-missing keys, however it does not provide a mechanism to declare some keys as required and others as potentially-missing..."> <meta property="og:type" content="website"> <meta property="og:url" content="https://peps.python.org/pep-0655/"> <meta property="og:site_name" content="Python Enhancement Proposals (PEPs)"> <meta property="og:image" content="https://peps.python.org/_static/og-image.png"> <meta property="og:image:alt" content="Python PEPs"> <meta property="og:image:width" content="200"> <meta property="og:image:height" content="200"> <meta name="description" content="PEP 589 defines notation for declaring a TypedDict with all required keys and notation for defining a TypedDict with all potentially-missing keys, however it does not provide a mechanism to declare some keys as required and others as potentially-missing..."> <meta name="theme-color" content="#3776ab"> </head> <body> <svg xmlns="http://www.w3.org/2000/svg" style="display: none;"> <symbol id="svg-sun-half" viewBox="0 0 24 24" pointer-events="all"> <title>Following system colour scheme</title> <svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"> <circle cx="12" cy="12" r="9"></circle> <path d="M12 3v18m0-12l4.65-4.65M12 14.3l7.37-7.37M12 19.6l8.85-8.85"></path> </svg> </symbol> <symbol id="svg-moon" viewBox="0 0 24 24" pointer-events="all"> <title>Selected dark colour scheme</title> <svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"> <path stroke="none" d="M0 0h24v24H0z" fill="none"></path> <path d="M12 3c.132 0 .263 0 .393 0a7.5 7.5 0 0 0 7.92 12.446a9 9 0 1 1 -8.313 -12.454z"></path> </svg> </symbol> <symbol id="svg-sun" viewBox="0 0 24 24" pointer-events="all"> <title>Selected light colour scheme</title> <svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"> <circle cx="12" cy="12" r="5"></circle> <line x1="12" y1="1" x2="12" y2="3"></line> <line x1="12" y1="21" x2="12" y2="23"></line> <line x1="4.22" y1="4.22" x2="5.64" y2="5.64"></line> <line x1="18.36" y1="18.36" x2="19.78" y2="19.78"></line> <line x1="1" y1="12" x2="3" y2="12"></line> <line x1="21" y1="12" x2="23" y2="12"></line> <line x1="4.22" y1="19.78" x2="5.64" y2="18.36"></line> <line x1="18.36" y1="5.64" x2="19.78" y2="4.22"></line> </svg> </symbol> </svg> <script> document.documentElement.dataset.colour_scheme = localStorage.getItem("colour_scheme") || "auto" </script> <section id="pep-page-section"> <header> <h1>Python Enhancement Proposals</h1> <ul class="breadcrumbs"> <li><a href="https://www.python.org/" title="The Python Programming Language">Python</a> » </li> <li><a href="../pep-0000/">PEP Index</a> » </li> <li>PEP 655</li> </ul> <button id="colour-scheme-cycler" onClick="setColourScheme(nextColourScheme())"> <svg aria-hidden="true" class="colour-scheme-icon-when-auto"><use href="#svg-sun-half"></use></svg> <svg aria-hidden="true" class="colour-scheme-icon-when-dark"><use href="#svg-moon"></use></svg> <svg aria-hidden="true" class="colour-scheme-icon-when-light"><use href="#svg-sun"></use></svg> Toggle light / dark / auto colour theme </button> </header> <article> <section id="pep-content"> <h1 class="page-title">PEP 655 – Marking individual TypedDict items as required or potentially-missing</h1> <dl class="rfc2822 field-list simple"> <dt class="field-odd">Author:</dt> <dd class="field-odd">David Foster <david at dafoster.net></dd> <dt class="field-even">Sponsor:</dt> <dd class="field-even">Guido van Rossum <guido at python.org></dd> <dt class="field-odd">Discussions-To:</dt> <dd class="field-odd"><a class="reference external" href="https://mail.python.org/archives/list/typing-sig@python.org/thread/53XVOD5ZUKJ263MWA6AUPEA6J7LBBLNV/">Typing-SIG thread</a></dd> <dt class="field-even">Status:</dt> <dd class="field-even"><abbr title="Accepted and implementation complete, or no longer active">Final</abbr></dd> <dt class="field-odd">Type:</dt> <dd class="field-odd"><abbr title="Normative PEP with a new feature for Python, implementation change for CPython or interoperability standard for the ecosystem">Standards Track</abbr></dd> <dt class="field-even">Topic:</dt> <dd class="field-even"><a class="reference external" href="../topic/typing/">Typing</a></dd> <dt class="field-odd">Created:</dt> <dd class="field-odd">30-Jan-2021</dd> <dt class="field-even">Python-Version:</dt> <dd class="field-even">3.11</dd> <dt class="field-odd">Post-History:</dt> <dd class="field-odd">31-Jan-2021, 11-Feb-2021, 20-Feb-2021, 26-Feb-2021, 17-Jan-2022, 28-Jan-2022</dd> <dt class="field-even">Resolution:</dt> <dd class="field-even"><a class="reference external" href="https://mail.python.org/archives/list/python-dev@python.org/message/AJEDNVC3FXM5QXNNW5CR4UCT4KI5XVUE/">Python-Dev message</a></dd> </dl> <hr class="docutils" /> <section id="contents"> <details><summary>Table of Contents</summary><ul class="simple"> <li><a class="reference internal" href="#abstract">Abstract</a></li> <li><a class="reference internal" href="#motivation">Motivation</a></li> <li><a class="reference internal" href="#rationale">Rationale</a></li> <li><a class="reference internal" href="#specification">Specification</a><ul> <li><a class="reference internal" href="#interaction-with-total-false">Interaction with <code class="docutils literal notranslate">total=False</code></a></li> <li><a class="reference internal" href="#interaction-with-annotated">Interaction with <code class="docutils literal notranslate">Annotated[]</code></a></li> <li><a class="reference internal" href="#runtime-behavior">Runtime behavior</a><ul> <li><a class="reference internal" href="#interaction-with-get-type-hints">Interaction with <code class="docutils literal notranslate">get_type_hints()</code></a></li> <li><a class="reference internal" href="#interaction-with-get-origin-and-get-args">Interaction with <code class="docutils literal notranslate">get_origin()</code> and <code class="docutils literal notranslate">get_args()</code></a></li> <li><a class="reference internal" href="#interaction-with-required-keys-and-optional-keys">Interaction with <code class="docutils literal notranslate">__required_keys__</code> and <code class="docutils literal notranslate">__optional_keys__</code></a></li> </ul> </li> </ul> </li> <li><a class="reference internal" href="#backwards-compatibility">Backwards Compatibility</a></li> <li><a class="reference internal" href="#how-to-teach-this">How to Teach This</a><ul> <li><a class="reference internal" href="#usage-in-python-3-11">Usage in Python <3.11</a></li> </ul> </li> <li><a class="reference internal" href="#reference-implementation">Reference Implementation</a></li> <li><a class="reference internal" href="#rejected-ideas">Rejected Ideas</a><ul> <li><a class="reference internal" href="#special-syntax-around-the-key-of-a-typeddict-item">Special syntax around the key of a TypedDict item</a></li> <li><a class="reference internal" href="#marking-required-or-potentially-missing-keys-with-an-operator">Marking required or potentially-missing keys with an operator</a></li> <li><a class="reference internal" href="#marking-absence-of-a-value-with-a-special-constant">Marking absence of a value with a special constant</a><ul> <li><a class="reference internal" href="#misalignment-with-how-unions-apply-to-values">Misalignment with how unions apply to values</a></li> <li><a class="reference internal" href="#misalignment-with-how-unions-are-subdivided">Misalignment with how unions are subdivided</a></li> <li><a class="reference internal" href="#difficult-to-implement">Difficult to implement</a></li> <li><a class="reference internal" href="#introduces-a-second-null-like-value-into-python">Introduces a second null-like value into Python</a></li> </ul> </li> <li><a class="reference internal" href="#replace-optional-with-nullable-repurpose-optional-to-mean-optional-item">Replace Optional with Nullable. Repurpose Optional to mean “optional item”.</a></li> <li><a class="reference internal" href="#change-optional-to-mean-optional-item-in-certain-contexts-instead-of-nullable">Change Optional to mean “optional item” in certain contexts instead of “nullable”</a></li> <li><a class="reference internal" href="#various-synonyms-for-potentially-missing-item">Various synonyms for “potentially-missing item”</a></li> </ul> </li> <li><a class="reference internal" href="#references">References</a></li> <li><a class="reference internal" href="#copyright">Copyright</a></li> </ul> </details></section> <div class="pep-banner canonical-typing-spec sticky-banner admonition attention"> Attention This PEP is a historical document: see <a class="reference external" href="https://typing.readthedocs.io/en/latest/spec/typeddict.html#required-notrequired" title="(in typing)">Required and NotRequired</a>, <a class="reference external" href="https://docs.python.org/3/library/typing.html#typing.Required" title="(in Python v3.13)"><code class="xref py py-data docutils literal notranslate">typing.Required</code></a> and <a class="reference external" href="https://docs.python.org/3/library/typing.html#typing.NotRequired" title="(in Python v3.13)"><code class="xref py py-data docutils literal notranslate">typing.NotRequired</code></a> for up-to-date specs and documentation. Canonical typing specs are maintained at the <a class="reference external" href="https://typing.readthedocs.io/en/latest/spec/">typing specs site</a>; runtime typing behaviour is described in the CPython documentation. × See the <a class="reference external" href="https://typing.readthedocs.io/en/latest/spec/meta.html">typing specification update process</a> for how to propose changes to the typing spec. </div> <section id="abstract"> <h2><a class="toc-backref" href="#abstract" role="doc-backlink">Abstract</a></h2> <a class="pep reference internal" href="../pep-0589/" title="PEP 589 – TypedDict: Type Hints for Dictionaries with a Fixed Set of Keys">PEP 589</a> defines notation for declaring a TypedDict with all required keys and notation for defining a TypedDict with <a class="pep reference internal" href="../pep-0589/#totality" title="PEP 589 – TypedDict: Type Hints for Dictionaries with a Fixed Set of Keys § Totality">all potentially-missing keys</a>, however it does not provide a mechanism to declare some keys as required and others as potentially-missing. This PEP introduces two new notations: <code class="docutils literal notranslate">Required[]</code>, which can be used on individual items of a TypedDict to mark them as required, and <code class="docutils literal notranslate">NotRequired[]</code>, which can be used on individual items to mark them as potentially-missing. This PEP makes no Python grammar changes. Correct usage of required and potentially-missing keys of TypedDicts is intended to be enforced only by static type checkers and need not be enforced by Python itself at runtime. </section> <section id="motivation"> <h2><a class="toc-backref" href="#motivation" role="doc-backlink">Motivation</a></h2> It is not uncommon to want to define a TypedDict with some keys that are required and others that are potentially-missing. Currently the only way to define such a TypedDict is to declare one TypedDict with one value for <code class="docutils literal notranslate">total</code> and then inherit it from another TypedDict with a different value for <code class="docutils literal notranslate">total</code>: <div class="highlight-default notranslate"><div class="highlight"><pre>class _MovieBase(TypedDict): # implicitly total=True title: str class Movie(_MovieBase, total=False): year: int </pre></div> </div> Having to declare two different TypedDict types for this purpose is cumbersome. This PEP introduces two new type qualifiers, <code class="docutils literal notranslate">typing.Required</code> and <code class="docutils literal notranslate">typing.NotRequired</code>, which allow defining a single TypedDict with a mix of both required and potentially-missing keys: <div class="highlight-default notranslate"><div class="highlight"><pre>class Movie(TypedDict): title: str year: NotRequired[int] </pre></div> </div> This PEP also makes it possible to define TypedDicts in the <a class="pep reference internal" href="../pep-0589/#alternative-syntax" title="PEP 589 – TypedDict: Type Hints for Dictionaries with a Fixed Set of Keys § Alternative Syntax">alternative functional syntax</a> with a mix of required and potentially-missing keys, which is not currently possible at all because the alternative syntax does not support inheritance: <div class="highlight-default notranslate"><div class="highlight"><pre>Actor = TypedDict('Actor', { 'name': str, # "in" is a keyword, so the functional syntax is necessary 'in': NotRequired[List[str]], }) </pre></div> </div> </section> <section id="rationale"> <h2><a class="toc-backref" href="#rationale" role="doc-backlink">Rationale</a></h2> One might think it unusual to propose notation that prioritizes marking required keys rather than potentially-missing keys, as is customary in other languages like TypeScript: <div class="highlight-typescript notranslate"><div class="highlight"><pre>interface Movie { title: string; year?: number; // ? marks potentially-missing keys } </pre></div> </div> The difficulty is that the best word for marking a potentially-missing key, <code class="docutils literal notranslate">Optional[]</code>, is already used in Python for a completely different purpose: marking values that could be either of a particular type or <code class="docutils literal notranslate">None</code>. In particular the following does not work: <div class="highlight-default notranslate"><div class="highlight"><pre>class Movie(TypedDict): ... year: Optional[int] # means int|None, not potentially-missing! </pre></div> </div> Attempting to use any synonym of “optional” to mark potentially-missing keys (like <code class="docutils literal notranslate">Missing[]</code>) would be too similar to <code class="docutils literal notranslate">Optional[]</code> and be easy to confuse with it. Thus it was decided to focus on positive-form phrasing for required keys instead, which is straightforward to spell as <code class="docutils literal notranslate">Required[]</code>. Nevertheless it is common for folks wanting to extend a regular (<code class="docutils literal notranslate">total=True</code>) TypedDict to only want to add a small number of potentially-missing keys, which necessitates a way to mark keys that are not required and potentially-missing, and so we also allow the <code class="docutils literal notranslate">NotRequired[]</code> form for that case. </section> <section id="specification"> <h2><a class="toc-backref" href="#specification" role="doc-backlink">Specification</a></h2> The <code class="docutils literal notranslate">typing.Required</code> type qualifier is used to indicate that a variable declared in a TypedDict definition is a required key: <div class="highlight-default notranslate"><div class="highlight"><pre>class Movie(TypedDict, total=False): title: Required[str] year: int </pre></div> </div> Additionally the <code class="docutils literal notranslate">typing.NotRequired</code> type qualifier is used to indicate that a variable declared in a TypedDict definition is a potentially-missing key: <div class="highlight-default notranslate"><div class="highlight"><pre>class Movie(TypedDict): # implicitly total=True title: str year: NotRequired[int] </pre></div> </div> It is an error to use <code class="docutils literal notranslate">Required[]</code> or <code class="docutils literal notranslate">NotRequired[]</code> in any location that is not an item of a TypedDict. Type checkers must enforce this restriction. It is valid to use <code class="docutils literal notranslate">Required[]</code> and <code class="docutils literal notranslate">NotRequired[]</code> even for items where it is redundant, to enable additional explicitness if desired: <div class="highlight-default notranslate"><div class="highlight"><pre>class Movie(TypedDict): title: Required[str] # redundant year: NotRequired[int] </pre></div> </div> It is an error to use both <code class="docutils literal notranslate">Required[]</code> and <code class="docutils literal notranslate">NotRequired[]</code> at the same time: <div class="highlight-default notranslate"><div class="highlight"><pre>class Movie(TypedDict): title: str year: NotRequired[Required[int]] # ERROR </pre></div> </div> Type checkers must enforce this restriction. The runtime implementations of <code class="docutils literal notranslate">Required[]</code> and <code class="docutils literal notranslate">NotRequired[]</code> may also enforce this restriction. The <a class="pep reference internal" href="../pep-0589/#alternative-syntax" title="PEP 589 – TypedDict: Type Hints for Dictionaries with a Fixed Set of Keys § Alternative Syntax">alternative functional syntax</a> for TypedDict also supports <code class="docutils literal notranslate">Required[]</code> and <code class="docutils literal notranslate">NotRequired[]</code>: <div class="highlight-default notranslate"><div class="highlight"><pre>Movie = TypedDict('Movie', {'name': str, 'year': NotRequired[int]}) </pre></div> </div> <section id="interaction-with-total-false"> <h3><a class="toc-backref" href="#interaction-with-total-false" role="doc-backlink">Interaction with <code class="docutils literal notranslate">total=False</code></a></h3> Any <a class="pep reference internal" href="../pep-0589/" title="PEP 589 – TypedDict: Type Hints for Dictionaries with a Fixed Set of Keys">PEP 589</a>-style TypedDict declared with <code class="docutils literal notranslate">total=False</code> is equivalent to a TypedDict with an implicit <code class="docutils literal notranslate">total=True</code> definition with all of its keys marked as <code class="docutils literal notranslate">NotRequired[]</code>. Therefore: <div class="highlight-default notranslate"><div class="highlight"><pre>class _MovieBase(TypedDict): # implicitly total=True title: str class Movie(_MovieBase, total=False): year: int </pre></div> </div> is equivalent to: <div class="highlight-default notranslate"><div class="highlight"><pre>class _MovieBase(TypedDict): title: str class Movie(_MovieBase): year: NotRequired[int] </pre></div> </div> </section> <section id="interaction-with-annotated"> <h3><a class="toc-backref" href="#interaction-with-annotated" role="doc-backlink">Interaction with <code class="docutils literal notranslate">Annotated[]</code></a></h3> <code class="docutils literal notranslate">Required[]</code> and <code class="docutils literal notranslate">NotRequired[]</code> can be used with <code class="docutils literal notranslate">Annotated[]</code>, in any nesting order: <div class="highlight-default notranslate"><div class="highlight"><pre>class Movie(TypedDict): title: str year: NotRequired[Annotated[int, ValueRange(-9999, 9999)]] # ok </pre></div> </div> <div class="highlight-default notranslate"><div class="highlight"><pre>class Movie(TypedDict): title: str year: Annotated[NotRequired[int], ValueRange(-9999, 9999)] # ok </pre></div> </div> In particular allowing <code class="docutils literal notranslate">Annotated[]</code> to be the outermost annotation for an item allows better interoperability with non-typing uses of annotations, which may always want <code class="docutils literal notranslate">Annotated[]</code> as the outermost annotation. <a class="footnote-reference brackets" href="#id6" id="id1">[3]</a> </section> <section id="runtime-behavior"> <h3><a class="toc-backref" href="#runtime-behavior" role="doc-backlink">Runtime behavior</a></h3> <section id="interaction-with-get-type-hints"> <h4><a class="toc-backref" href="#interaction-with-get-type-hints" role="doc-backlink">Interaction with <code class="docutils literal notranslate">get_type_hints()</code></a></h4> <code class="docutils literal notranslate">typing.get_type_hints(...)</code> applied to a TypedDict will by default strip out any <code class="docutils literal notranslate">Required[]</code> or <code class="docutils literal notranslate">NotRequired[]</code> type qualifiers, since these qualifiers are expected to be inconvenient for code casually introspecting type annotations. <code class="docutils literal notranslate">typing.get_type_hints(..., include_extras=True)</code> however will retain <code class="docutils literal notranslate">Required[]</code> and <code class="docutils literal notranslate">NotRequired[]</code> type qualifiers, for advanced code introspecting type annotations that wishes to preserve all annotations in the original source: <div class="highlight-default notranslate"><div class="highlight"><pre>class Movie(TypedDict): title: str year: NotRequired[int] assert get_type_hints(Movie) == \ {'title': str, 'year': int} assert get_type_hints(Movie, include_extras=True) == \ {'title': str, 'year': NotRequired[int]} </pre></div> </div> </section> <section id="interaction-with-get-origin-and-get-args"> <h4><a class="toc-backref" href="#interaction-with-get-origin-and-get-args" role="doc-backlink">Interaction with <code class="docutils literal notranslate">get_origin()</code> and <code class="docutils literal notranslate">get_args()</code></a></h4> <code class="docutils literal notranslate">typing.get_origin()</code> and <code class="docutils literal notranslate">typing.get_args()</code> will be updated to recognize <code class="docutils literal notranslate">Required[]</code> and <code class="docutils literal notranslate">NotRequired[]</code>: <div class="highlight-default notranslate"><div class="highlight"><pre>assert get_origin(Required[int]) is Required assert get_args(Required[int]) == (int,) assert get_origin(NotRequired[int]) is NotRequired assert get_args(NotRequired[int]) == (int,) </pre></div> </div> </section> <section id="interaction-with-required-keys-and-optional-keys"> <h4><a class="toc-backref" href="#interaction-with-required-keys-and-optional-keys" role="doc-backlink">Interaction with <code class="docutils literal notranslate">__required_keys__</code> and <code class="docutils literal notranslate">__optional_keys__</code></a></h4> An item marked with <code class="docutils literal notranslate">Required[]</code> will always appear in the <code class="docutils literal notranslate">__required_keys__</code> for its enclosing TypedDict. Similarly an item marked with <code class="docutils literal notranslate">NotRequired[]</code> will always appear in <code class="docutils literal notranslate">__optional_keys__</code>. <div class="highlight-default notranslate"><div class="highlight"><pre>assert Movie.__required_keys__ == frozenset({'title'}) assert Movie.__optional_keys__ == frozenset({'year'}) </pre></div> </div> </section> </section> </section> <section id="backwards-compatibility"> <h2><a class="toc-backref" href="#backwards-compatibility" role="doc-backlink">Backwards Compatibility</a></h2> No backward incompatible changes are made by this PEP. </section> <section id="how-to-teach-this"> <h2><a class="toc-backref" href="#how-to-teach-this" role="doc-backlink">How to Teach This</a></h2> To define a TypedDict where most keys are required and some are potentially-missing, define a single TypedDict as normal (without the <code class="docutils literal notranslate">total</code> keyword) and mark those few keys that are potentially-missing with <code class="docutils literal notranslate">NotRequired[]</code>. To define a TypedDict where most keys are potentially-missing and a few are required, define a <code class="docutils literal notranslate">total=False</code> TypedDict and mark those few keys that are required with <code class="docutils literal notranslate">Required[]</code>. If some items accept <code class="docutils literal notranslate">None</code> in addition to a regular value, it is recommended that the <code class="docutils literal notranslate">TYPE|None</code> notation be preferred over <code class="docutils literal notranslate">Optional[TYPE]</code> for marking such item values, to avoid using <code class="docutils literal notranslate">Required[]</code> or <code class="docutils literal notranslate">NotRequired[]</code> alongside <code class="docutils literal notranslate">Optional[]</code> within the same TypedDict definition: Yes: <div class="good highlight-default notranslate"><div class="highlight"><pre>from __future__ import annotations # for Python 3.7-3.9 class Dog(TypedDict): name: str owner: NotRequired[str|None] </pre></div> </div> Okay (required for Python 3.5.3-3.6): <div class="maybe highlight-default notranslate"><div class="highlight"><pre>class Dog(TypedDict): name: str owner: 'NotRequired[str|None]' </pre></div> </div> No: <div class="bad highlight-default notranslate"><div class="highlight"><pre>class Dog(TypedDict): name: str # ick; avoid using both Optional and NotRequired owner: NotRequired[Optional[str]] </pre></div> </div> <section id="usage-in-python-3-11"> <h3><a class="toc-backref" href="#usage-in-python-3-11" role="doc-backlink">Usage in Python <3.11</a></h3> If your code supports Python <3.11 and wishes to use <code class="docutils literal notranslate">Required[]</code> or <code class="docutils literal notranslate">NotRequired[]</code> then it should use <code class="docutils literal notranslate">typing_extensions.TypedDict</code> rather than <code class="docutils literal notranslate">typing.TypedDict</code> because the latter will not understand <code class="docutils literal notranslate">(Not)Required[]</code>. In particular <code class="docutils literal notranslate">__required_keys__</code> and <code class="docutils literal notranslate">__optional_keys__</code> on the resulting TypedDict type will not be correct: Yes (Python 3.11+ only): <div class="good highlight-default notranslate"><div class="highlight"><pre>from typing import NotRequired, TypedDict class Dog(TypedDict): name: str owner: NotRequired[str|None] </pre></div> </div> Yes (Python <3.11 and 3.11+): <div class="good highlight-default notranslate"><div class="highlight"><pre>from __future__ import annotations # for Python 3.7-3.9 from typing_extensions import NotRequired, TypedDict # for Python <3.11 with (Not)Required class Dog(TypedDict): name: str owner: NotRequired[str|None] </pre></div> </div> No (Python <3.11 and 3.11+): <div class="bad highlight-default notranslate"><div class="highlight"><pre>from typing import TypedDict # oops: should import from typing_extensions instead from typing_extensions import NotRequired class Movie(TypedDict): title: str year: NotRequired[int] assert Movie.__required_keys__ == frozenset({'title', 'year'}) # yikes assert Movie.__optional_keys__ == frozenset() # yikes </pre></div> </div> </section> </section> <section id="reference-implementation"> <h2><a class="toc-backref" href="#reference-implementation" role="doc-backlink">Reference Implementation</a></h2> The <a class="reference external" href="http://www.mypy-lang.org/">mypy</a> <a class="reference external" href="https://mypy-lang.blogspot.com/2021/12/mypy-0930-released.html">0.930</a>, <a class="reference external" href="https://github.com/Microsoft/pyright">pyright</a> <a class="reference external" href="https://github.com/microsoft/pyright/commit/7ed245b1845173090c6404e49912e8cbfb3417c8">1.1.117</a>, and <a class="reference external" href="https://github.com/quora/pyanalyze">pyanalyze</a> <a class="reference external" href="https://pyanalyze.readthedocs.io/en/latest/changelog.html#version-0-4-0-november-18-2021">0.4.0</a> type checkers support <code class="docutils literal notranslate">Required</code> and <code class="docutils literal notranslate">NotRequired</code>. A reference implementation of the runtime component is provided in the <a class="reference external" href="https://github.com/python/typing/tree/master/typing_extensions">typing_extensions</a> module. </section> <section id="rejected-ideas"> <h2><a class="toc-backref" href="#rejected-ideas" role="doc-backlink">Rejected Ideas</a></h2> <section id="special-syntax-around-the-key-of-a-typeddict-item"> <h3><a class="toc-backref" href="#special-syntax-around-the-key-of-a-typeddict-item" role="doc-backlink">Special syntax around the key of a TypedDict item</a></h3> <div class="highlight-default notranslate"><div class="highlight"><pre>class MyThing(TypedDict): opt1?: str # may not exist, but if exists, value is string opt2: Optional[str] # always exists, but may have None value </pre></div> </div> This notation would require Python grammar changes and it is not believed that marking TypedDict items as required or potentially-missing would meet the high bar required to make such grammar changes. <div class="highlight-default notranslate"><div class="highlight"><pre>class MyThing(TypedDict): Optional[opt1]: str # may not exist, but if exists, value is string opt2: Optional[str] # always exists, but may have None value </pre></div> </div> This notation causes <code class="docutils literal notranslate">Optional[]</code> to take on different meanings depending on where it is positioned, which is inconsistent and confusing. Also, “let’s just not put funny syntax before the colon.” <a class="footnote-reference brackets" href="#id4" id="id2">[1]</a> </section> <section id="marking-required-or-potentially-missing-keys-with-an-operator"> <h3><a class="toc-backref" href="#marking-required-or-potentially-missing-keys-with-an-operator" role="doc-backlink">Marking required or potentially-missing keys with an operator</a></h3> We could use unary <code class="docutils literal notranslate">+</code> as shorthand to mark a required key, unary <code class="docutils literal notranslate">-</code> to mark a potentially-missing key, or unary <code class="docutils literal notranslate">~</code> to mark a key with opposite-of-normal totality: <div class="highlight-default notranslate"><div class="highlight"><pre>class MyThing(TypedDict, total=False): req1: +int # + means a required key, or Required[] opt1: str req2: +float class MyThing(TypedDict): req1: int opt1: -str # - means a potentially-missing key, or NotRequired[] req2: float class MyThing(TypedDict): req1: int opt1: ~str # ~ means a opposite-of-normal-totality key req2: float </pre></div> </div> Such operators could be implemented on <code class="docutils literal notranslate">type</code> via the <code class="docutils literal notranslate">__pos__</code>, <code class="docutils literal notranslate">__neg__</code> and <code class="docutils literal notranslate">__invert__</code> special methods without modifying the grammar. It was decided that it would be prudent to introduce long-form notation (i.e. <code class="docutils literal notranslate">Required[]</code> and <code class="docutils literal notranslate">NotRequired[]</code>) before introducing any short-form notation. Future PEPs may reconsider introducing this or other short-form notation options. Note when reconsidering introducing this short-form notation that <code class="docutils literal notranslate">+</code>, <code class="docutils literal notranslate">-</code>, and <code class="docutils literal notranslate">~</code> already have existing meanings in the Python typing world: covariant, contravariant, and invariant: <div class="highlight-default notranslate"><div class="highlight"><pre>>>> from typing import TypeVar >>> (TypeVar('T', covariant=True), TypeVar('U', contravariant=True), TypeVar('V')) (+T, -U, ~V) </pre></div> </div> </section> <section id="marking-absence-of-a-value-with-a-special-constant"> <h3><a class="toc-backref" href="#marking-absence-of-a-value-with-a-special-constant" role="doc-backlink">Marking absence of a value with a special constant</a></h3> We could introduce a new type-level constant which signals the absence of a value when used as a union member, similar to JavaScript’s <code class="docutils literal notranslate">undefined</code> type, perhaps called <code class="docutils literal notranslate">Missing</code>: <div class="highlight-default notranslate"><div class="highlight"><pre>class MyThing(TypedDict): req1: int opt1: str|Missing req2: float </pre></div> </div> Such a <code class="docutils literal notranslate">Missing</code> constant could also be used for other scenarios such as the type of a variable which is only conditionally defined: <div class="highlight-default notranslate"><div class="highlight"><pre>class MyClass: attr: int|Missing def __init__(self, set_attr: bool) -> None: if set_attr: self.attr = 10 </pre></div> </div> <div class="highlight-default notranslate"><div class="highlight"><pre>def foo(set_attr: bool) -> None: if set_attr: attr = 10 reveal_type(attr) # int|Missing </pre></div> </div> <section id="misalignment-with-how-unions-apply-to-values"> <h4><a class="toc-backref" href="#misalignment-with-how-unions-apply-to-values" role="doc-backlink">Misalignment with how unions apply to values</a></h4> However this use of <code class="docutils literal notranslate">...|Missing</code>, equivalent to <code class="docutils literal notranslate">Union[..., Missing]</code>, doesn’t align well with what a union normally means: <code class="docutils literal notranslate">Union[...]</code> always describes the type of a value that is present. By contrast missingness or non-totality is a property of a variable instead. Current precedent for marking properties of a variable include <code class="docutils literal notranslate">Final[...]</code> and <code class="docutils literal notranslate">ClassVar[...]</code>, which the proposal for <code class="docutils literal notranslate">Required[...]</code> is aligned with. </section> <section id="misalignment-with-how-unions-are-subdivided"> <h4><a class="toc-backref" href="#misalignment-with-how-unions-are-subdivided" role="doc-backlink">Misalignment with how unions are subdivided</a></h4> Furthermore the use of <code class="docutils literal notranslate">Union[..., Missing]</code> doesn’t align with the usual ways that union values are broken down: Normally you can eliminate components of a union type using <code class="docutils literal notranslate">isinstance</code> checks: <div class="highlight-default notranslate"><div class="highlight"><pre>class Packet: data: Union[str, bytes] def send_data(packet: Packet) -> None: if isinstance(packet.data, str): reveal_type(packet.data) # str packet_bytes = packet.data.encode('utf-8') else: reveal_type(packet.data) # bytes packet_bytes = packet.data socket.send(packet_bytes) </pre></div> </div> However if we were to allow <code class="docutils literal notranslate">Union[..., Missing]</code> you’d either have to eliminate the <code class="docutils literal notranslate">Missing</code> case with <code class="docutils literal notranslate">hasattr</code> for object attributes: <div class="highlight-default notranslate"><div class="highlight"><pre>class Packet: data: Union[str, Missing] def send_data(packet: Packet) -> None: if hasattr(packet, 'data'): reveal_type(packet.data) # str packet_bytes = packet.data.encode('utf-8') else: reveal_type(packet.data) # Missing? error? packet_bytes = b'' socket.send(packet_bytes) </pre></div> </div> or a check against <code class="docutils literal notranslate">locals()</code> for local variables: <div class="highlight-default notranslate"><div class="highlight"><pre>def send_data(packet_data: Optional[str]) -> None: packet_bytes: Union[str, Missing] if packet_data is not None: packet_bytes = packet.data.encode('utf-8') if 'packet_bytes' in locals(): reveal_type(packet_bytes) # bytes socket.send(packet_bytes) else: reveal_type(packet_bytes) # Missing? error? </pre></div> </div> or a check via other means, such as against <code class="docutils literal notranslate">globals()</code> for global variables: <div class="highlight-default notranslate"><div class="highlight"><pre>warning: Union[str, Missing] import sys if sys.version_info < (3, 6): warning = 'Your version of Python is unsupported!' if 'warning' in globals(): reveal_type(warning) # str print(warning) else: reveal_type(warning) # Missing? error? </pre></div> </div> Weird and inconsistent. <code class="docutils literal notranslate">Missing</code> is not really a value at all; it’s an absence of definition and such an absence should be treated specially. </section> <section id="difficult-to-implement"> <h4><a class="toc-backref" href="#difficult-to-implement" role="doc-backlink">Difficult to implement</a></h4> Eric Traut from the Pyright type checker team has stated that implementing a <code class="docutils literal notranslate">Union[..., Missing]</code>-style notation would be difficult. <a class="footnote-reference brackets" href="#id5" id="id3">[2]</a> </section> <section id="introduces-a-second-null-like-value-into-python"> <h4><a class="toc-backref" href="#introduces-a-second-null-like-value-into-python" role="doc-backlink">Introduces a second null-like value into Python</a></h4> Defining a new <code class="docutils literal notranslate">Missing</code> type-level constant would be very close to introducing a new <code class="docutils literal notranslate">Missing</code> value-level constant at runtime, creating a second null-like runtime value in addition to <code class="docutils literal notranslate">None</code>. Having two different null-like constants in Python (<code class="docutils literal notranslate">None</code> and <code class="docutils literal notranslate">Missing</code>) would be confusing. Many newcomers to JavaScript already have difficulty distinguishing between its analogous constants <code class="docutils literal notranslate">null</code> and <code class="docutils literal notranslate">undefined</code>. </section> </section> <section id="replace-optional-with-nullable-repurpose-optional-to-mean-optional-item"> <h3><a class="toc-backref" href="#replace-optional-with-nullable-repurpose-optional-to-mean-optional-item" role="doc-backlink">Replace Optional with Nullable. Repurpose Optional to mean “optional item”.</a></h3> <code class="docutils literal notranslate">Optional[]</code> is too ubiquitous to deprecate, although use of it may fade over time in favor of the <code class="docutils literal notranslate">T|None</code> notation specified by <a class="pep reference internal" href="../pep-0604/" title="PEP 604 – Allow writing union types as X | Y">PEP 604</a>. </section> <section id="change-optional-to-mean-optional-item-in-certain-contexts-instead-of-nullable"> <h3><a class="toc-backref" href="#change-optional-to-mean-optional-item-in-certain-contexts-instead-of-nullable" role="doc-backlink">Change Optional to mean “optional item” in certain contexts instead of “nullable”</a></h3> Consider the use of a special flag on a TypedDict definition to alter the interpretation of <code class="docutils literal notranslate">Optional</code> inside the TypedDict to mean “optional item” rather than its usual meaning of “nullable”: <div class="highlight-default notranslate"><div class="highlight"><pre>class MyThing(TypedDict, optional_as_missing=True): req1: int opt1: Optional[str] </pre></div> </div> or: <div class="highlight-default notranslate"><div class="highlight"><pre>class MyThing(TypedDict, optional_as_nullable=False): req1: int opt1: Optional[str] </pre></div> </div> This would add more confusion for users because it would mean that in some contexts the meaning of <code class="docutils literal notranslate">Optional[]</code> is different than in other contexts, and it would be easy to overlook the flag. </section> <section id="various-synonyms-for-potentially-missing-item"> <h3><a class="toc-backref" href="#various-synonyms-for-potentially-missing-item" role="doc-backlink">Various synonyms for “potentially-missing item”</a></h3> <ul class="simple"> <li>Omittable – too easy to confuse with optional</li> <li>OptionalItem, OptionalKey – two words; too easy to confuse with optional</li> <li>MayExist, MissingOk – two words</li> <li>Droppable – too similar to Rust’s <code class="docutils literal notranslate">Drop</code>, which means something different</li> <li>Potential – too vague</li> <li>Open – sounds like applies to an entire structure rather then to an item</li> <li>Excludable</li> <li>Checked</li> </ul> </section> </section> <section id="references"> <h2><a class="toc-backref" href="#references" role="doc-backlink">References</a></h2> <aside class="footnote-list brackets"> <aside class="footnote brackets" id="id4" role="doc-footnote"> <dt class="label" id="id4">[<a href="#id2">1</a>]</dt> <dd><a class="reference external" href="https://mail.python.org/archives/list/typing-sig@python.org/message/4I3GPIWDUKV6GUCHDMORGUGRE4F4SXGR/">https://mail.python.org/archives/list/typing-sig@python.org/message/4I3GPIWDUKV6GUCHDMORGUGRE4F4SXGR/</a></aside> <aside class="footnote brackets" id="id5" role="doc-footnote"> <dt class="label" id="id5">[<a href="#id3">2</a>]</dt> <dd><a class="reference external" href="https://mail.python.org/archives/list/typing-sig@python.org/message/S2VJSVG6WCIWPBZ54BOJPG56KXVSLZK6/">https://mail.python.org/archives/list/typing-sig@python.org/message/S2VJSVG6WCIWPBZ54BOJPG56KXVSLZK6/</a></aside> <aside class="footnote brackets" id="id6" role="doc-footnote"> <dt class="label" id="id6">[<a href="#id1">3</a>]</dt> <dd><a class="reference external" href="https://bugs.python.org/issue46491">https://bugs.python.org/issue46491</a></aside> </aside> </section> <section id="copyright"> <h2><a class="toc-backref" href="#copyright" role="doc-backlink">Copyright</a></h2> This document is placed in the public domain or under the CC0-1.0-Universal license, whichever is more permissive. </section> </section> <hr class="docutils" /> Source: <a class="reference external" href="https://github.com/python/peps/blob/main/peps/pep-0655.rst">https://github.com/python/peps/blob/main/peps/pep-0655.rst</a> Last modified: <a class="reference external" href="https://github.com/python/peps/commits/main/peps/pep-0655.rst">2024-06-16 22:42:44 GMT</a> </article> <nav id="pep-sidebar"> <h2>Contents</h2> <ul> <li><a class="reference internal" href="#abstract">Abstract</a></li> <li><a class="reference internal" href="#motivation">Motivation</a></li> <li><a class="reference internal" href="#rationale">Rationale</a></li> <li><a class="reference internal" href="#specification">Specification</a><ul> <li><a class="reference internal" href="#interaction-with-total-false">Interaction with <code class="docutils literal notranslate">total=False</code></a></li> <li><a class="reference internal" href="#interaction-with-annotated">Interaction with <code class="docutils literal notranslate">Annotated[]</code></a></li> <li><a class="reference internal" href="#runtime-behavior">Runtime behavior</a><ul> <li><a class="reference internal" href="#interaction-with-get-type-hints">Interaction with <code class="docutils literal notranslate">get_type_hints()</code></a></li> <li><a class="reference internal" href="#interaction-with-get-origin-and-get-args">Interaction with <code class="docutils literal notranslate">get_origin()</code> and <code class="docutils literal notranslate">get_args()</code></a></li> <li><a class="reference internal" href="#interaction-with-required-keys-and-optional-keys">Interaction with <code class="docutils literal notranslate">__required_keys__</code> and <code class="docutils literal notranslate">__optional_keys__</code></a></li> </ul> </li> </ul> </li> <li><a class="reference internal" href="#backwards-compatibility">Backwards Compatibility</a></li> <li><a class="reference internal" href="#how-to-teach-this">How to Teach This</a><ul> <li><a class="reference internal" href="#usage-in-python-3-11">Usage in Python <3.11</a></li> </ul> </li> <li><a class="reference internal" href="#reference-implementation">Reference Implementation</a></li> <li><a class="reference internal" href="#rejected-ideas">Rejected Ideas</a><ul> <li><a class="reference internal" href="#special-syntax-around-the-key-of-a-typeddict-item">Special syntax around the key of a TypedDict item</a></li> <li><a class="reference internal" href="#marking-required-or-potentially-missing-keys-with-an-operator">Marking required or potentially-missing keys with an operator</a></li> <li><a class="reference internal" href="#marking-absence-of-a-value-with-a-special-constant">Marking absence of a value with a special constant</a><ul> <li><a class="reference internal" href="#misalignment-with-how-unions-apply-to-values">Misalignment with how unions apply to values</a></li> <li><a class="reference internal" href="#misalignment-with-how-unions-are-subdivided">Misalignment with how unions are subdivided</a></li> <li><a class="reference internal" href="#difficult-to-implement">Difficult to implement</a></li> <li><a class="reference internal" href="#introduces-a-second-null-like-value-into-python">Introduces a second null-like value into Python</a></li> </ul> </li> <li><a class="reference internal" href="#replace-optional-with-nullable-repurpose-optional-to-mean-optional-item">Replace Optional with Nullable. Repurpose Optional to mean “optional item”.</a></li> <li><a class="reference internal" href="#change-optional-to-mean-optional-item-in-certain-contexts-instead-of-nullable">Change Optional to mean “optional item” in certain contexts instead of “nullable”</a></li> <li><a class="reference internal" href="#various-synonyms-for-potentially-missing-item">Various synonyms for “potentially-missing item”</a></li> </ul> </li> <li><a class="reference internal" href="#references">References</a></li> <li><a class="reference internal" href="#copyright">Copyright</a></li> </ul> <a id="source" href="https://github.com/python/peps/blob/main/peps/pep-0655.rst">Page Source (GitHub)</a> </nav> </section> <script src="../_static/colour_scheme.js"></script> <script src="../_static/wrap_tables.js"></script> <script src="../_static/sticky_banner.js"></script> </body> </html>