CINXE.COM

<!DOCTYPE html> <html lang="en"> <head> <meta charset="utf-8"> <meta name="viewport" content="width=device-width, initial-scale=1.0"> <meta name="color-scheme" content="light dark"> <title>PEP 472 – Support for indexing with keyword arguments | peps.python.org</title> <link rel="shortcut icon" href="../_static/py.png"> <link rel="canonical" href="https://peps.python.org/pep-0472/"> <link rel="stylesheet" href="../_static/style.css" type="text/css"> <link rel="stylesheet" href="../_static/mq.css" type="text/css"> <link rel="stylesheet" href="../_static/pygments.css" type="text/css" media="(prefers-color-scheme: light)" id="pyg-light"> <link rel="stylesheet" href="../_static/pygments_dark.css" type="text/css" media="(prefers-color-scheme: dark)" id="pyg-dark"> <link rel="alternate" type="application/rss+xml" title="Latest PEPs" href="https://peps.python.org/peps.rss"> <meta property="og:title" content='PEP 472 – Support for indexing with keyword arguments | peps.python.org'> <meta property="og:description" content="This PEP proposes an extension of the indexing operation to support keyword arguments. Notations in the form a[K=3,R=2] would become legal syntax. For future-proofing considerations, a[1:2, K=3, R=4] are considered and may be allowed as well, depending ..."> <meta property="og:type" content="website"> <meta property="og:url" content="https://peps.python.org/pep-0472/"> <meta property="og:site_name" content="Python Enhancement Proposals (PEPs)"> <meta property="og:image" content="https://peps.python.org/_static/og-image.png"> <meta property="og:image:alt" content="Python PEPs"> <meta property="og:image:width" content="200"> <meta property="og:image:height" content="200"> <meta name="description" content="This PEP proposes an extension of the indexing operation to support keyword arguments. Notations in the form a[K=3,R=2] would become legal syntax. For future-proofing considerations, a[1:2, K=3, R=4] are considered and may be allowed as well, depending ..."> <meta name="theme-color" content="#3776ab"> </head> <body> <svg xmlns="http://www.w3.org/2000/svg" style="display: none;"> <symbol id="svg-sun-half" viewBox="0 0 24 24" pointer-events="all"> <title>Following system colour scheme</title> <svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"> <circle cx="12" cy="12" r="9"></circle> <path d="M12 3v18m0-12l4.65-4.65M12 14.3l7.37-7.37M12 19.6l8.85-8.85"></path> </svg> </symbol> <symbol id="svg-moon" viewBox="0 0 24 24" pointer-events="all"> <title>Selected dark colour scheme</title> <svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"> <path stroke="none" d="M0 0h24v24H0z" fill="none"></path> <path d="M12 3c.132 0 .263 0 .393 0a7.5 7.5 0 0 0 7.92 12.446a9 9 0 1 1 -8.313 -12.454z"></path> </svg> </symbol> <symbol id="svg-sun" viewBox="0 0 24 24" pointer-events="all"> <title>Selected light colour scheme</title> <svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"> <circle cx="12" cy="12" r="5"></circle> <line x1="12" y1="1" x2="12" y2="3"></line> <line x1="12" y1="21" x2="12" y2="23"></line> <line x1="4.22" y1="4.22" x2="5.64" y2="5.64"></line> <line x1="18.36" y1="18.36" x2="19.78" y2="19.78"></line> <line x1="1" y1="12" x2="3" y2="12"></line> <line x1="21" y1="12" x2="23" y2="12"></line> <line x1="4.22" y1="19.78" x2="5.64" y2="18.36"></line> <line x1="18.36" y1="5.64" x2="19.78" y2="4.22"></line> </svg> </symbol> </svg> <script> document.documentElement.dataset.colour_scheme = localStorage.getItem("colour_scheme") || "auto" </script> <section id="pep-page-section"> <header> <h1>Python Enhancement Proposals</h1> <ul class="breadcrumbs"> <li><a href="https://www.python.org/" title="The Python Programming Language">Python</a> » </li> <li><a href="../pep-0000/">PEP Index</a> » </li> <li>PEP 472</li> </ul> <button id="colour-scheme-cycler" onClick="setColourScheme(nextColourScheme())"> <svg aria-hidden="true" class="colour-scheme-icon-when-auto"><use href="#svg-sun-half"></use></svg> <svg aria-hidden="true" class="colour-scheme-icon-when-dark"><use href="#svg-moon"></use></svg> <svg aria-hidden="true" class="colour-scheme-icon-when-light"><use href="#svg-sun"></use></svg> Toggle light / dark / auto colour theme </button> </header> <article> <section id="pep-content"> <h1 class="page-title">PEP 472 – Support for indexing with keyword arguments</h1> <dl class="rfc2822 field-list simple"> <dt class="field-odd">Author:</dt> <dd class="field-odd">Stefano Borini, Joseph Martinot-Lagarde</dd> <dt class="field-even">Discussions-To:</dt> <dd class="field-even"><a class="reference external" href="https://mail.python.org/archives/list/python-ideas@python.org/">Python-Ideas list</a></dd> <dt class="field-odd">Status:</dt> <dd class="field-odd"><abbr title="Formally declined and will not be accepted">Rejected</abbr></dd> <dt class="field-even">Type:</dt> <dd class="field-even"><abbr title="Normative PEP with a new feature for Python, implementation change for CPython or interoperability standard for the ecosystem">Standards Track</abbr></dd> <dt class="field-odd">Created:</dt> <dd class="field-odd">24-Jun-2014</dd> <dt class="field-even">Python-Version:</dt> <dd class="field-even">3.6</dd> <dt class="field-odd">Post-History:</dt> <dd class="field-odd">02-Jul-2014</dd> <dt class="field-even">Resolution:</dt> <dd class="field-even"><a class="reference external" href="https://mail.python.org/pipermail/python-dev/2019-March/156693.html">Python-Dev message</a></dd> </dl> <hr class="docutils" /> <section id="contents"> <details><summary>Table of Contents</summary><ul class="simple"> <li><a class="reference internal" href="#abstract">Abstract</a></li> <li><a class="reference internal" href="#motivation">Motivation</a></li> <li><a class="reference internal" href="#use-cases">Use cases</a></li> <li><a class="reference internal" href="#current-implementation">Current implementation</a></li> <li><a class="reference internal" href="#specifications">Specifications</a><ul> <li><a class="reference internal" href="#strategy-strict-dictionary">Strategy “Strict dictionary”</a><ul> <li><a class="reference internal" href="#pros">Pros</a></li> <li><a class="reference internal" href="#neutral">Neutral</a></li> <li><a class="reference internal" href="#cons">Cons</a></li> </ul> </li> <li><a class="reference internal" href="#strategy-mixed-dictionary">Strategy “mixed dictionary”</a><ul> <li><a class="reference internal" href="#id1">Pros</a></li> <li><a class="reference internal" href="#id2">Cons</a></li> </ul> </li> <li><a class="reference internal" href="#strategy-named-tuple">Strategy “named tuple”</a><ul> <li><a class="reference internal" href="#id3">Pros</a></li> <li><a class="reference internal" href="#id4">Cons</a></li> </ul> </li> <li><a class="reference internal" href="#strategy-new-argument-contents">Strategy “New argument contents”</a><ul> <li><a class="reference internal" href="#id5">Pros</a></li> <li><a class="reference internal" href="#id6">Cons</a></li> </ul> </li> <li><a class="reference internal" href="#strategy-kwargs-argument">Strategy “kwargs argument”</a><ul> <li><a class="reference internal" href="#id7">Pros</a></li> <li><a class="reference internal" href="#id8">Cons</a></li> </ul> </li> </ul> </li> <li><a class="reference internal" href="#c-interface">C interface</a></li> <li><a class="reference internal" href="#alternative-solutions">Alternative Solutions</a><ul> <li><a class="reference internal" href="#use-a-method">Use a method</a></li> <li><a class="reference internal" href="#emulate-requested-behavior-by-abusing-the-slice-object">Emulate requested behavior by abusing the slice object</a></li> <li><a class="reference internal" href="#pass-a-dictionary-as-an-additional-index">Pass a dictionary as an additional index</a></li> </ul> </li> <li><a class="reference internal" href="#additional-comments">Additional Comments</a><ul> <li><a class="reference internal" href="#relevance-of-ordering-of-keyword-arguments">Relevance of ordering of keyword arguments</a></li> <li><a class="reference internal" href="#need-for-homogeneity-of-behavior">Need for homogeneity of behavior</a></li> <li><a class="reference internal" href="#having-get-become-obsolete-for-indexing-with-default-fallback">Having .get() become obsolete for indexing with default fallback</a></li> <li><a class="reference internal" href="#on-degeneracy-of-notation">On degeneracy of notation</a></li> </ul> </li> <li><a class="reference internal" href="#references">References</a></li> <li><a class="reference internal" href="#copyright">Copyright</a></li> </ul> </details></section> <section id="abstract"> <h2><a class="toc-backref" href="#abstract" role="doc-backlink">Abstract</a></h2> This PEP proposes an extension of the indexing operation to support keyword arguments. Notations in the form <code class="docutils literal notranslate">a[K=3,R=2]</code> would become legal syntax. For future-proofing considerations, <code class="docutils literal notranslate">a[1:2, K=3, R=4]</code> are considered and may be allowed as well, depending on the choice for implementation. In addition to a change in the parser, the index protocol (<code class="docutils literal notranslate">__getitem__</code>, <code class="docutils literal notranslate">__setitem__</code> and <code class="docutils literal notranslate">__delitem__</code>) will also potentially require adaptation. </section> <section id="motivation"> <h2><a class="toc-backref" href="#motivation" role="doc-backlink">Motivation</a></h2> The indexing syntax carries a strong semantic content, differentiating it from a method call: it implies referring to a subset of data. We believe this semantic association to be important, and wish to expand the strategies allowed to refer to this data. As a general observation, the number of indices needed by an indexing operation depends on the dimensionality of the data: one-dimensional data (e.g. a list) requires one index (e.g. <code class="docutils literal notranslate">a[3]</code>), two-dimensional data (e.g. a matrix) requires two indices (e.g. <code class="docutils literal notranslate">a[2,3]</code>) and so on. Each index is a selector along one of the axes of the dimensionality, and the position in the index tuple is the metainformation needed to associate each index to the corresponding axis. The current python syntax focuses exclusively on position to express the association to the axes, and also contains syntactic sugar to refer to non-punctiform selection (slices) <div class="highlight-default notranslate"><div class="highlight"><pre>>>> a[3] # returns the fourth element of a >>> a[1:10:2] # slice notation (extract a non-trivial data subset) >>> a[3,2] # multiple indexes (for multidimensional arrays) </pre></div> </div> The additional notation proposed in this PEP would allow notations involving keyword arguments in the indexing operation, e.g. <div class="highlight-default notranslate"><div class="highlight"><pre>>>> a[K=3, R=2] </pre></div> </div> which would allow to refer to axes by conventional names. One must additionally consider the extended form that allows both positional and keyword specification <div class="highlight-default notranslate"><div class="highlight"><pre>>>> a[3,R=3,K=4] </pre></div> </div> This PEP will explore different strategies to enable the use of these notations. </section> <section id="use-cases"> <h2><a class="toc-backref" href="#use-cases" role="doc-backlink">Use cases</a></h2> The following practical use cases present two broad categories of usage of a keyworded specification: Indexing and contextual option. For indexing: <ol class="arabic"> <li>To provide a more communicative meaning to the index, preventing e.g. accidental inversion of indexes<div class="highlight-default notranslate"><div class="highlight"><pre>>>> gridValues[x=3, y=5, z=8] >>> rain[time=0:12, location=location] </pre></div> </div> </li> <li>In some domain, such as computational physics and chemistry, the use of a notation such as <code class="docutils literal notranslate">Basis[Z=5]</code> is a Domain Specific Language notation to represent a level of accuracy<div class="highlight-default notranslate"><div class="highlight"><pre>>>> low_accuracy_energy = computeEnergy(molecule, BasisSet[Z=3]) </pre></div> </div> In this case, the index operation would return a basis set at the chosen level of accuracy (represented by the parameter Z). The reason behind an indexing is that the BasisSet object could be internally represented as a numeric table, where rows (the “coefficient” axis, hidden to the user in this example) are associated to individual elements (e.g. row 0:5 contains coefficients for element 1, row 5:8 coefficients for element 2) and each column is associated to a given degree of accuracy (“accuracy” or “Z” axis) so that first column is low accuracy, second column is medium accuracy and so on. With that indexing, the user would obtain another object representing the contents of the column of the internal table for accuracy level 3. </li> </ol> Additionally, the keyword specification can be used as an option contextual to the indexing. Specifically: <ol class="arabic"> <li>A “default” option allows to specify a default return value when the index is not present<div class="highlight-default notranslate"><div class="highlight"><pre>>>> lst = [1, 2, 3] >>> value = lst[5, default=0] # value is 0 </pre></div> </div> </li> <li>For a sparse dataset, to specify an interpolation strategy to infer a missing point from e.g. its surrounding data.<div class="highlight-default notranslate"><div class="highlight"><pre>>>> value = array[1, 3, interpolate=spline_interpolator] </pre></div> </div> </li> <li>A unit could be specified with the same mechanism<div class="highlight-default notranslate"><div class="highlight"><pre>>>> value = array[1, 3, unit="degrees"] </pre></div> </div> </li> </ol> How the notation is interpreted is up to the implementing class. </section> <section id="current-implementation"> <h2><a class="toc-backref" href="#current-implementation" role="doc-backlink">Current implementation</a></h2> Currently, the indexing operation is handled by methods <code class="docutils literal notranslate">__getitem__</code>, <code class="docutils literal notranslate">__setitem__</code> and <code class="docutils literal notranslate">__delitem__</code>. These methods’ signature accept one argument for the index (with <code class="docutils literal notranslate">__setitem__</code> accepting an additional argument for the set value). In the following, we will analyze <code class="docutils literal notranslate">__getitem__(self, idx)</code> exclusively, with the same considerations implied for the remaining two methods. When an indexing operation is performed, <code class="docutils literal notranslate">__getitem__(self, idx)</code> is called. Traditionally, the full content between square brackets is turned into a single object passed to argument <code class="docutils literal notranslate">idx</code>: <ul class="simple"> <li>When a single element is passed, e.g. <code class="docutils literal notranslate">a[2]</code>, <code class="docutils literal notranslate">idx</code> will be <code class="docutils literal notranslate">2</code>.</li> <li>When multiple elements are passed, they must be separated by commas: <code class="docutils literal notranslate">a[2, 3]</code>. In this case, <code class="docutils literal notranslate">idx</code> will be a tuple <code class="docutils literal notranslate">(2, 3)</code>. With <code class="docutils literal notranslate">a[2, 3, "hello", {}]</code> <code class="docutils literal notranslate">idx</code> will be <code class="docutils literal notranslate">(2, 3, "hello", {})</code>.</li> <li>A slicing notation e.g. <code class="docutils literal notranslate">a[2:10]</code> will produce a slice object, or a tuple containing slice objects if multiple values were passed.</li> </ul> Except for its unique ability to handle slice notation, the indexing operation has similarities to a plain method call: it acts like one when invoked with only one element; If the number of elements is greater than one, the <code class="docutils literal notranslate">idx</code> argument behaves like a <code class="docutils literal notranslate">*args</code>. However, as stated in the Motivation section, an indexing operation has the strong semantic implication of extraction of a subset out of a larger set, which is not automatically associated to a regular method call unless appropriate naming is chosen. Moreover, its different visual style is important for readability. </section> <section id="specifications"> <h2><a class="toc-backref" href="#specifications" role="doc-backlink">Specifications</a></h2> The implementation should try to preserve the current signature for <code class="docutils literal notranslate">__getitem__</code>, or modify it in a backward-compatible way. We will present different alternatives, taking into account the possible cases that need to be addressed <div class="highlight-default notranslate"><div class="highlight"><pre>C0. a[1]; a[1,2] # Traditional indexing C1. a[Z=3] C2. a[Z=3, R=4] C3. a[1, Z=3] C4. a[1, Z=3, R=4] C5. a[1, 2, Z=3] C6. a[1, 2, Z=3, R=4] C7. a[1, Z=3, 2, R=4] # Interposed ordering </pre></div> </div> <section id="strategy-strict-dictionary"> <h3><a class="toc-backref" href="#strategy-strict-dictionary" role="doc-backlink">Strategy “Strict dictionary”</a></h3> This strategy acknowledges that <code class="docutils literal notranslate">__getitem__</code> is special in accepting only one object, and the nature of that object must be non-ambiguous in its specification of the axes: it can be either by order, or by name. As a result of this assumption, in presence of keyword arguments, the passed entity is a dictionary and all labels must be specified. <div class="highlight-default notranslate"><div class="highlight"><pre>C0. a[1]; a[1,2] -> idx = 1; idx = (1, 2) C1. a[Z=3] -> idx = {"Z": 3} C2. a[Z=3, R=4] -> idx = {"Z": 3, "R": 4} C3. a[1, Z=3] -> raise SyntaxError C4. a[1, Z=3, R=4] -> raise SyntaxError C5. a[1, 2, Z=3] -> raise SyntaxError C6. a[1, 2, Z=3, R=4] -> raise SyntaxError C7. a[1, Z=3, 2, R=4] -> raise SyntaxError </pre></div> </div> <section id="pros"> <h4><a class="toc-backref" href="#pros" role="doc-backlink">Pros</a></h4> <ul class="simple"> <li>Strong conceptual similarity between the tuple case and the dictionary case. In the first case, we are specifying a tuple, so we are naturally defining a plain set of values separated by commas. In the second, we are specifying a dictionary, so we are specifying a homogeneous set of key/value pairs, as in <code class="docutils literal notranslate">dict(Z=3, R=4)</code>;</li> <li>Simple and easy to parse on the <code class="docutils literal notranslate">__getitem__</code> side: if it gets a tuple, determine the axes using positioning. If it gets a dictionary, use the keywords.</li> <li>C interface does not need changes.</li> </ul> </section> <section id="neutral"> <h4><a class="toc-backref" href="#neutral" role="doc-backlink">Neutral</a></h4> <ul class="simple"> <li>Degeneracy of <code class="docutils literal notranslate">a[{"Z": 3, "R": 4}]</code> with <code class="docutils literal notranslate">a[Z=3, R=4]</code> means the notation is syntactic sugar.</li> </ul> </section> <section id="cons"> <h4><a class="toc-backref" href="#cons" role="doc-backlink">Cons</a></h4> <ul class="simple"> <li>Very strict.</li> <li>Destroys ordering of the passed arguments. Preserving the order would be possible with an OrderedDict as drafted by <a class="pep reference internal" href="../pep-0468/" title="PEP 468 – Preserving the order of **kwargs in a function.">PEP 468</a>.</li> <li>Does not allow use cases with mixed positional/keyword arguments such as <code class="docutils literal notranslate">a[1, 2, default=5]</code>.</li> </ul> </section> </section> <section id="strategy-mixed-dictionary"> <h3><a class="toc-backref" href="#strategy-mixed-dictionary" role="doc-backlink">Strategy “mixed dictionary”</a></h3> This strategy relaxes the above constraint to return a dictionary containing both numbers and strings as keys. <div class="highlight-default notranslate"><div class="highlight"><pre>C0. a[1]; a[1,2] -> idx = 1; idx = (1, 2) C1. a[Z=3] -> idx = {"Z": 3} C2. a[Z=3, R=4] -> idx = {"Z": 3, "R": 4} C3. a[1, Z=3] -> idx = { 0: 1, "Z": 3} C4. a[1, Z=3, R=4] -> idx = { 0: 1, "Z": 3, "R": 4} C5. a[1, 2, Z=3] -> idx = { 0: 1, 1: 2, "Z": 3} C6. a[1, 2, Z=3, R=4] -> idx = { 0: 1, 1: 2, "Z": 3, "R": 4} C7. a[1, Z=3, 2, R=4] -> idx = { 0: 1, "Z": 3, 2: 2, "R": 4} </pre></div> </div> <section id="id1"> <h4><a class="toc-backref" href="#id1" role="doc-backlink">Pros</a></h4> <ul class="simple"> <li>Opens for mixed cases.</li> </ul> </section> <section id="id2"> <h4><a class="toc-backref" href="#id2" role="doc-backlink">Cons</a></h4> <ul class="simple"> <li>Destroys ordering information for string keys. We have no way of saying if <code class="docutils literal notranslate">"Z"</code> in C7 was in position 1 or 3.</li> <li>Implies switching from a tuple to a dict as soon as one specified index has a keyword argument. May be confusing to parse.</li> </ul> </section> </section> <section id="strategy-named-tuple"> <h3><a class="toc-backref" href="#strategy-named-tuple" role="doc-backlink">Strategy “named tuple”</a></h3> Return a named tuple for <code class="docutils literal notranslate">idx</code> instead of a tuple. Keyword arguments would obviously have their stated name as key, and positional argument would have an underscore followed by their order: <div class="highlight-default notranslate"><div class="highlight"><pre>C0. a[1]; a[1,2] -> idx = 1; idx = (_0=1, _1=2) C1. a[Z=3] -> idx = (Z=3) C2. a[Z=3, R=2] -> idx = (Z=3, R=2) C3. a[1, Z=3] -> idx = (_0=1, Z=3) C4. a[1, Z=3, R=2] -> idx = (_0=1, Z=3, R=2) C5. a[1, 2, Z=3] -> idx = (_0=1, _2=2, Z=3) C6. a[1, 2, Z=3, R=4] -> (_0=1, _1=2, Z=3, R=4) C7. a[1, Z=3, 2, R=4] -> (_0=1, Z=3, _1=2, R=4) or (_0=1, Z=3, _2=2, R=4) or raise SyntaxError </pre></div> </div> The required typename of the namedtuple could be <code class="docutils literal notranslate">Index</code> or the name of the argument in the function definition, it keeps the ordering and is easy to analyse by using the <code class="docutils literal notranslate">_fields</code> attribute. It is backward compatible, provided that C0 with more than one entry now passes a namedtuple instead of a plain tuple. <section id="id3"> <h4><a class="toc-backref" href="#id3" role="doc-backlink">Pros</a></h4> <ul class="simple"> <li>Looks nice. namedtuple transparently replaces tuple and gracefully degrades to the old behavior.</li> <li>Does not require a change in the C interface</li> </ul> </section> <section id="id4"> <h4><a class="toc-backref" href="#id4" role="doc-backlink">Cons</a></h4> <ul class="simple"> <li>According to <a class="reference external" href="https://mail.python.org/pipermail/python-ideas/2013-June/021257.html">some sources</a> namedtuple is not well developed. To include it as such important object would probably require rework and improvement;</li> <li>The namedtuple fields, and thus the type, will have to change according to the passed arguments. This can be a performance bottleneck, and makes it impossible to guarantee that two subsequent index accesses get the same Index class;</li> <li>the <code class="docutils literal notranslate">_n</code> “magic” fields are a bit unusual, but ipython already uses them for result history.</li> <li>Python currently has no builtin namedtuple. The current one is available in the “collections” module in the standard library.</li> <li>Differently from a function, the two notations <code class="docutils literal notranslate">gridValues[x=3, y=5, z=8]</code> and <code class="docutils literal notranslate">gridValues[3,5,8]</code> would not gracefully match if the order is modified at call time (e.g. we ask for <code class="docutils literal notranslate">gridValues[y=5, z=8, x=3])</code>. In a function, we can pre-define argument names so that keyword arguments are properly matched. Not so in <code class="docutils literal notranslate">__getitem__</code>, leaving the task for interpreting and matching to <code class="docutils literal notranslate">__getitem__</code> itself.</li> </ul> </section> </section> <section id="strategy-new-argument-contents"> <h3><a class="toc-backref" href="#strategy-new-argument-contents" role="doc-backlink">Strategy “New argument contents”</a></h3> In the current implementation, when many arguments are passed to <code class="docutils literal notranslate">__getitem__</code>, they are grouped in a tuple and this tuple is passed to <code class="docutils literal notranslate">__getitem__</code> as the single argument <code class="docutils literal notranslate">idx</code>. This strategy keeps the current signature, but expands the range of variability in type and contents of <code class="docutils literal notranslate">idx</code> to more complex representations. We identify four possible ways to implement this strategy: <ul class="simple"> <li>P1: uses a single dictionary for the keyword arguments.</li> <li>P2: uses individual single-item dictionaries.</li> <li>P3: similar to P2, but replaces single-item dictionaries with a <code class="docutils literal notranslate">(key, value)</code> tuple.</li> <li>P4: similar to P2, but uses a special and additional new object: <code class="docutils literal notranslate">keyword()</code></li> </ul> Some of these possibilities lead to degenerate notations, i.e. indistinguishable from an already possible representation. Once again, the proposed notation becomes syntactic sugar for these representations. Under this strategy, the old behavior for C0 is unchanged. <div class="highlight-default notranslate"><div class="highlight"><pre>C0: a[1] -> idx = 1 # integer a[1,2] -> idx = (1,2) # tuple </pre></div> </div> In C1, we can use either a dictionary or a tuple to represent key and value pair for the specific indexing entry. We need to have a tuple with a tuple in C1 because otherwise we cannot differentiate <code class="docutils literal notranslate">a["Z", 3]</code> from <code class="docutils literal notranslate">a[Z=3]</code>. <div class="highlight-default notranslate"><div class="highlight"><pre>C1: a[Z=3] -> idx = {"Z": 3} # P1/P2 dictionary with single key or idx = (("Z", 3),) # P3 tuple of tuples or idx = keyword("Z", 3) # P4 keyword object </pre></div> </div> As you can see, notation P1/P2 implies that <code class="docutils literal notranslate">a[Z=3]</code> and <code class="docutils literal notranslate">a[{"Z": 3}]</code> will call <code class="docutils literal notranslate">__getitem__</code> passing the exact same value, and is therefore syntactic sugar for the latter. Same situation occurs, although with different index, for P3. Using a keyword object as in P4 would remove this degeneracy. For the C2 case: <div class="highlight-default notranslate"><div class="highlight"><pre>C2. a[Z=3, R=4] -> idx = {"Z": 3, "R": 4} # P1 dictionary/ordereddict or idx = ({"Z": 3}, {"R": 4}) # P2 tuple of two single-key dict or idx = (("Z", 3), ("R", 4)) # P3 tuple of tuples or idx = (keyword("Z", 3), keyword("R", 4) ) # P4 keyword objects </pre></div> </div> P1 naturally maps to the traditional <code class="docutils literal notranslate">**kwargs</code> behavior, however it breaks the convention that two or more entries for the index produce a tuple. P2 preserves this behavior, and additionally preserves the order. Preserving the order would also be possible with an OrderedDict as drafted by <a class="pep reference internal" href="../pep-0468/" title="PEP 468 – Preserving the order of **kwargs in a function.">PEP 468</a>. The remaining cases are here shown: <div class="highlight-default notranslate"><div class="highlight"><pre>C3. a[1, Z=3] -> idx = (1, {"Z": 3}) # P1/P2 or idx = (1, ("Z", 3)) # P3 or idx = (1, keyword("Z", 3)) # P4 C4. a[1, Z=3, R=4] -> idx = (1, {"Z": 3, "R": 4}) # P1 or idx = (1, {"Z": 3}, {"R": 4}) # P2 or idx = (1, ("Z", 3), ("R", 4)) # P3 or idx = (1, keyword("Z", 3), keyword("R", 4)) # P4 C5. a[1, 2, Z=3] -> idx = (1, 2, {"Z": 3}) # P1/P2 or idx = (1, 2, ("Z", 3)) # P3 or idx = (1, 2, keyword("Z", 3)) # P4 C6. a[1, 2, Z=3, R=4] -> idx = (1, 2, {"Z":3, "R": 4}) # P1 or idx = (1, 2, {"Z": 3}, {"R": 4}) # P2 or idx = (1, 2, ("Z", 3), ("R", 4)) # P3 or idx = (1, 2, keyword("Z", 3), keyword("R", 4)) # P4 C7. a[1, Z=3, 2, R=4] -> idx = (1, 2, {"Z": 3, "R": 4}) # P1. Pack the keyword arguments. Ugly. or raise SyntaxError # P1. Same behavior as in function calls. or idx = (1, {"Z": 3}, 2, {"R": 4}) # P2 or idx = (1, ("Z", 3), 2, ("R", 4)) # P3 or idx = (1, keyword("Z", 3), 2, keyword("R", 4)) # P4 </pre></div> </div> <section id="id5"> <h4><a class="toc-backref" href="#id5" role="doc-backlink">Pros</a></h4> <ul class="simple"> <li>Signature is unchanged;</li> <li>P2/P3 can preserve ordering of keyword arguments as specified at indexing,</li> <li>P1 needs an OrderedDict, but would destroy interposed ordering if allowed: all keyword indexes would be dumped into the dictionary;</li> <li>Stays within traditional types: tuples and dicts. Evt. OrderedDict;</li> <li>Some proposed strategies are similar in behavior to a traditional function call;</li> <li>The C interface for <code class="docutils literal notranslate">PyObject_GetItem</code> and family would remain unchanged.</li> </ul> </section> <section id="id6"> <h4><a class="toc-backref" href="#id6" role="doc-backlink">Cons</a></h4> <ul class="simple"> <li>Apparently complex and wasteful;</li> <li>Degeneracy in notation (e.g. <code class="docutils literal notranslate">a[Z=3]</code> and <code class="docutils literal notranslate">a[{"Z":3}]</code> are equivalent and indistinguishable notations at the <code class="docutils literal notranslate">__[get|set|del]item__</code> level). This behavior may or may not be acceptable.</li> <li>for P4, an additional object similar in nature to slice() is needed, but only to disambiguate the above degeneracy.</li> <li><code class="docutils literal notranslate">idx</code> type and layout seems to change depending on the whims of the caller;</li> <li>May be complex to parse what is passed, especially in the case of tuple of tuples;</li> <li>P2 Creates a lot of single keys dictionary as members of a tuple. Looks ugly. P3 would be lighter and easier to use than the tuple of dicts, and still preserves order (unlike the regular dict), but would result in clumsy extraction of keywords.</li> </ul> </section> </section> <section id="strategy-kwargs-argument"> <h3><a class="toc-backref" href="#strategy-kwargs-argument" role="doc-backlink">Strategy “kwargs argument”</a></h3> <code class="docutils literal notranslate">__getitem__</code> accepts an optional <code class="docutils literal notranslate">**kwargs</code> argument which should be keyword only. <code class="docutils literal notranslate">idx</code> also becomes optional to support a case where no non-keyword arguments are allowed. The signature would then be either <div class="highlight-default notranslate"><div class="highlight"><pre>__getitem__(self, idx) __getitem__(self, idx, **kwargs) __getitem__(self, **kwargs) </pre></div> </div> Applied to our cases would produce: <div class="highlight-default notranslate"><div class="highlight"><pre>C0. a[1,2] -> idx=(1,2); kwargs={} C1. a[Z=3] -> idx=None ; kwargs={"Z":3} C2. a[Z=3, R=4] -> idx=None ; kwargs={"Z":3, "R":4} C3. a[1, Z=3] -> idx=1 ; kwargs={"Z":3} C4. a[1, Z=3, R=4] -> idx=1 ; kwargs={"Z":3, "R":4} C5. a[1, 2, Z=3] -> idx=(1,2); kwargs={"Z":3} C6. a[1, 2, Z=3, R=4] -> idx=(1,2); kwargs={"Z":3, "R":4} C7. a[1, Z=3, 2, R=4] -> raise SyntaxError # in agreement to function behavior </pre></div> </div> Empty indexing <code class="docutils literal notranslate">a[]</code> of course remains invalid syntax. <section id="id7"> <h4><a class="toc-backref" href="#id7" role="doc-backlink">Pros</a></h4> <ul class="simple"> <li>Similar to function call, evolves naturally from it;</li> <li>Use of keyword indexing with an object whose <code class="docutils literal notranslate">__getitem__</code> doesn’t have a kwargs will fail in an obvious way. That’s not the case for the other strategies.</li> </ul> </section> <section id="id8"> <h4><a class="toc-backref" href="#id8" role="doc-backlink">Cons</a></h4> <ul class="simple"> <li>It doesn’t preserve order, unless an OrderedDict is used;</li> <li>Forbids C7, but is it really needed?</li> <li>Requires a change in the C interface to pass an additional PyObject for the keyword arguments.</li> </ul> </section> </section> </section> <section id="c-interface"> <h2><a class="toc-backref" href="#c-interface" role="doc-backlink">C interface</a></h2> As briefly introduced in the previous analysis, the C interface would potentially have to change to allow the new feature. Specifically, <code class="docutils literal notranslate">PyObject_GetItem</code> and related routines would have to accept an additional <code class="docutils literal notranslate">PyObject *kw</code> argument for Strategy “kwargs argument”. The remaining strategies would not require a change in the C function signatures, but the different nature of the passed object would potentially require adaptation. Strategy “named tuple” would behave correctly without any change: the class returned by the factory method in collections returns a subclass of tuple, meaning that <code class="docutils literal notranslate">PyTuple_*</code> functions can handle the resulting object. </section> <section id="alternative-solutions"> <h2><a class="toc-backref" href="#alternative-solutions" role="doc-backlink">Alternative Solutions</a></h2> In this section, we present alternative solutions that would workaround the missing feature and make the proposed enhancement not worth of implementation. <section id="use-a-method"> <h3><a class="toc-backref" href="#use-a-method" role="doc-backlink">Use a method</a></h3> One could keep the indexing as is, and use a traditional <code class="docutils literal notranslate">get()</code> method for those cases where basic indexing is not enough. This is a good point, but as already reported in the introduction, methods have a different semantic weight from indexing, and you can’t use slices directly in methods. Compare e.g. <code class="docutils literal notranslate">a[1:3, Z=2]</code> with <code class="docutils literal notranslate">a.get(slice(1,3), Z=2)</code>. The authors however recognize this argument as compelling, and the advantage in semantic expressivity of a keyword-based indexing may be offset by a rarely used feature that does not bring enough benefit and may have limited adoption. </section> <section id="emulate-requested-behavior-by-abusing-the-slice-object"> <h3><a class="toc-backref" href="#emulate-requested-behavior-by-abusing-the-slice-object" role="doc-backlink">Emulate requested behavior by abusing the slice object</a></h3> This extremely creative method exploits the slice objects’ behavior, provided that one accepts to use strings (or instantiate properly named placeholder objects for the keys), and accept to use “:” instead of “=”. <div class="highlight-default notranslate"><div class="highlight"><pre>>>> a["K":3] slice('K', 3, None) >>> a["K":3, "R":4] (slice('K', 3, None), slice('R', 4, None)) >>> </pre></div> </div> While clearly smart, this approach does not allow easy inquire of the key/value pair, it’s too clever and esotheric, and does not allow to pass a slice as in <code class="docutils literal notranslate">a[K=1:10:2]</code>. However, Tim Delaney comments <blockquote> <div>“I really do think that <code class="docutils literal notranslate">a[b=c, d=e]</code> should just be syntax sugar for <code class="docutils literal notranslate">a['b':c, 'd':e]</code>. It’s simple to explain, and gives the greatest backwards compatibility. In particular, libraries that already abused slices in this way will just continue to work with the new syntax.”</div></blockquote> We think this behavior would produce inconvenient results. The library Pandas uses strings as labels, allowing notation such as <div class="highlight-default notranslate"><div class="highlight"><pre>>>> a[:, "A":"F"] </pre></div> </div> to extract data from column “A” to column “F”. Under the above comment, this notation would be equally obtained with <div class="highlight-default notranslate"><div class="highlight"><pre>>>> a[:, A="F"] </pre></div> </div> which is weird and collides with the intended meaning of keyword in indexing, that is, specifying the axis through conventional names rather than positioning. </section> <section id="pass-a-dictionary-as-an-additional-index"> <h3><a class="toc-backref" href="#pass-a-dictionary-as-an-additional-index" role="doc-backlink">Pass a dictionary as an additional index</a></h3> <div class="highlight-default notranslate"><div class="highlight"><pre>>>> a[1, 2, {"K": 3}] </pre></div> </div> this notation, although less elegant, can already be used and achieves similar results. It’s evident that the proposed Strategy “New argument contents” can be interpreted as syntactic sugar for this notation. </section> </section> <section id="additional-comments"> <h2><a class="toc-backref" href="#additional-comments" role="doc-backlink">Additional Comments</a></h2> Commenters also expressed the following relevant points: <section id="relevance-of-ordering-of-keyword-arguments"> <h3><a class="toc-backref" href="#relevance-of-ordering-of-keyword-arguments" role="doc-backlink">Relevance of ordering of keyword arguments</a></h3> As part of the discussion of this PEP, it’s important to decide if the ordering information of the keyword arguments is important, and if indexes and keys can be ordered in an arbitrary way (e.g. <code class="docutils literal notranslate">a[1,Z=3,2,R=4]</code>). <a class="pep reference internal" href="../pep-0468/" title="PEP 468 – Preserving the order of **kwargs in a function.">PEP 468</a> tries to address the first point by proposing the use of an ordereddict, however one would be inclined to accept that keyword arguments in indexing are equivalent to kwargs in function calls, and therefore as of today equally unordered, and with the same restrictions. </section> <section id="need-for-homogeneity-of-behavior"> <h3><a class="toc-backref" href="#need-for-homogeneity-of-behavior" role="doc-backlink">Need for homogeneity of behavior</a></h3> Relative to Strategy “New argument contents”, a comment from Ian Cordasco points out that <blockquote> <div>“it would be unreasonable for just one method to behave totally differently from the standard behaviour in Python. It would be confusing for only <code class="docutils literal notranslate">__getitem__</code> (and ostensibly, <code class="docutils literal notranslate">__setitem__</code>) to take keyword arguments but instead of turning them into a dictionary, turn them into individual single-item dictionaries.” We agree with his point, however it must be pointed out that <code class="docutils literal notranslate">__getitem__</code> is already special in some regards when it comes to passed arguments.</div></blockquote> Chris Angelico also states: <blockquote> <div>“it seems very odd to start out by saying “here, let’s give indexing the option to carry keyword args, just like with function calls”, and then come back and say “oh, but unlike function calls, they’re inherently ordered and carried very differently”.” Again, we agree on this point. The most straightforward strategy to keep homogeneity would be Strategy “kwargs argument”, opening to a <code class="docutils literal notranslate">**kwargs</code> argument on <code class="docutils literal notranslate">__getitem__</code>.</div></blockquote> One of the authors (Stefano Borini) thinks that only the “strict dictionary” strategy is worth of implementation. It is non-ambiguous, simple, does not force complex parsing, and addresses the problem of referring to axes either by position or by name. The “options” use case is probably best handled with a different approach, and may be irrelevant for this PEP. The alternative “named tuple” is another valid choice. </section> <section id="having-get-become-obsolete-for-indexing-with-default-fallback"> <h3><a class="toc-backref" href="#having-get-become-obsolete-for-indexing-with-default-fallback" role="doc-backlink">Having .get() become obsolete for indexing with default fallback</a></h3> Introducing a “default” keyword could make <code class="docutils literal notranslate">dict.get()</code> obsolete, which would be replaced by <code class="docutils literal notranslate">d["key", default=3]</code>. Chris Angelico however states: <blockquote> <div>“Currently, you need to write <code class="docutils literal notranslate">__getitem__</code> (which raises an exception on finding a problem) plus something else, e.g. <code class="docutils literal notranslate">get()</code>, which returns a default instead. By your proposal, both branches would go inside <code class="docutils literal notranslate">__getitem__</code>, which means they could share code; but there still need to be two branches.”</div></blockquote> Additionally, Chris continues: <blockquote> <div>“There’ll be an ad-hoc and fairly arbitrary puddle of names (some will go <code class="docutils literal notranslate">default=</code>, others will say that’s way too long and go <code class="docutils literal notranslate">def=</code>, except that that’s a keyword so they’ll use <code class="docutils literal notranslate">dflt=</code> or something…), unless there’s a strong force pushing people to one consistent name.”.</div></blockquote> This argument is valid but it’s equally valid for any function call, and is generally fixed by established convention and documentation. </section> <section id="on-degeneracy-of-notation"> <h3><a class="toc-backref" href="#on-degeneracy-of-notation" role="doc-backlink">On degeneracy of notation</a></h3> User Drekin commented: “The case of <code class="docutils literal notranslate">a[Z=3]</code> and <code class="docutils literal notranslate">a[{"Z": 3}]</code> is similar to current <code class="docutils literal notranslate">a[1, 2]</code> and <code class="docutils literal notranslate">a[(1, 2)]</code>. Even though one may argue that the parentheses are actually not part of tuple notation but are just needed because of syntax, it may look as degeneracy of notation when compared to function call: <code class="docutils literal notranslate">f(1, 2)</code> is not the same thing as <code class="docutils literal notranslate">f((1, 2))</code>.”. </section> </section> <section id="references"> <h2><a class="toc-backref" href="#references" role="doc-backlink">References</a></h2> <ul class="simple"> <li><a class="reference external" href="https://mail.python.org/archives/list/python-ideas@python.org/thread/MNBVLLYNQU3OVJR3JSEEQUDXMM3X5Z3J/#MNBVLLYNQU3OVJR3JSEEQUDXMM3X5Z3J">“keyword-only args in __getitem__”</a></li> <li><a class="reference external" href="https://mail.python.org/archives/list/python-ideas@python.org/thread/GDSUYS7BT6YEXHM73HMJAMAENNCK7RDP/#N2JVLQD6REAV2KGBKASNTOTHJJSKOPTH">“Accepting keyword arguments for __getitem__”</a></li> <li><a class="reference external" href="https://mail.python.org/pipermail/python-ideas/2014-July/028250.html">“PEP pre-draft: Support for indexing with keyword arguments”</a></li> </ul> </section> <section id="copyright"> <h2><a class="toc-backref" href="#copyright" role="doc-backlink">Copyright</a></h2> This document has been placed in the public domain. </section> </section> <hr class="docutils" /> Source: <a class="reference external" href="https://github.com/python/peps/blob/main/peps/pep-0472.rst">https://github.com/python/peps/blob/main/peps/pep-0472.rst</a> Last modified: <a class="reference external" href="https://github.com/python/peps/commits/main/peps/pep-0472.rst">2024-12-19 20:04:05 GMT</a> </article> <nav id="pep-sidebar"> <h2>Contents</h2> <ul> <li><a class="reference internal" href="#abstract">Abstract</a></li> <li><a class="reference internal" href="#motivation">Motivation</a></li> <li><a class="reference internal" href="#use-cases">Use cases</a></li> <li><a class="reference internal" href="#current-implementation">Current implementation</a></li> <li><a class="reference internal" href="#specifications">Specifications</a><ul> <li><a class="reference internal" href="#strategy-strict-dictionary">Strategy “Strict dictionary”</a><ul> <li><a class="reference internal" href="#pros">Pros</a></li> <li><a class="reference internal" href="#neutral">Neutral</a></li> <li><a class="reference internal" href="#cons">Cons</a></li> </ul> </li> <li><a class="reference internal" href="#strategy-mixed-dictionary">Strategy “mixed dictionary”</a><ul> <li><a class="reference internal" href="#id1">Pros</a></li> <li><a class="reference internal" href="#id2">Cons</a></li> </ul> </li> <li><a class="reference internal" href="#strategy-named-tuple">Strategy “named tuple”</a><ul> <li><a class="reference internal" href="#id3">Pros</a></li> <li><a class="reference internal" href="#id4">Cons</a></li> </ul> </li> <li><a class="reference internal" href="#strategy-new-argument-contents">Strategy “New argument contents”</a><ul> <li><a class="reference internal" href="#id5">Pros</a></li> <li><a class="reference internal" href="#id6">Cons</a></li> </ul> </li> <li><a class="reference internal" href="#strategy-kwargs-argument">Strategy “kwargs argument”</a><ul> <li><a class="reference internal" href="#id7">Pros</a></li> <li><a class="reference internal" href="#id8">Cons</a></li> </ul> </li> </ul> </li> <li><a class="reference internal" href="#c-interface">C interface</a></li> <li><a class="reference internal" href="#alternative-solutions">Alternative Solutions</a><ul> <li><a class="reference internal" href="#use-a-method">Use a method</a></li> <li><a class="reference internal" href="#emulate-requested-behavior-by-abusing-the-slice-object">Emulate requested behavior by abusing the slice object</a></li> <li><a class="reference internal" href="#pass-a-dictionary-as-an-additional-index">Pass a dictionary as an additional index</a></li> </ul> </li> <li><a class="reference internal" href="#additional-comments">Additional Comments</a><ul> <li><a class="reference internal" href="#relevance-of-ordering-of-keyword-arguments">Relevance of ordering of keyword arguments</a></li> <li><a class="reference internal" href="#need-for-homogeneity-of-behavior">Need for homogeneity of behavior</a></li> <li><a class="reference internal" href="#having-get-become-obsolete-for-indexing-with-default-fallback">Having .get() become obsolete for indexing with default fallback</a></li> <li><a class="reference internal" href="#on-degeneracy-of-notation">On degeneracy of notation</a></li> </ul> </li> <li><a class="reference internal" href="#references">References</a></li> <li><a class="reference internal" href="#copyright">Copyright</a></li> </ul> <a id="source" href="https://github.com/python/peps/blob/main/peps/pep-0472.rst">Page Source (GitHub)</a> </nav> </section> <script src="../_static/colour_scheme.js"></script> <script src="../_static/wrap_tables.js"></script> <script src="../_static/sticky_banner.js"></script> </body> </html>