CINXE.COM
PEP 618 – Add Optional Length-Checking To zip | peps.python.org
<!DOCTYPE html> <html lang="en"> <head> <meta charset="utf-8"> <meta name="viewport" content="width=device-width, initial-scale=1.0"> <meta name="color-scheme" content="light dark"> <title>PEP 618 – Add Optional Length-Checking To zip | peps.python.org</title> <link rel="shortcut icon" href="../_static/py.png"> <link rel="canonical" href="https://peps.python.org/pep-0618/"> <link rel="stylesheet" href="../_static/style.css" type="text/css"> <link rel="stylesheet" href="../_static/mq.css" type="text/css"> <link rel="stylesheet" href="../_static/pygments.css" type="text/css" media="(prefers-color-scheme: light)" id="pyg-light"> <link rel="stylesheet" href="../_static/pygments_dark.css" type="text/css" media="(prefers-color-scheme: dark)" id="pyg-dark"> <link rel="alternate" type="application/rss+xml" title="Latest PEPs" href="https://peps.python.org/peps.rss"> <meta property="og:title" content='PEP 618 – Add Optional Length-Checking To zip | peps.python.org'> <meta property="og:description" content="This PEP proposes adding an optional strict boolean keyword parameter to the built-in zip. When enabled, a ValueError is raised if one of the arguments is exhausted before the others."> <meta property="og:type" content="website"> <meta property="og:url" content="https://peps.python.org/pep-0618/"> <meta property="og:site_name" content="Python Enhancement Proposals (PEPs)"> <meta property="og:image" content="https://peps.python.org/_static/og-image.png"> <meta property="og:image:alt" content="Python PEPs"> <meta property="og:image:width" content="200"> <meta property="og:image:height" content="200"> <meta name="description" content="This PEP proposes adding an optional strict boolean keyword parameter to the built-in zip. When enabled, a ValueError is raised if one of the arguments is exhausted before the others."> <meta name="theme-color" content="#3776ab"> </head> <body> <svg xmlns="http://www.w3.org/2000/svg" style="display: none;"> <symbol id="svg-sun-half" viewBox="0 0 24 24" pointer-events="all"> <title>Following system colour scheme</title> <svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"> <circle cx="12" cy="12" r="9"></circle> <path d="M12 3v18m0-12l4.65-4.65M12 14.3l7.37-7.37M12 19.6l8.85-8.85"></path> </svg> </symbol> <symbol id="svg-moon" viewBox="0 0 24 24" pointer-events="all"> <title>Selected dark colour scheme</title> <svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"> <path stroke="none" d="M0 0h24v24H0z" fill="none"></path> <path d="M12 3c.132 0 .263 0 .393 0a7.5 7.5 0 0 0 7.92 12.446a9 9 0 1 1 -8.313 -12.454z"></path> </svg> </symbol> <symbol id="svg-sun" viewBox="0 0 24 24" pointer-events="all"> <title>Selected light colour scheme</title> <svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"> <circle cx="12" cy="12" r="5"></circle> <line x1="12" y1="1" x2="12" y2="3"></line> <line x1="12" y1="21" x2="12" y2="23"></line> <line x1="4.22" y1="4.22" x2="5.64" y2="5.64"></line> <line x1="18.36" y1="18.36" x2="19.78" y2="19.78"></line> <line x1="1" y1="12" x2="3" y2="12"></line> <line x1="21" y1="12" x2="23" y2="12"></line> <line x1="4.22" y1="19.78" x2="5.64" y2="18.36"></line> <line x1="18.36" y1="5.64" x2="19.78" y2="4.22"></line> </svg> </symbol> </svg> <script> document.documentElement.dataset.colour_scheme = localStorage.getItem("colour_scheme") || "auto" </script> <section id="pep-page-section"> <header> <h1>Python Enhancement Proposals</h1> <ul class="breadcrumbs"> <li><a href="https://www.python.org/" title="The Python Programming Language">Python</a> » </li> <li><a href="../pep-0000/">PEP Index</a> » </li> <li>PEP 618</li> </ul> <button id="colour-scheme-cycler" onClick="setColourScheme(nextColourScheme())"> <svg aria-hidden="true" class="colour-scheme-icon-when-auto"><use href="#svg-sun-half"></use></svg> <svg aria-hidden="true" class="colour-scheme-icon-when-dark"><use href="#svg-moon"></use></svg> <svg aria-hidden="true" class="colour-scheme-icon-when-light"><use href="#svg-sun"></use></svg> <span class="visually-hidden">Toggle light / dark / auto colour theme</span> </button> </header> <article> <section id="pep-content"> <h1 class="page-title">PEP 618 – Add Optional Length-Checking To zip</h1> <dl class="rfc2822 field-list simple"> <dt class="field-odd">Author<span class="colon">:</span></dt> <dd class="field-odd">Brandt Bucher <brandt at python.org></dd> <dt class="field-even">Sponsor<span class="colon">:</span></dt> <dd class="field-even">Antoine Pitrou <antoine at python.org></dd> <dt class="field-odd">BDFL-Delegate<span class="colon">:</span></dt> <dd class="field-odd">Guido van Rossum <guido at python.org></dd> <dt class="field-even">Status<span class="colon">:</span></dt> <dd class="field-even"><abbr title="Accepted and implementation complete, or no longer active">Final</abbr></dd> <dt class="field-odd">Type<span class="colon">:</span></dt> <dd class="field-odd"><abbr title="Normative PEP with a new feature for Python, implementation change for CPython or interoperability standard for the ecosystem">Standards Track</abbr></dd> <dt class="field-even">Created<span class="colon">:</span></dt> <dd class="field-even">01-May-2020</dd> <dt class="field-odd">Python-Version<span class="colon">:</span></dt> <dd class="field-odd">3.10</dd> <dt class="field-even">Post-History<span class="colon">:</span></dt> <dd class="field-even">01-May-2020, 10-May-2020, 16-Jun-2020</dd> <dt class="field-odd">Resolution<span class="colon">:</span></dt> <dd class="field-odd"><a class="reference external" href="https://mail.python.org/archives/list/python-dev@python.org/message/NLWB7FVJGMBBMCF4P3ZKUIE53JPDOWJ3">Python-Dev message</a></dd> </dl> <hr class="docutils" /> <section id="contents"> <details><summary>Table of Contents</summary><ul class="simple"> <li><a class="reference internal" href="#abstract">Abstract</a></li> <li><a class="reference internal" href="#motivation">Motivation</a></li> <li><a class="reference internal" href="#rationale">Rationale</a></li> <li><a class="reference internal" href="#specification">Specification</a></li> <li><a class="reference internal" href="#backward-compatibility">Backward Compatibility</a></li> <li><a class="reference internal" href="#reference-implementation">Reference Implementation</a></li> <li><a class="reference internal" href="#rejected-ideas">Rejected Ideas</a><ul> <li><a class="reference internal" href="#add-itertools-zip-strict">Add <code class="docutils literal notranslate"><span class="pre">itertools.zip_strict</span></code></a><ul> <li><a class="reference internal" href="#precedent">Precedent</a></li> <li><a class="reference internal" href="#usability">Usability</a></li> <li><a class="reference internal" href="#maintenance-cost">Maintenance Cost</a></li> </ul> </li> <li><a class="reference internal" href="#add-several-modes-to-switch-between">Add Several “Modes” To Switch Between</a></li> <li><a class="reference internal" href="#add-a-method-or-alternate-constructor-to-the-zip-type">Add A Method Or Alternate Constructor To The <code class="docutils literal notranslate"><span class="pre">zip</span></code> Type</a></li> <li><a class="reference internal" href="#change-the-default-behavior-of-zip">Change The Default Behavior Of <code class="docutils literal notranslate"><span class="pre">zip</span></code></a></li> <li><a class="reference internal" href="#accept-a-callback-to-handle-remaining-items">Accept A Callback To Handle Remaining Items</a></li> <li><a class="reference internal" href="#raise-an-assertionerror">Raise An <code class="docutils literal notranslate"><span class="pre">AssertionError</span></code></a></li> <li><a class="reference internal" href="#add-a-similar-feature-to-map">Add A Similar Feature to <code class="docutils literal notranslate"><span class="pre">map</span></code></a></li> <li><a class="reference internal" href="#do-nothing">Do Nothing</a></li> </ul> </li> <li><a class="reference internal" href="#references">References</a><ul> <li><a class="reference internal" href="#examples">Examples</a></li> </ul> </li> <li><a class="reference internal" href="#copyright">Copyright</a></li> </ul> </details></section> <section id="abstract"> <h2><a class="toc-backref" href="#abstract" role="doc-backlink">Abstract</a></h2> <p>This PEP proposes adding an optional <code class="docutils literal notranslate"><span class="pre">strict</span></code> boolean keyword parameter to the built-in <code class="docutils literal notranslate"><span class="pre">zip</span></code>. When enabled, a <code class="docutils literal notranslate"><span class="pre">ValueError</span></code> is raised if one of the arguments is exhausted before the others.</p> </section> <section id="motivation"> <h2><a class="toc-backref" href="#motivation" role="doc-backlink">Motivation</a></h2> <p>It is clear from the author’s personal experience and a <a class="reference internal" href="#examples">survey of the standard library</a> that much (if not most) <code class="docutils literal notranslate"><span class="pre">zip</span></code> usage involves iterables that <em>must</em> be of equal length. Sometimes this invariant is proven true from the context of the surrounding code, but often the data being zipped is passed from the caller, sourced separately, or generated in some fashion. In any of these cases, the default behavior of <code class="docutils literal notranslate"><span class="pre">zip</span></code> means that faulty refactoring or logic errors could easily result in silently losing data. These bugs are not only difficult to diagnose, but difficult to even detect at all.</p> <p>It is easy to come up with simple cases where this could be a problem. For example, the following code may work fine when <code class="docutils literal notranslate"><span class="pre">items</span></code> is a sequence, but silently start producing shortened, mismatched results if <code class="docutils literal notranslate"><span class="pre">items</span></code> is refactored by the caller to be a consumable iterator:</p> <div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="k">def</span><span class="w"> </span><span class="nf">apply_calculations</span><span class="p">(</span><span class="n">items</span><span class="p">):</span> <span class="n">transformed</span> <span class="o">=</span> <span class="n">transform</span><span class="p">(</span><span class="n">items</span><span class="p">)</span> <span class="k">for</span> <span class="n">i</span><span class="p">,</span> <span class="n">t</span> <span class="ow">in</span> <span class="nb">zip</span><span class="p">(</span><span class="n">items</span><span class="p">,</span> <span class="n">transformed</span><span class="p">):</span> <span class="k">yield</span> <span class="n">calculate</span><span class="p">(</span><span class="n">i</span><span class="p">,</span> <span class="n">t</span><span class="p">)</span> </pre></div> </div> <p>There are several other ways in which <code class="docutils literal notranslate"><span class="pre">zip</span></code> is commonly used. Idiomatic tricks are especially susceptible, because they are often employed by users who lack a complete understanding of how the code works. One example is unpacking into <code class="docutils literal notranslate"><span class="pre">zip</span></code> to lazily “unzip” or “transpose” nested iterables:</p> <div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="gp">>>> </span><span class="n">x</span> <span class="o">=</span> <span class="p">[[</span><span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="mi">3</span><span class="p">],</span> <span class="p">[</span><span class="s2">"one"</span> <span class="s2">"two"</span> <span class="s2">"three"</span><span class="p">]]</span> <span class="gp">>>> </span><span class="n">xt</span> <span class="o">=</span> <span class="nb">list</span><span class="p">(</span><span class="nb">zip</span><span class="p">(</span><span class="o">*</span><span class="n">x</span><span class="p">))</span> </pre></div> </div> <p>Another is “chunking” data into equal-sized groups:</p> <div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="gp">>>> </span><span class="n">n</span> <span class="o">=</span> <span class="mi">3</span> <span class="gp">>>> </span><span class="n">x</span> <span class="o">=</span> <span class="nb">range</span><span class="p">(</span><span class="n">n</span> <span class="o">**</span> <span class="mi">2</span><span class="p">),</span> <span class="gp">>>> </span><span class="n">xn</span> <span class="o">=</span> <span class="nb">list</span><span class="p">(</span><span class="nb">zip</span><span class="p">(</span><span class="o">*</span><span class="p">[</span><span class="nb">iter</span><span class="p">(</span><span class="n">x</span><span class="p">)]</span> <span class="o">*</span> <span class="n">n</span><span class="p">))</span> </pre></div> </div> <p>In the first case, non-rectangular data is usually a logic error. In the second case, data with a length that is not a multiple of <code class="docutils literal notranslate"><span class="pre">n</span></code> is often an error as well. However, both of these idioms will silently omit the tail-end items of malformed input.</p> <p>Perhaps most convincingly, the use of <code class="docutils literal notranslate"><span class="pre">zip</span></code> in the standard-library <code class="docutils literal notranslate"><span class="pre">ast</span></code> module created a bug in <code class="docutils literal notranslate"><span class="pre">literal_eval</span></code> which <a class="reference external" href="https://bugs.python.org/issue40355">silently dropped parts of malformed nodes</a>:</p> <div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="gp">>>> </span><span class="kn">from</span><span class="w"> </span><span class="nn">ast</span><span class="w"> </span><span class="kn">import</span> <span class="n">Constant</span><span class="p">,</span> <span class="n">Dict</span><span class="p">,</span> <span class="n">literal_eval</span> <span class="gp">>>> </span><span class="n">nasty_dict</span> <span class="o">=</span> <span class="n">Dict</span><span class="p">(</span><span class="n">keys</span><span class="o">=</span><span class="p">[</span><span class="n">Constant</span><span class="p">(</span><span class="kc">None</span><span class="p">)],</span> <span class="n">values</span><span class="o">=</span><span class="p">[])</span> <span class="gp">>>> </span><span class="n">literal_eval</span><span class="p">(</span><span class="n">nasty_dict</span><span class="p">)</span> <span class="c1"># Like eval("{None: }")</span> <span class="go">{}</span> </pre></div> </div> <p>In fact, the author has <a class="reference internal" href="#examples">counted dozens of other call sites</a> in Python’s standard library and tooling where it would be appropriate to enable this new feature immediately.</p> </section> <section id="rationale"> <h2><a class="toc-backref" href="#rationale" role="doc-backlink">Rationale</a></h2> <p>Some critics assert that constant boolean switches are a “code-smell”, or go against Python’s design philosophy. However, Python currently contains several examples of boolean keyword parameters on built-in functions which are typically called with compile-time constants:</p> <ul class="simple"> <li><code class="docutils literal notranslate"><span class="pre">compile(...,</span> <span class="pre">dont_inherit=True)</span></code></li> <li><code class="docutils literal notranslate"><span class="pre">open(...,</span> <span class="pre">closefd=False)</span></code></li> <li><code class="docutils literal notranslate"><span class="pre">print(...,</span> <span class="pre">flush=True)</span></code></li> <li><code class="docutils literal notranslate"><span class="pre">sorted(...,</span> <span class="pre">reverse=True)</span></code></li> </ul> <p>Many more exist in the standard library.</p> <p>The idea and name for this new parameter were <a class="reference external" href="https://mail.python.org/archives/list/python-ideas@python.org/message/6GFUADSQ5JTF7W7OGWF7XF2NH2XUTUQM">originally proposed</a> by Ram Rachum. The thread received over 100 replies, with the alternative “equal” receiving a similar amount of support.</p> <p>The author does not have a strong preference between the two choices, though “equal equals” <em>is</em> a bit awkward in prose. It may also (wrongly) imply some notion of “equality” between the zipped items:</p> <div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="gp">>>> </span><span class="n">z</span> <span class="o">=</span> <span class="nb">zip</span><span class="p">([</span><span class="mf">2.0</span><span class="p">,</span> <span class="mf">4.0</span><span class="p">,</span> <span class="mf">6.0</span><span class="p">],</span> <span class="p">[</span><span class="mi">2</span><span class="p">,</span> <span class="mi">4</span><span class="p">,</span> <span class="mi">8</span><span class="p">],</span> <span class="n">equal</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span> </pre></div> </div> </section> <section id="specification"> <h2><a class="toc-backref" href="#specification" role="doc-backlink">Specification</a></h2> <p>When the built-in <code class="docutils literal notranslate"><span class="pre">zip</span></code> is called with the keyword-only argument <code class="docutils literal notranslate"><span class="pre">strict=True</span></code>, the resulting iterator will raise a <code class="docutils literal notranslate"><span class="pre">ValueError</span></code> if the arguments are exhausted at differing lengths. This error will occur at the point when iteration would normally stop today.</p> </section> <section id="backward-compatibility"> <h2><a class="toc-backref" href="#backward-compatibility" role="doc-backlink">Backward Compatibility</a></h2> <p>This change is fully backward-compatible. <code class="docutils literal notranslate"><span class="pre">zip</span></code> currently takes no keyword arguments, and the “non-strict” default behavior when <code class="docutils literal notranslate"><span class="pre">strict</span></code> is omitted remains unchanged.</p> </section> <section id="reference-implementation"> <h2><a class="toc-backref" href="#reference-implementation" role="doc-backlink">Reference Implementation</a></h2> <p>The author has drafted a <a class="reference external" href="https://github.com/python/cpython/pull/20921">C implementation</a>.</p> <p>An approximate Python translation is:</p> <div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="k">def</span><span class="w"> </span><span class="nf">zip</span><span class="p">(</span><span class="o">*</span><span class="n">iterables</span><span class="p">,</span> <span class="n">strict</span><span class="o">=</span><span class="kc">False</span><span class="p">):</span> <span class="k">if</span> <span class="ow">not</span> <span class="n">iterables</span><span class="p">:</span> <span class="k">return</span> <span class="n">iterators</span> <span class="o">=</span> <span class="nb">tuple</span><span class="p">(</span><span class="nb">iter</span><span class="p">(</span><span class="n">iterable</span><span class="p">)</span> <span class="k">for</span> <span class="n">iterable</span> <span class="ow">in</span> <span class="n">iterables</span><span class="p">)</span> <span class="k">try</span><span class="p">:</span> <span class="k">while</span> <span class="kc">True</span><span class="p">:</span> <span class="n">items</span> <span class="o">=</span> <span class="p">[]</span> <span class="k">for</span> <span class="n">iterator</span> <span class="ow">in</span> <span class="n">iterators</span><span class="p">:</span> <span class="n">items</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="nb">next</span><span class="p">(</span><span class="n">iterator</span><span class="p">))</span> <span class="k">yield</span> <span class="nb">tuple</span><span class="p">(</span><span class="n">items</span><span class="p">)</span> <span class="k">except</span> <span class="ne">StopIteration</span><span class="p">:</span> <span class="k">if</span> <span class="ow">not</span> <span class="n">strict</span><span class="p">:</span> <span class="k">return</span> <span class="k">if</span> <span class="n">items</span><span class="p">:</span> <span class="n">i</span> <span class="o">=</span> <span class="nb">len</span><span class="p">(</span><span class="n">items</span><span class="p">)</span> <span class="n">plural</span> <span class="o">=</span> <span class="s2">" "</span> <span class="k">if</span> <span class="n">i</span> <span class="o">==</span> <span class="mi">1</span> <span class="k">else</span> <span class="s2">"s 1-"</span> <span class="n">msg</span> <span class="o">=</span> <span class="sa">f</span><span class="s2">"zip() argument </span><span class="si">{</span><span class="n">i</span><span class="o">+</span><span class="mi">1</span><span class="si">}</span><span class="s2"> is shorter than argument</span><span class="si">{</span><span class="n">plural</span><span class="si">}{</span><span class="n">i</span><span class="si">}</span><span class="s2">"</span> <span class="k">raise</span> <span class="ne">ValueError</span><span class="p">(</span><span class="n">msg</span><span class="p">)</span> <span class="n">sentinel</span> <span class="o">=</span> <span class="nb">object</span><span class="p">()</span> <span class="k">for</span> <span class="n">i</span><span class="p">,</span> <span class="n">iterator</span> <span class="ow">in</span> <span class="nb">enumerate</span><span class="p">(</span><span class="n">iterators</span><span class="p">[</span><span class="mi">1</span><span class="p">:],</span> <span class="mi">1</span><span class="p">):</span> <span class="k">if</span> <span class="nb">next</span><span class="p">(</span><span class="n">iterator</span><span class="p">,</span> <span class="n">sentinel</span><span class="p">)</span> <span class="ow">is</span> <span class="ow">not</span> <span class="n">sentinel</span><span class="p">:</span> <span class="n">plural</span> <span class="o">=</span> <span class="s2">" "</span> <span class="k">if</span> <span class="n">i</span> <span class="o">==</span> <span class="mi">1</span> <span class="k">else</span> <span class="s2">"s 1-"</span> <span class="n">msg</span> <span class="o">=</span> <span class="sa">f</span><span class="s2">"zip() argument </span><span class="si">{</span><span class="n">i</span><span class="o">+</span><span class="mi">1</span><span class="si">}</span><span class="s2"> is longer than argument</span><span class="si">{</span><span class="n">plural</span><span class="si">}{</span><span class="n">i</span><span class="si">}</span><span class="s2">"</span> <span class="k">raise</span> <span class="ne">ValueError</span><span class="p">(</span><span class="n">msg</span><span class="p">)</span> </pre></div> </div> </section> <section id="rejected-ideas"> <h2><a class="toc-backref" href="#rejected-ideas" role="doc-backlink">Rejected Ideas</a></h2> <section id="add-itertools-zip-strict"> <h3><a class="toc-backref" href="#add-itertools-zip-strict" role="doc-backlink">Add <code class="docutils literal notranslate"><span class="pre">itertools.zip_strict</span></code></a></h3> <p>This is the alternative with the most support on the Python-Ideas mailing list, so it deserves to be discussed in some detail here. It does not have any disqualifying flaws, and could work well enough as a substitute if this PEP is rejected.</p> <p>With that in mind, this section aims to outline why adding an optional parameter to <code class="docutils literal notranslate"><span class="pre">zip</span></code> is a smaller change that ultimately does a better job of solving the problems motivating this PEP.</p> <section id="precedent"> <h4><a class="toc-backref" href="#precedent" role="doc-backlink">Precedent</a></h4> <p>It seems that a great deal of the motivation driving this alternative is that <code class="docutils literal notranslate"><span class="pre">zip_longest</span></code> already exists in <code class="docutils literal notranslate"><span class="pre">itertools</span></code>. However, <code class="docutils literal notranslate"><span class="pre">zip_longest</span></code> is in many ways a much more complicated, specialized utility: it takes on the responsibility of filling in missing values, a job neither of the other variants needs to concern themselves with.</p> <p>If both <code class="docutils literal notranslate"><span class="pre">zip</span></code> and <code class="docutils literal notranslate"><span class="pre">zip_longest</span></code> lived alongside each other in <code class="docutils literal notranslate"><span class="pre">itertools</span></code> or as builtins, then adding <code class="docutils literal notranslate"><span class="pre">zip_strict</span></code> in the same location would indeed be a much stronger argument. However, the new “strict” variant is conceptually <em>much</em> closer to <code class="docutils literal notranslate"><span class="pre">zip</span></code> in interface and behavior than <code class="docutils literal notranslate"><span class="pre">zip_longest</span></code>, while still not meeting the high bar of being its own builtin. Given this situation, it seems most natural for <code class="docutils literal notranslate"><span class="pre">zip</span></code> to grow this new option in-place.</p> </section> <section id="usability"> <h4><a class="toc-backref" href="#usability" role="doc-backlink">Usability</a></h4> <p>If <code class="docutils literal notranslate"><span class="pre">zip</span></code> is capable of preventing this class of bug, it becomes much simpler for users to enable the check at call sites with this property. Compare this with importing a drop-in replacement for a built-in utility, which feels somewhat heavy just to check a tricky condition that should “always” be true.</p> <p>Some have also argued that a new function buried in the standard library is somehow more “discoverable” than a keyword parameter on the built-in itself. The author does not agree with this assessment.</p> </section> <section id="maintenance-cost"> <h4><a class="toc-backref" href="#maintenance-cost" role="doc-backlink">Maintenance Cost</a></h4> <p>While implementation should only be a secondary concern when making usability improvements, it is important to recognize that adding a new utility is significantly more complicated than modifying an existing one. The CPython implementation accompanying this PEP is simple and has no measurable performance impact on default <code class="docutils literal notranslate"><span class="pre">zip</span></code> behavior, while adding an entirely new utility to <code class="docutils literal notranslate"><span class="pre">itertools</span></code> would require either:</p> <ul class="simple"> <li>Duplicating much of the existing <code class="docutils literal notranslate"><span class="pre">zip</span></code> logic, as <code class="docutils literal notranslate"><span class="pre">zip_longest</span></code> already does.</li> <li>Significantly refactoring either <code class="docutils literal notranslate"><span class="pre">zip</span></code>, <code class="docutils literal notranslate"><span class="pre">zip_longest</span></code>, or both to share a common or inherited implementation (which may impact performance).</li> </ul> </section> </section> <section id="add-several-modes-to-switch-between"> <h3><a class="toc-backref" href="#add-several-modes-to-switch-between" role="doc-backlink">Add Several “Modes” To Switch Between</a></h3> <p>This option only makes more sense than a binary flag if we anticipate having three or more modes. The “obvious” three choices for these enumerated or constant modes would be “shortest” (the current <code class="docutils literal notranslate"><span class="pre">zip</span></code> behavior), “strict” (the proposed behavior), and “longest” (the <code class="docutils literal notranslate"><span class="pre">itertools.zip_longest</span></code> behavior).</p> <p>However, it doesn’t seem like adding behaviors other than the current default and the proposed “strict” mode is worth the additional complexity. The clearest candidate, “longest”, would require a new <code class="docutils literal notranslate"><span class="pre">fillvalue</span></code> parameter (which is meaningless for both other modes). This mode is also already handled perfectly by <code class="docutils literal notranslate"><span class="pre">itertools.zip_longest</span></code>, and adding it would create two ways of doing the same thing. It’s not clear which would be the “obvious” choice: the <code class="docutils literal notranslate"><span class="pre">mode</span></code> parameter on the built-in <code class="docutils literal notranslate"><span class="pre">zip</span></code>, or the long-lived namesake utility in <code class="docutils literal notranslate"><span class="pre">itertools</span></code>.</p> </section> <section id="add-a-method-or-alternate-constructor-to-the-zip-type"> <h3><a class="toc-backref" href="#add-a-method-or-alternate-constructor-to-the-zip-type" role="doc-backlink">Add A Method Or Alternate Constructor To The <code class="docutils literal notranslate"><span class="pre">zip</span></code> Type</a></h3> <p>Consider the following two options, which have both been proposed:</p> <div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="gp">>>> </span><span class="n">zm</span> <span class="o">=</span> <span class="nb">zip</span><span class="p">(</span><span class="o">*</span><span class="n">iters</span><span class="p">)</span><span class="o">.</span><span class="n">strict</span><span class="p">()</span> <span class="gp">>>> </span><span class="n">zd</span> <span class="o">=</span> <span class="nb">zip</span><span class="o">.</span><span class="n">strict</span><span class="p">(</span><span class="o">*</span><span class="n">iters</span><span class="p">)</span> </pre></div> </div> <p>It’s not obvious which one will succeed, or how the other will fail. If <code class="docutils literal notranslate"><span class="pre">zip.strict</span></code> is implemented as a method, <code class="docutils literal notranslate"><span class="pre">zm</span></code> will succeed, but <code class="docutils literal notranslate"><span class="pre">zd</span></code> will fail in one of several confusing ways:</p> <ul class="simple"> <li>Yield results that aren’t wrapped in a tuple (if <code class="docutils literal notranslate"><span class="pre">iters</span></code> contains just one item, a <code class="docutils literal notranslate"><span class="pre">zip</span></code> iterator).</li> <li>Raise a <code class="docutils literal notranslate"><span class="pre">TypeError</span></code> for an incorrect argument type (if <code class="docutils literal notranslate"><span class="pre">iters</span></code> contains just one item, not a <code class="docutils literal notranslate"><span class="pre">zip</span></code> iterator).</li> <li>Raise a <code class="docutils literal notranslate"><span class="pre">TypeError</span></code> for an incorrect number of arguments (otherwise).</li> </ul> <p>If <code class="docutils literal notranslate"><span class="pre">zip.strict</span></code> is implemented as a <code class="docutils literal notranslate"><span class="pre">classmethod</span></code> or <code class="docutils literal notranslate"><span class="pre">staticmethod</span></code>, <code class="docutils literal notranslate"><span class="pre">zd</span></code> will succeed, and <code class="docutils literal notranslate"><span class="pre">zm</span></code> will silently yield nothing (which is the problem we are trying to avoid in the first place).</p> <p>This proposal is further complicated by the fact that CPython’s actual <code class="docutils literal notranslate"><span class="pre">zip</span></code> type is currently an undocumented implementation detail. This means that choosing one of the above behaviors will effectively “lock in” the current implementation (or at least require it to be emulated) going forward.</p> </section> <section id="change-the-default-behavior-of-zip"> <h3><a class="toc-backref" href="#change-the-default-behavior-of-zip" role="doc-backlink">Change The Default Behavior Of <code class="docutils literal notranslate"><span class="pre">zip</span></code></a></h3> <p>There is nothing “wrong” with the default behavior of <code class="docutils literal notranslate"><span class="pre">zip</span></code>, since there are many cases where it is indeed the correct way to handle unequally-sized inputs. It’s extremely useful, for example, when dealing with infinite iterators.</p> <p><code class="docutils literal notranslate"><span class="pre">itertools.zip_longest</span></code> already exists to service those cases where the “extra” tail-end data is still needed.</p> </section> <section id="accept-a-callback-to-handle-remaining-items"> <h3><a class="toc-backref" href="#accept-a-callback-to-handle-remaining-items" role="doc-backlink">Accept A Callback To Handle Remaining Items</a></h3> <p>While able to do basically anything a user could need, this solution makes handling the more common cases (like rejecting mismatched lengths) unnecessarily complicated and non-obvious.</p> </section> <section id="raise-an-assertionerror"> <h3><a class="toc-backref" href="#raise-an-assertionerror" role="doc-backlink">Raise An <code class="docutils literal notranslate"><span class="pre">AssertionError</span></code></a></h3> <p>There are no built-in functions or types that raise an <code class="docutils literal notranslate"><span class="pre">AssertionError</span></code> as part of their API. Further, the <a class="reference external" href="https://docs.python.org/3.9/library/exceptions.html?highlight=assertionerror#AssertionError">official documentation</a> simply reads (in its entirety):</p> <blockquote> <div>Raised when an <code class="docutils literal notranslate"><span class="pre">assert</span></code> statement fails.</div></blockquote> <p>Since this feature has nothing to do with Python’s <code class="docutils literal notranslate"><span class="pre">assert</span></code> statement, raising an <code class="docutils literal notranslate"><span class="pre">AssertionError</span></code> here would be inappropriate. Users desiring a check that is disabled in optimized mode (like an <code class="docutils literal notranslate"><span class="pre">assert</span></code> statement) can use <code class="docutils literal notranslate"><span class="pre">strict=__debug__</span></code> instead.</p> </section> <section id="add-a-similar-feature-to-map"> <h3><a class="toc-backref" href="#add-a-similar-feature-to-map" role="doc-backlink">Add A Similar Feature to <code class="docutils literal notranslate"><span class="pre">map</span></code></a></h3> <p>This PEP does not propose any changes to <code class="docutils literal notranslate"><span class="pre">map</span></code>, since the use of <code class="docutils literal notranslate"><span class="pre">map</span></code> with multiple iterable arguments is quite rare. However, this PEP’s ruling shall serve as precedent such a future discussion (should it occur).</p> <p>If rejected, the feature is realistically not worth pursuing. If accepted, such a change to <code class="docutils literal notranslate"><span class="pre">map</span></code> should not require its own PEP (though, like all enhancements, its usefulness should be carefully considered). For consistency, it should follow same API and semantics debated here for <code class="docutils literal notranslate"><span class="pre">zip</span></code>.</p> </section> <section id="do-nothing"> <h3><a class="toc-backref" href="#do-nothing" role="doc-backlink">Do Nothing</a></h3> <p>This option is perhaps the least attractive.</p> <p>Silently truncated data is a particularly nasty class of bug, and hand-writing a robust solution that gets this right <a class="reference external" href="https://stackoverflow.com/questions/32954486/zip-iterators-asserting-for-equal-length-in-python">isn’t trivial</a>. The real-world motivating examples from Python’s own standard library are evidence that it’s <em>very</em> easy to fall into the sort of trap that this feature aims to avoid.</p> </section> </section> <section id="references"> <h2><a class="toc-backref" href="#references" role="doc-backlink">References</a></h2> <section id="examples"> <h3><a class="toc-backref" href="#examples" role="doc-backlink">Examples</a></h3> <div class="admonition note"> <p class="admonition-title">Note</p> <p>This listing is not exhaustive.</p> </div> <ul class="simple"> <li><a class="reference external" href="https://github.com/python/cpython/blob/27c0d9b54abaa4112d5a317b8aa78b39ad60a808/Lib/_pydecimal.py#L3394">https://github.com/python/cpython/blob/27c0d9b54abaa4112d5a317b8aa78b39ad60a808/Lib/_pydecimal.py#L3394</a></li> <li><a class="reference external" href="https://github.com/python/cpython/blob/27c0d9b54abaa4112d5a317b8aa78b39ad60a808/Lib/_pydecimal.py#L3418">https://github.com/python/cpython/blob/27c0d9b54abaa4112d5a317b8aa78b39ad60a808/Lib/_pydecimal.py#L3418</a></li> <li><a class="reference external" href="https://github.com/python/cpython/blob/27c0d9b54abaa4112d5a317b8aa78b39ad60a808/Lib/_pydecimal.py#L3435">https://github.com/python/cpython/blob/27c0d9b54abaa4112d5a317b8aa78b39ad60a808/Lib/_pydecimal.py#L3435</a></li> <li><a class="reference external" href="https://github.com/python/cpython/blob/27c0d9b54abaa4112d5a317b8aa78b39ad60a808/Lib/ast.py#L94-L95">https://github.com/python/cpython/blob/27c0d9b54abaa4112d5a317b8aa78b39ad60a808/Lib/ast.py#L94-L95</a></li> <li><a class="reference external" href="https://github.com/python/cpython/blob/27c0d9b54abaa4112d5a317b8aa78b39ad60a808/Lib/ast.py#L1184">https://github.com/python/cpython/blob/27c0d9b54abaa4112d5a317b8aa78b39ad60a808/Lib/ast.py#L1184</a></li> <li><a class="reference external" href="https://github.com/python/cpython/blob/27c0d9b54abaa4112d5a317b8aa78b39ad60a808/Lib/ast.py#L1275">https://github.com/python/cpython/blob/27c0d9b54abaa4112d5a317b8aa78b39ad60a808/Lib/ast.py#L1275</a></li> <li><a class="reference external" href="https://github.com/python/cpython/blob/27c0d9b54abaa4112d5a317b8aa78b39ad60a808/Lib/ast.py#L1363">https://github.com/python/cpython/blob/27c0d9b54abaa4112d5a317b8aa78b39ad60a808/Lib/ast.py#L1363</a></li> <li><a class="reference external" href="https://github.com/python/cpython/blob/27c0d9b54abaa4112d5a317b8aa78b39ad60a808/Lib/ast.py#L1391">https://github.com/python/cpython/blob/27c0d9b54abaa4112d5a317b8aa78b39ad60a808/Lib/ast.py#L1391</a></li> <li><a class="reference external" href="https://github.com/python/cpython/blob/27c0d9b54abaa4112d5a317b8aa78b39ad60a808/Lib/copy.py#L217">https://github.com/python/cpython/blob/27c0d9b54abaa4112d5a317b8aa78b39ad60a808/Lib/copy.py#L217</a></li> <li><a class="reference external" href="https://github.com/python/cpython/blob/27c0d9b54abaa4112d5a317b8aa78b39ad60a808/Lib/csv.py#L142">https://github.com/python/cpython/blob/27c0d9b54abaa4112d5a317b8aa78b39ad60a808/Lib/csv.py#L142</a></li> <li><a class="reference external" href="https://github.com/python/cpython/blob/27c0d9b54abaa4112d5a317b8aa78b39ad60a808/Lib/dis.py#L462">https://github.com/python/cpython/blob/27c0d9b54abaa4112d5a317b8aa78b39ad60a808/Lib/dis.py#L462</a></li> <li><a class="reference external" href="https://github.com/python/cpython/blob/27c0d9b54abaa4112d5a317b8aa78b39ad60a808/Lib/filecmp.py#L142">https://github.com/python/cpython/blob/27c0d9b54abaa4112d5a317b8aa78b39ad60a808/Lib/filecmp.py#L142</a></li> <li><a class="reference external" href="https://github.com/python/cpython/blob/27c0d9b54abaa4112d5a317b8aa78b39ad60a808/Lib/filecmp.py#L143">https://github.com/python/cpython/blob/27c0d9b54abaa4112d5a317b8aa78b39ad60a808/Lib/filecmp.py#L143</a></li> <li><a class="reference external" href="https://github.com/python/cpython/blob/27c0d9b54abaa4112d5a317b8aa78b39ad60a808/Lib/inspect.py#L1440">https://github.com/python/cpython/blob/27c0d9b54abaa4112d5a317b8aa78b39ad60a808/Lib/inspect.py#L1440</a></li> <li><a class="reference external" href="https://github.com/python/cpython/blob/27c0d9b54abaa4112d5a317b8aa78b39ad60a808/Lib/inspect.py#L2095">https://github.com/python/cpython/blob/27c0d9b54abaa4112d5a317b8aa78b39ad60a808/Lib/inspect.py#L2095</a></li> <li><a class="reference external" href="https://github.com/python/cpython/blob/27c0d9b54abaa4112d5a317b8aa78b39ad60a808/Lib/os.py#L510">https://github.com/python/cpython/blob/27c0d9b54abaa4112d5a317b8aa78b39ad60a808/Lib/os.py#L510</a></li> <li><a class="reference external" href="https://github.com/python/cpython/blob/27c0d9b54abaa4112d5a317b8aa78b39ad60a808/Lib/plistlib.py#L577">https://github.com/python/cpython/blob/27c0d9b54abaa4112d5a317b8aa78b39ad60a808/Lib/plistlib.py#L577</a></li> <li><a class="reference external" href="https://github.com/python/cpython/blob/27c0d9b54abaa4112d5a317b8aa78b39ad60a808/Lib/tarfile.py#L1317">https://github.com/python/cpython/blob/27c0d9b54abaa4112d5a317b8aa78b39ad60a808/Lib/tarfile.py#L1317</a></li> <li><a class="reference external" href="https://github.com/python/cpython/blob/27c0d9b54abaa4112d5a317b8aa78b39ad60a808/Lib/tarfile.py#L1323">https://github.com/python/cpython/blob/27c0d9b54abaa4112d5a317b8aa78b39ad60a808/Lib/tarfile.py#L1323</a></li> <li><a class="reference external" href="https://github.com/python/cpython/blob/27c0d9b54abaa4112d5a317b8aa78b39ad60a808/Lib/tarfile.py#L1339">https://github.com/python/cpython/blob/27c0d9b54abaa4112d5a317b8aa78b39ad60a808/Lib/tarfile.py#L1339</a></li> <li><a class="reference external" href="https://github.com/python/cpython/blob/27c0d9b54abaa4112d5a317b8aa78b39ad60a808/Lib/turtle.py#L3015">https://github.com/python/cpython/blob/27c0d9b54abaa4112d5a317b8aa78b39ad60a808/Lib/turtle.py#L3015</a></li> <li><a class="reference external" href="https://github.com/python/cpython/blob/27c0d9b54abaa4112d5a317b8aa78b39ad60a808/Lib/turtle.py#L3071">https://github.com/python/cpython/blob/27c0d9b54abaa4112d5a317b8aa78b39ad60a808/Lib/turtle.py#L3071</a></li> <li><a class="reference external" href="https://github.com/python/cpython/blob/27c0d9b54abaa4112d5a317b8aa78b39ad60a808/Lib/turtle.py#L3901">https://github.com/python/cpython/blob/27c0d9b54abaa4112d5a317b8aa78b39ad60a808/Lib/turtle.py#L3901</a></li> </ul> </section> </section> <section id="copyright"> <h2><a class="toc-backref" href="#copyright" role="doc-backlink">Copyright</a></h2> <p>This document is placed in the public domain or under the CC0-1.0-Universal license, whichever is more permissive.</p> </section> </section> <hr class="docutils" /> <p>Source: <a class="reference external" href="https://github.com/python/peps/blob/main/peps/pep-0618.rst">https://github.com/python/peps/blob/main/peps/pep-0618.rst</a></p> <p>Last modified: <a class="reference external" href="https://github.com/python/peps/commits/main/peps/pep-0618.rst">2025-02-01 08:59:27 GMT</a></p> </article> <nav id="pep-sidebar"> <h2>Contents</h2> <ul> <li><a class="reference internal" href="#abstract">Abstract</a></li> <li><a class="reference internal" href="#motivation">Motivation</a></li> <li><a class="reference internal" href="#rationale">Rationale</a></li> <li><a class="reference internal" href="#specification">Specification</a></li> <li><a class="reference internal" href="#backward-compatibility">Backward Compatibility</a></li> <li><a class="reference internal" href="#reference-implementation">Reference Implementation</a></li> <li><a class="reference internal" href="#rejected-ideas">Rejected Ideas</a><ul> <li><a class="reference internal" href="#add-itertools-zip-strict">Add <code class="docutils literal notranslate"><span class="pre">itertools.zip_strict</span></code></a><ul> <li><a class="reference internal" href="#precedent">Precedent</a></li> <li><a class="reference internal" href="#usability">Usability</a></li> <li><a class="reference internal" href="#maintenance-cost">Maintenance Cost</a></li> </ul> </li> <li><a class="reference internal" href="#add-several-modes-to-switch-between">Add Several “Modes” To Switch Between</a></li> <li><a class="reference internal" href="#add-a-method-or-alternate-constructor-to-the-zip-type">Add A Method Or Alternate Constructor To The <code class="docutils literal notranslate"><span class="pre">zip</span></code> Type</a></li> <li><a class="reference internal" href="#change-the-default-behavior-of-zip">Change The Default Behavior Of <code class="docutils literal notranslate"><span class="pre">zip</span></code></a></li> <li><a class="reference internal" href="#accept-a-callback-to-handle-remaining-items">Accept A Callback To Handle Remaining Items</a></li> <li><a class="reference internal" href="#raise-an-assertionerror">Raise An <code class="docutils literal notranslate"><span class="pre">AssertionError</span></code></a></li> <li><a class="reference internal" href="#add-a-similar-feature-to-map">Add A Similar Feature to <code class="docutils literal notranslate"><span class="pre">map</span></code></a></li> <li><a class="reference internal" href="#do-nothing">Do Nothing</a></li> </ul> </li> <li><a class="reference internal" href="#references">References</a><ul> <li><a class="reference internal" href="#examples">Examples</a></li> </ul> </li> <li><a class="reference internal" href="#copyright">Copyright</a></li> </ul> <br> <a id="source" href="https://github.com/python/peps/blob/main/peps/pep-0618.rst">Page Source (GitHub)</a> </nav> </section> <script src="../_static/colour_scheme.js"></script> <script src="../_static/wrap_tables.js"></script> <script src="../_static/sticky_banner.js"></script> </body> </html>