CINXE.COM

<!DOCTYPE html> <html lang="en"> <head> <meta charset="utf-8"> <meta name="viewport" content="width=device-width, initial-scale=1.0"> <meta name="color-scheme" content="light dark"> <title>PEP 3106 – Revamping dict.keys(), .values() and .items() | peps.python.org</title> <link rel="shortcut icon" href="../_static/py.png"> <link rel="canonical" href="https://peps.python.org/pep-3106/"> <link rel="stylesheet" href="../_static/style.css" type="text/css"> <link rel="stylesheet" href="../_static/mq.css" type="text/css"> <link rel="stylesheet" href="../_static/pygments.css" type="text/css" media="(prefers-color-scheme: light)" id="pyg-light"> <link rel="stylesheet" href="../_static/pygments_dark.css" type="text/css" media="(prefers-color-scheme: dark)" id="pyg-dark"> <link rel="alternate" type="application/rss+xml" title="Latest PEPs" href="https://peps.python.org/peps.rss"> <meta property="og:title" content='PEP 3106 – Revamping dict.keys(), .values() and .items() | peps.python.org'> <meta property="og:description" content="This PEP proposes to change the .keys(), .values() and .items() methods of the built-in dict type to return a set-like or unordered container object whose contents are derived from the underlying dictionary rather than a list which is a copy of the keys..."> <meta property="og:type" content="website"> <meta property="og:url" content="https://peps.python.org/pep-3106/"> <meta property="og:site_name" content="Python Enhancement Proposals (PEPs)"> <meta property="og:image" content="https://peps.python.org/_static/og-image.png"> <meta property="og:image:alt" content="Python PEPs"> <meta property="og:image:width" content="200"> <meta property="og:image:height" content="200"> <meta name="description" content="This PEP proposes to change the .keys(), .values() and .items() methods of the built-in dict type to return a set-like or unordered container object whose contents are derived from the underlying dictionary rather than a list which is a copy of the keys..."> <meta name="theme-color" content="#3776ab"> </head> <body> <svg xmlns="http://www.w3.org/2000/svg" style="display: none;"> <symbol id="svg-sun-half" viewBox="0 0 24 24" pointer-events="all"> <title>Following system colour scheme</title> <svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"> <circle cx="12" cy="12" r="9"></circle> <path d="M12 3v18m0-12l4.65-4.65M12 14.3l7.37-7.37M12 19.6l8.85-8.85"></path> </svg> </symbol> <symbol id="svg-moon" viewBox="0 0 24 24" pointer-events="all"> <title>Selected dark colour scheme</title> <svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"> <path stroke="none" d="M0 0h24v24H0z" fill="none"></path> <path d="M12 3c.132 0 .263 0 .393 0a7.5 7.5 0 0 0 7.92 12.446a9 9 0 1 1 -8.313 -12.454z"></path> </svg> </symbol> <symbol id="svg-sun" viewBox="0 0 24 24" pointer-events="all"> <title>Selected light colour scheme</title> <svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"> <circle cx="12" cy="12" r="5"></circle> <line x1="12" y1="1" x2="12" y2="3"></line> <line x1="12" y1="21" x2="12" y2="23"></line> <line x1="4.22" y1="4.22" x2="5.64" y2="5.64"></line> <line x1="18.36" y1="18.36" x2="19.78" y2="19.78"></line> <line x1="1" y1="12" x2="3" y2="12"></line> <line x1="21" y1="12" x2="23" y2="12"></line> <line x1="4.22" y1="19.78" x2="5.64" y2="18.36"></line> <line x1="18.36" y1="5.64" x2="19.78" y2="4.22"></line> </svg> </symbol> </svg> <script> document.documentElement.dataset.colour_scheme = localStorage.getItem("colour_scheme") || "auto" </script> <section id="pep-page-section"> <header> <h1>Python Enhancement Proposals</h1> <ul class="breadcrumbs"> <li><a href="https://www.python.org/" title="The Python Programming Language">Python</a> » </li> <li><a href="../pep-0000/">PEP Index</a> » </li> <li>PEP 3106</li> </ul> <button id="colour-scheme-cycler" onClick="setColourScheme(nextColourScheme())"> <svg aria-hidden="true" class="colour-scheme-icon-when-auto"><use href="#svg-sun-half"></use></svg> <svg aria-hidden="true" class="colour-scheme-icon-when-dark"><use href="#svg-moon"></use></svg> <svg aria-hidden="true" class="colour-scheme-icon-when-light"><use href="#svg-sun"></use></svg> Toggle light / dark / auto colour theme </button> </header> <article> <section id="pep-content"> <h1 class="page-title">PEP 3106 – Revamping dict.keys(), .values() and .items()</h1> <dl class="rfc2822 field-list simple"> <dt class="field-odd">Author:</dt> <dd class="field-odd">Guido van Rossum</dd> <dt class="field-even">Status:</dt> <dd class="field-even"><abbr title="Accepted and implementation complete, or no longer active">Final</abbr></dd> <dt class="field-odd">Type:</dt> <dd class="field-odd"><abbr title="Normative PEP with a new feature for Python, implementation change for CPython or interoperability standard for the ecosystem">Standards Track</abbr></dd> <dt class="field-even">Created:</dt> <dd class="field-even">19-Dec-2006</dd> <dt class="field-odd">Python-Version:</dt> <dd class="field-odd">3.0</dd> <dt class="field-even">Post-History:</dt> <dd class="field-even"></dd> </dl> <hr class="docutils" /> <section id="contents"> <details><summary>Table of Contents</summary><ul class="simple"> <li><a class="reference internal" href="#abstract">Abstract</a></li> <li><a class="reference internal" href="#introduction">Introduction</a></li> <li><a class="reference internal" href="#specification">Specification</a></li> <li><a class="reference internal" href="#open-issues">Open Issues</a></li> <li><a class="reference internal" href="#references">References</a></li> </ul> </details></section> <section id="abstract"> <h2><a class="toc-backref" href="#abstract" role="doc-backlink">Abstract</a></h2> This PEP proposes to change the .keys(), .values() and .items() methods of the built-in dict type to return a set-like or unordered container object whose contents are derived from the underlying dictionary rather than a list which is a copy of the keys, etc.; and to remove the .iterkeys(), .itervalues() and .iteritems() methods. The approach is inspired by that taken in the Java Collections Framework <a class="footnote-reference brackets" href="#id3" id="id1">[1]</a>. </section> <section id="introduction"> <h2><a class="toc-backref" href="#introduction" role="doc-backlink">Introduction</a></h2> It has long been the plan to change the .keys(), .values() and .items() methods of the built-in dict type to return a more lightweight object than a list, and to get rid of .iterkeys(), .itervalues() and .iteritems(). The idea is that code that currently (in 2.x) reads: <div class="highlight-default notranslate"><div class="highlight"><pre>for k, v in d.iteritems(): ... </pre></div> </div> should be rewritten as: <div class="highlight-default notranslate"><div class="highlight"><pre>for k, v in d.items(): ... </pre></div> </div> (and similar for .itervalues() and .iterkeys(), except the latter is redundant since we can write that loop as <code class="docutils literal notranslate">for k in d</code>.) Code that currently reads: <div class="highlight-default notranslate"><div class="highlight"><pre>a = d.keys() # assume we really want a list here </pre></div> </div> (etc.) should be rewritten as <blockquote> <div>a = list(d.keys())</div></blockquote> There are (at least) two ways to accomplish this. The original plan was to simply let .keys(), .values() and .items() return an iterator, i.e. exactly what iterkeys(), itervalues() and iteritems() return in Python 2.x. However, the Java Collections Framework <a class="footnote-reference brackets" href="#id3" id="id2">[1]</a> suggests that a better solution is possible: the methods return objects with set behavior (for .keys() and .items()) or multiset (== bag) behavior (for .values()) that do not contain copies of the keys, values or items, but rather reference the underlying dict and pull their values out of the dict as needed. The advantage of this approach is that one can still write code like this: <div class="highlight-default notranslate"><div class="highlight"><pre>a = d.items() for k, v in a: ... # And later, again: for k, v in a: ... </pre></div> </div> Effectively, iter(d.keys()) (etc.) in Python 3.0 will do what d.iterkeys() (etc.) does in Python 2.x; but in most contexts we don’t have to write the iter() call because it is implied by a for-loop. The objects returned by the .keys() and .items() methods behave like sets. The object returned by the values() method behaves like a much simpler unordered collection – it cannot be a set because duplicate values are possible. Because of the set behavior, it will be possible to check whether two dicts have the same keys by simply testing: <div class="highlight-default notranslate"><div class="highlight"><pre>if a.keys() == b.keys(): ... </pre></div> </div> and similarly for .items(). These operations are thread-safe only to the extent that using them in a thread-unsafe way may cause an exception but will not cause corruption of the internal representation. As in Python 2.x, mutating a dict while iterating over it using an iterator has an undefined effect and will in most cases raise a RuntimeError exception. (This is similar to the guarantees made by the Java Collections Framework.) The objects returned by .keys() and .items() are fully interoperable with instances of the built-in set and frozenset types; for example: <div class="highlight-default notranslate"><div class="highlight"><pre>set(d.keys()) == d.keys() </pre></div> </div> is guaranteed to be True (except when d is being modified simultaneously by another thread). </section> <section id="specification"> <h2><a class="toc-backref" href="#specification" role="doc-backlink">Specification</a></h2> I’m using pseudo-code to specify the semantics: <div class="highlight-default notranslate"><div class="highlight"><pre>class dict: # Omitting all other dict methods for brevity. # The .iterkeys(), .itervalues() and .iteritems() methods # will be removed. def keys(self): return d_keys(self) def items(self): return d_items(self) def values(self): return d_values(self) class d_keys: def __init__(self, d): self.__d = d def __len__(self): return len(self.__d) def __contains__(self, key): return key in self.__d def __iter__(self): for key in self.__d: yield key # The following operations should be implemented to be # compatible with sets; this can be done by exploiting # the above primitive operations: # # <, <=, ==, !=, >=, > (returning a bool) # &, |, ^, - (returning a new, real set object) # # as well as their method counterparts (.union(), etc.). # # To specify the semantics, we can specify x == y as: # # set(x) == set(y) if both x and y are d_keys instances # set(x) == y if x is a d_keys instance # x == set(y) if y is a d_keys instance # # and so on for all other operations. class d_items: def __init__(self, d): self.__d = d def __len__(self): return len(self.__d) def __contains__(self, (key, value)): return key in self.__d and self.__d[key] == value def __iter__(self): for key in self.__d: yield key, self.__d[key] # As well as the set operations mentioned for d_keys above. # However the specifications suggested there will not work if # the values aren't hashable. Fortunately, the operations can # still be implemented efficiently. For example, this is how # intersection can be specified: def __and__(self, other): if isinstance(other, (set, frozenset, d_keys)): result = set() for item in other: if item in self: result.add(item) return result if not isinstance(other, d_items): return NotImplemented d = {} if len(other) < len(self): self, other = other, self for item in self: if item in other: key, value = item d[key] = value return d.items() # And here is equality: def __eq__(self, other): if isinstance(other, (set, frozenset, d_keys)): if len(self) != len(other): return False for item in other: if item not in self: return False return True if not isinstance(other, d_items): return NotImplemented # XXX We could also just compare the underlying dicts... if len(self) != len(other): return False for item in self: if item not in other: return False return True def __ne__(self, other): # XXX Perhaps object.__ne__() should be defined this way. result = self.__eq__(other) if result is not NotImplemented: result = not result return result class d_values: def __init__(self, d): self.__d = d def __len__(self): return len(self.__d) def __contains__(self, value): # This is slow, and it's what "x in y" uses as a fallback # if __contains__ is not defined; but I'd rather make it # explicit that it is supported. for v in self: if v == value: return True return False def __iter__(self): for key in self.__d: yield self.__d[key] def __eq__(self, other): if not isinstance(other, d_values): return NotImplemented if len(self) != len(other): return False # XXX Sometimes this could be optimized, but these are the # semantics: we can't depend on the values to be hashable # or comparable. olist = list(other) for x in self: try: olist.remove(x) except ValueError: return False assert olist == [] return True def __ne__(self, other): result = self.__eq__(other) if result is not NotImplemented: result = not result return result </pre></div> </div> Notes: The view objects are not directly mutable, but don’t implement __hash__(); their value can change if the underlying dict is mutated. The only requirements on the underlying dict are that it implements __getitem__(), __contains__(), __iter__(), and __len__(). We don’t implement .copy() – the presence of a .copy() method suggests that the copy has the same type as the original, but that’s not feasible without copying the underlying dict. If you want a copy of a specific type, like list or set, you can just pass one of the above to the list() or set() constructor. The specification implies that the order in which items are returned by .keys(), .values() and .items() is the same (just as it was in Python 2.x), because the order is all derived from the dict iterator (which is presumably arbitrary but stable as long as a dict isn’t modified). This can be expressed by the following invariant: <div class="highlight-default notranslate"><div class="highlight"><pre>list(d.items()) == list(zip(d.keys(), d.values())) </pre></div> </div> </section> <section id="open-issues"> <h2><a class="toc-backref" href="#open-issues" role="doc-backlink">Open Issues</a></h2> Do we need more of a motivation? I would think that being able to do set operations on keys and items without having to copy them should speak for itself. I’ve left out the implementation of various set operations. These could still present small surprises. It would be okay if multiple calls to d.keys() (etc.) returned the same object, since the object’s only state is the dict to which it refers. Is this worth having extra slots in the dict object for? Should that be a weak reference or should the d_keys (etc.) object live forever once created? Strawman: probably not worth the extra slots in every dict. Should d_keys, d_values and d_items have a public instance variable or method through which one can retrieve the underlying dict? Strawman: yes (but what should it be called?). I’m soliciting better names than d_keys, d_values and d_items. These classes could be public so that their implementations could be reused by the .keys(), .values() and .items() methods of other mappings. Or should they? Should the d_keys, d_values and d_items classes be reusable? Strawman: yes. Should they be subclassable? Strawman: yes (but see below). A particularly nasty issue is whether operations that are specified in terms of other operations (e.g. .discard()) must really be implemented in terms of those other operations; this may appear irrelevant but it becomes relevant if these classes are ever subclassed. Historically, Python has a really poor track record of specifying the semantics of highly optimized built-in types clearly in such cases; my strawman is to continue that trend. Subclassing may still be useful to add new methods, for example. I’ll leave the decisions (especially about naming) up to whoever submits a working implementation. </section> <section id="references"> <h2><a class="toc-backref" href="#references" role="doc-backlink">References</a></h2> <aside class="footnote-list brackets"> <aside class="footnote brackets" id="id3" role="doc-footnote"> <dt class="label" id="id3">[1] (<a href='#id1'>1</a>, <a href='#id2'>2</a>) </dt> <dd>Java Collections Framework <a class="reference external" href="http://java.sun.com/docs/books/tutorial/collections/index.html">http://java.sun.com/docs/books/tutorial/collections/index.html</a></aside> </aside> </section> </section> <hr class="docutils" /> Source: <a class="reference external" href="https://github.com/python/peps/blob/main/peps/pep-3106.rst">https://github.com/python/peps/blob/main/peps/pep-3106.rst</a> Last modified: <a class="reference external" href="https://github.com/python/peps/commits/main/peps/pep-3106.rst">2025-02-01 08:59:27 GMT</a> </article> <nav id="pep-sidebar"> <h2>Contents</h2> <ul> <li><a class="reference internal" href="#abstract">Abstract</a></li> <li><a class="reference internal" href="#introduction">Introduction</a></li> <li><a class="reference internal" href="#specification">Specification</a></li> <li><a class="reference internal" href="#open-issues">Open Issues</a></li> <li><a class="reference internal" href="#references">References</a></li> </ul> <a id="source" href="https://github.com/python/peps/blob/main/peps/pep-3106.rst">Page Source (GitHub)</a> </nav> </section> <script src="../_static/colour_scheme.js"></script> <script src="../_static/wrap_tables.js"></script> <script src="../_static/sticky_banner.js"></script> </body> </html>