CINXE.COM

<!DOCTYPE html> <html lang="en"> <head> <meta charset="utf-8"> <meta name="viewport" content="width=device-width, initial-scale=1.0"> <meta name="color-scheme" content="light dark"> <title>PEP 307 – Extensions to the pickle protocol | peps.python.org</title> <link rel="shortcut icon" href="../_static/py.png"> <link rel="canonical" href="https://peps.python.org/pep-0307/"> <link rel="stylesheet" href="../_static/style.css" type="text/css"> <link rel="stylesheet" href="../_static/mq.css" type="text/css"> <link rel="stylesheet" href="../_static/pygments.css" type="text/css" media="(prefers-color-scheme: light)" id="pyg-light"> <link rel="stylesheet" href="../_static/pygments_dark.css" type="text/css" media="(prefers-color-scheme: dark)" id="pyg-dark"> <link rel="alternate" type="application/rss+xml" title="Latest PEPs" href="https://peps.python.org/peps.rss"> <meta property="og:title" content='PEP 307 – Extensions to the pickle protocol | peps.python.org'> <meta property="og:description" content="Pickling new-style objects in Python 2.2 is done somewhat clumsily and causes pickle size to bloat compared to classic class instances. This PEP documents a new pickle protocol in Python 2.3 that takes care of this and many other pickle issues."> <meta property="og:type" content="website"> <meta property="og:url" content="https://peps.python.org/pep-0307/"> <meta property="og:site_name" content="Python Enhancement Proposals (PEPs)"> <meta property="og:image" content="https://peps.python.org/_static/og-image.png"> <meta property="og:image:alt" content="Python PEPs"> <meta property="og:image:width" content="200"> <meta property="og:image:height" content="200"> <meta name="description" content="Pickling new-style objects in Python 2.2 is done somewhat clumsily and causes pickle size to bloat compared to classic class instances. This PEP documents a new pickle protocol in Python 2.3 that takes care of this and many other pickle issues."> <meta name="theme-color" content="#3776ab"> </head> <body> <svg xmlns="http://www.w3.org/2000/svg" style="display: none;"> <symbol id="svg-sun-half" viewBox="0 0 24 24" pointer-events="all"> <title>Following system colour scheme</title> <svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"> <circle cx="12" cy="12" r="9"></circle> <path d="M12 3v18m0-12l4.65-4.65M12 14.3l7.37-7.37M12 19.6l8.85-8.85"></path> </svg> </symbol> <symbol id="svg-moon" viewBox="0 0 24 24" pointer-events="all"> <title>Selected dark colour scheme</title> <svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"> <path stroke="none" d="M0 0h24v24H0z" fill="none"></path> <path d="M12 3c.132 0 .263 0 .393 0a7.5 7.5 0 0 0 7.92 12.446a9 9 0 1 1 -8.313 -12.454z"></path> </svg> </symbol> <symbol id="svg-sun" viewBox="0 0 24 24" pointer-events="all"> <title>Selected light colour scheme</title> <svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"> <circle cx="12" cy="12" r="5"></circle> <line x1="12" y1="1" x2="12" y2="3"></line> <line x1="12" y1="21" x2="12" y2="23"></line> <line x1="4.22" y1="4.22" x2="5.64" y2="5.64"></line> <line x1="18.36" y1="18.36" x2="19.78" y2="19.78"></line> <line x1="1" y1="12" x2="3" y2="12"></line> <line x1="21" y1="12" x2="23" y2="12"></line> <line x1="4.22" y1="19.78" x2="5.64" y2="18.36"></line> <line x1="18.36" y1="5.64" x2="19.78" y2="4.22"></line> </svg> </symbol> </svg> <script> document.documentElement.dataset.colour_scheme = localStorage.getItem("colour_scheme") || "auto" </script> <section id="pep-page-section"> <header> <h1>Python Enhancement Proposals</h1> <ul class="breadcrumbs"> <li><a href="https://www.python.org/" title="The Python Programming Language">Python</a> » </li> <li><a href="../pep-0000/">PEP Index</a> » </li> <li>PEP 307</li> </ul> <button id="colour-scheme-cycler" onClick="setColourScheme(nextColourScheme())"> <svg aria-hidden="true" class="colour-scheme-icon-when-auto"><use href="#svg-sun-half"></use></svg> <svg aria-hidden="true" class="colour-scheme-icon-when-dark"><use href="#svg-moon"></use></svg> <svg aria-hidden="true" class="colour-scheme-icon-when-light"><use href="#svg-sun"></use></svg> Toggle light / dark / auto colour theme </button> </header> <article> <section id="pep-content"> <h1 class="page-title">PEP 307 – Extensions to the pickle protocol</h1> <dl class="rfc2822 field-list simple"> <dt class="field-odd">Author:</dt> <dd class="field-odd">Guido van Rossum, Tim Peters</dd> <dt class="field-even">Status:</dt> <dd class="field-even"><abbr title="Accepted and implementation complete, or no longer active">Final</abbr></dd> <dt class="field-odd">Type:</dt> <dd class="field-odd"><abbr title="Normative PEP with a new feature for Python, implementation change for CPython or interoperability standard for the ecosystem">Standards Track</abbr></dd> <dt class="field-even">Created:</dt> <dd class="field-even">31-Jan-2003</dd> <dt class="field-odd">Python-Version:</dt> <dd class="field-odd">2.3</dd> <dt class="field-even">Post-History:</dt> <dd class="field-even">07-Feb-2003</dd> </dl> <hr class="docutils" /> <section id="contents"> <details><summary>Table of Contents</summary><ul class="simple"> <li><a class="reference internal" href="#introduction">Introduction</a></li> <li><a class="reference internal" href="#motivation">Motivation</a></li> <li><a class="reference internal" href="#protocol-versions">Protocol versions</a></li> <li><a class="reference internal" href="#security-issues">Security issues</a></li> <li><a class="reference internal" href="#extended-reduce-api">Extended <code class="docutils literal notranslate">__reduce__</code> API</a></li> <li><a class="reference internal" href="#the-reduce-ex-api">The <code class="docutils literal notranslate">__reduce_ex__</code> API</a></li> <li><a class="reference internal" href="#customizing-pickling-absent-a-reduce-implementation">Customizing pickling absent a <code class="docutils literal notranslate">__reduce__</code> implementation</a><ul> <li><a class="reference internal" href="#case-1-pickling-classic-class-instances">Case 1: pickling classic class instances</a><ul> <li><a class="reference internal" href="#the-getstate-method">The <code class="docutils literal notranslate">__getstate__</code> method</a></li> <li><a class="reference internal" href="#the-setstate-method">The <code class="docutils literal notranslate">__setstate__</code> method</a></li> <li><a class="reference internal" href="#the-getinitargs-method">The <code class="docutils literal notranslate">__getinitargs__</code> method</a></li> </ul> </li> <li><a class="reference internal" href="#case-2-pickling-new-style-class-instances-using-protocols-0-or-1">Case 2: pickling new-style class instances using protocols 0 or 1</a></li> <li><a class="reference internal" href="#case-3-pickling-new-style-class-instances-using-protocol-2">Case 3: pickling new-style class instances using protocol 2</a><ul> <li><a class="reference internal" href="#id1">The <code class="docutils literal notranslate">__getstate__</code> method</a></li> <li><a class="reference internal" href="#id2">The <code class="docutils literal notranslate">__setstate__</code> method</a></li> <li><a class="reference internal" href="#the-getnewargs-method">The <code class="docutils literal notranslate">__getnewargs__</code> method</a></li> </ul> </li> </ul> </li> <li><a class="reference internal" href="#the-newobj-unpickling-function">The <code class="docutils literal notranslate">__newobj__</code> unpickling function</a></li> <li><a class="reference internal" href="#the-extension-registry">The extension registry</a><ul> <li><a class="reference internal" href="#extension-registry-api">Extension registry API</a></li> </ul> </li> <li><a class="reference internal" href="#the-copy-module">The copy module</a></li> <li><a class="reference internal" href="#pickling-python-longs">Pickling Python longs</a></li> <li><a class="reference internal" href="#pickling-bools">Pickling bools</a></li> <li><a class="reference internal" href="#pickling-small-tuples">Pickling small tuples</a></li> <li><a class="reference internal" href="#protocol-identification">Protocol identification</a></li> <li><a class="reference internal" href="#pickling-of-large-lists-and-dicts">Pickling of large lists and dicts</a></li> <li><a class="reference internal" href="#copyright">Copyright</a></li> </ul> </details></section> <section id="introduction"> <h2><a class="toc-backref" href="#introduction" role="doc-backlink">Introduction</a></h2> Pickling new-style objects in Python 2.2 is done somewhat clumsily and causes pickle size to bloat compared to classic class instances. This PEP documents a new pickle protocol in Python 2.3 that takes care of this and many other pickle issues. There are two sides to specifying a new pickle protocol: the byte stream constituting pickled data must be specified, and the interface between objects and the pickling and unpickling engines must be specified. This PEP focuses on API issues, although it may occasionally touch on byte stream format details to motivate a choice. The pickle byte stream format is documented formally by the standard library module <code class="docutils literal notranslate">pickletools.py</code> (already checked into CVS for Python 2.3). This PEP attempts to fully document the interface between pickled objects and the pickling process, highlighting additions by specifying “new in this PEP”. (The interface to invoke pickling or unpickling is not covered fully, except for the changes to the API for specifying the pickling protocol to picklers.) </section> <section id="motivation"> <h2><a class="toc-backref" href="#motivation" role="doc-backlink">Motivation</a></h2> Pickling new-style objects causes serious pickle bloat. For example: <div class="highlight-default notranslate"><div class="highlight"><pre>class C(object): # Omit "(object)" for classic class pass x = C() x.foo = 42 print len(pickle.dumps(x, 1)) </pre></div> </div> The binary pickle for the classic object consumed 33 bytes, and for the new-style object 86 bytes. The reasons for the bloat are complex, but are mostly caused by the fact that new-style objects use <code class="docutils literal notranslate">__reduce__</code> in order to be picklable at all. After ample consideration we’ve concluded that the only way to reduce pickle sizes for new-style objects is to add new opcodes to the pickle protocol. The net result is that with the new protocol, the pickle size in the above example is 35 (two extra bytes are used at the start to indicate the protocol version, although this isn’t strictly necessary). </section> <section id="protocol-versions"> <h2><a class="toc-backref" href="#protocol-versions" role="doc-backlink">Protocol versions</a></h2> Previously, pickling (but not unpickling) distinguished between text mode and binary mode. By design, binary mode is a superset of text mode, and unpicklers don’t need to know in advance whether an incoming pickle uses text mode or binary mode. The virtual machine used for unpickling is the same regardless of the mode; certain opcodes simply aren’t used in text mode. Retroactively, text mode is now called protocol 0, and binary mode protocol 1. The new protocol is called protocol 2. In the tradition of pickling protocols, protocol 2 is a superset of protocol 1. But just so that future pickling protocols aren’t required to be supersets of the oldest protocols, a new opcode is inserted at the start of a protocol 2 pickle indicating that it is using protocol 2. To date, each release of Python has been able to read pickles written by all previous releases. Of course pickles written under protocol N can’t be read by versions of Python earlier than the one that introduced protocol N. Several functions, methods and constructors used for pickling used to take a positional argument named ‘bin’ which was a flag, defaulting to 0, indicating binary mode. This argument is renamed to ‘protocol’ and now gives the protocol number, still defaulting to 0. It so happens that passing 2 for the ‘bin’ argument in previous Python versions had the same effect as passing 1. Nevertheless, a special case is added here: passing a negative number selects the highest protocol version supported by a particular implementation. This works in previous Python versions, too, and so can be used to select the highest protocol available in a way that’s both backward and forward compatible. In addition, a new module constant <code class="docutils literal notranslate">HIGHEST_PROTOCOL</code> is supplied by both <code class="docutils literal notranslate">pickle</code> and <code class="docutils literal notranslate">cPickle</code>, equal to the highest protocol number the module can read. This is cleaner than passing -1, but cannot be used before Python 2.3. The <code class="docutils literal notranslate">pickle.py</code> module has supported passing the ‘bin’ value as a keyword argument rather than a positional argument. (This is not recommended, since <code class="docutils literal notranslate">cPickle</code> only accepts positional arguments, but it works…) Passing ‘bin’ as a keyword argument is deprecated, and a <code class="docutils literal notranslate">PendingDeprecationWarning</code> is issued in this case. You have to invoke the Python interpreter with <code class="docutils literal notranslate">-Wa</code> or a variation on that to see <code class="docutils literal notranslate">PendingDeprecationWarning</code> messages. In Python 2.4, the warning class may be upgraded to <code class="docutils literal notranslate">DeprecationWarning</code>. </section> <section id="security-issues"> <h2><a class="toc-backref" href="#security-issues" role="doc-backlink">Security issues</a></h2> In previous versions of Python, unpickling would do a “safety check” on certain operations, refusing to call functions or constructors that weren’t marked as “safe for unpickling” by either having an attribute <code class="docutils literal notranslate">__safe_for_unpickling__</code> set to 1, or by being registered in a global registry, <code class="docutils literal notranslate">copy_reg.safe_constructors</code>. This feature gives a false sense of security: nobody has ever done the necessary, extensive, code audit to prove that unpickling untrusted pickles cannot invoke unwanted code, and in fact bugs in the Python 2.2 <code class="docutils literal notranslate">pickle.py</code> module make it easy to circumvent these security measures. We firmly believe that, on the Internet, it is better to know that you are using an insecure protocol than to trust a protocol to be secure whose implementation hasn’t been thoroughly checked. Even high quality implementations of widely used protocols are routinely found flawed; Python’s pickle implementation simply cannot make such guarantees without a much larger time investment. Therefore, as of Python 2.3, all safety checks on unpickling are officially removed, and replaced with this warning: <div class="admonition warning"> Warning Do not unpickle data received from an untrusted or unauthenticated source. </div> The same warning applies to previous Python versions, despite the presence of safety checks there. </section> <section id="extended-reduce-api"> <h2><a class="toc-backref" href="#extended-reduce-api" role="doc-backlink">Extended <code class="docutils literal notranslate">__reduce__</code> API</a></h2> There are several APIs that a class can use to control pickling. Perhaps the most popular of these are <code class="docutils literal notranslate">__getstate__</code> and <code class="docutils literal notranslate">__setstate__</code>; but the most powerful one is <code class="docutils literal notranslate">__reduce__</code>. (There’s also <code class="docutils literal notranslate">__getinitargs__</code>, and we’re adding <code class="docutils literal notranslate">__getnewargs__</code> below.) There are several ways to provide <code class="docutils literal notranslate">__reduce__</code> functionality: a class can implement a <code class="docutils literal notranslate">__reduce__</code> method or a <code class="docutils literal notranslate">__reduce_ex__</code> method (see next section), or a reduce function can be declared in <code class="docutils literal notranslate">copy_reg</code> (<code class="docutils literal notranslate">copy_reg.dispatch_table</code> maps classes to functions). The return values are interpreted exactly the same, though, and we’ll refer to these collectively as <code class="docutils literal notranslate">__reduce__</code>. Important: pickling of classic class instances does not look for a <code class="docutils literal notranslate">__reduce__</code> or <code class="docutils literal notranslate">__reduce_ex__</code> method or a reduce function in the <code class="docutils literal notranslate">copy_reg</code> dispatch table, so that a classic class cannot provide <code class="docutils literal notranslate">__reduce__</code> functionality in the sense intended here. A classic class must use <code class="docutils literal notranslate">__getinitargs__</code> and/or <code class="docutils literal notranslate">__getstate__</code> to customize pickling. These are described below. <code class="docutils literal notranslate">__reduce__</code> must return either a string or a tuple. If it returns a string, this is an object whose state is not to be pickled, but instead a reference to an equivalent object referenced by name. Surprisingly, the string returned by <code class="docutils literal notranslate">__reduce__</code> should be the object’s local name (relative to its module); the <code class="docutils literal notranslate">pickle</code> module searches the module namespace to determine the object’s module. The rest of this section is concerned with the tuple returned by <code class="docutils literal notranslate">__reduce__</code>. It is a variable size tuple, of length 2 through 5. The first two items (function and arguments) are required. The remaining items are optional and may be left off from the end; giving <code class="docutils literal notranslate">None</code> for the value of an optional item acts the same as leaving it off. The last two items are new in this PEP. The items are, in order: <table class="docutils align-default"> <tbody> <tr class="row-odd"><td>function</td> <td>Required.A callable object (not necessarily a function) called to create the initial version of the object; state may be added to the object later to fully reconstruct the pickled state. This function must itself be picklable. See the section about <code class="docutils literal notranslate">__newobj__</code> for a special case (new in this PEP) here. </td> </tr> <tr class="row-even"><td>arguments</td> <td>Required.A tuple giving the argument list for the function. As a special case, designed for Zope 2’s <code class="docutils literal notranslate">ExtensionClass</code>, this may be <code class="docutils literal notranslate">None</code>; in that case, function should be a class or type, and <code class="docutils literal notranslate">function.__basicnew__()</code> is called to create the initial version of the object. This exception is deprecated. </td> </tr> </tbody> </table> Unpickling invokes <code class="docutils literal notranslate">function(*arguments)</code> to create an initial object, called obj below. If the remaining items are left off, that’s the end of unpickling for this object and obj is the result. Else obj is modified at unpickling time by each item specified, as follows. <table class="docutils align-default"> <tbody> <tr class="row-odd"><td>state</td> <td>Optional.Additional state. If this is not <code class="docutils literal notranslate">None</code>, the state is pickled, and <code class="docutils literal notranslate">obj.__setstate__(state)</code> will be called when unpickling. If no <code class="docutils literal notranslate">__setstate__</code> method is defined, a default implementation is provided, which assumes that state is a dictionary mapping instance variable names to their values. The default implementation calls <div class="highlight-default notranslate"><div class="highlight"><pre>obj.__dict__.update(state) </pre></div> </div> or, if the <code class="docutils literal notranslate">update()</code> call fails, <div class="highlight-default notranslate"><div class="highlight"><pre>for k, v in state.items(): setattr(obj, k, v) </pre></div> </div> </td> </tr> <tr class="row-even"><td>listitems</td> <td>Optional, and new in this PEP.If this is not <code class="docutils literal notranslate">None</code>, it should be an iterator (not a sequence!) yielding successive list items. These list items will be pickled, and appended to the object using either <code class="docutils literal notranslate">obj.append(item)</code> or <code class="docutils literal notranslate">obj.extend(list_of_items)</code>. This is primarily used for <code class="docutils literal notranslate">list</code> subclasses, but may be used by other classes as long as they have <code class="docutils literal notranslate">append()</code> and <code class="docutils literal notranslate">extend()</code> methods with the appropriate signature. (Whether <code class="docutils literal notranslate">append()</code> or <code class="docutils literal notranslate">extend()</code> is used depends on which pickle protocol version is used as well as the number of items to append, so both must be supported.) </td> </tr> <tr class="row-odd"><td>dictitems</td> <td>Optional, and new in this PEP.If this is not <code class="docutils literal notranslate">None</code>, it should be an iterator (not a sequence!) yielding successive dictionary items, which should be tuples of the form <code class="docutils literal notranslate">(key, value)</code>. These items will be pickled, and stored to the object using <code class="docutils literal notranslate">obj[key] = value</code>. This is primarily used for <code class="docutils literal notranslate">dict</code> subclasses, but may be used by other classes as long as they implement <code class="docutils literal notranslate">__setitem__</code>. </td> </tr> </tbody> </table> Note: in Python 2.2 and before, when using <code class="docutils literal notranslate">cPickle</code>, state would be pickled if present even if it is <code class="docutils literal notranslate">None</code>; the only safe way to avoid the <code class="docutils literal notranslate">__setstate__</code> call was to return a two-tuple from <code class="docutils literal notranslate">__reduce__</code>. (But <code class="docutils literal notranslate">pickle.py</code> would not pickle state if it was <code class="docutils literal notranslate">None</code>.) In Python 2.3, <code class="docutils literal notranslate">__setstate__</code> will never be called at unpickling time when <code class="docutils literal notranslate">__reduce__</code> returns a state with value <code class="docutils literal notranslate">None</code> at pickling time. A <code class="docutils literal notranslate">__reduce__</code> implementation that needs to work both under Python 2.2 and under Python 2.3 could check the variable <code class="docutils literal notranslate">pickle.format_version</code> to determine whether to use the listitems and dictitems features. If this value is <code class="docutils literal notranslate">>= "2.0"</code> then they are supported. If not, any list or dict items should be incorporated somehow in the ‘state’ return value, and the <code class="docutils literal notranslate">__setstate__</code> method should be prepared to accept list or dict items as part of the state (how this is done is up to the application). </section> <section id="the-reduce-ex-api"> <h2><a class="toc-backref" href="#the-reduce-ex-api" role="doc-backlink">The <code class="docutils literal notranslate">__reduce_ex__</code> API</a></h2> It is sometimes useful to know the protocol version when implementing <code class="docutils literal notranslate">__reduce__</code>. This can be done by implementing a method named <code class="docutils literal notranslate">__reduce_ex__</code> instead of <code class="docutils literal notranslate">__reduce__</code>. <code class="docutils literal notranslate">__reduce_ex__</code>, when it exists, is called in preference over <code class="docutils literal notranslate">__reduce__</code> (you may still provide <code class="docutils literal notranslate">__reduce__</code> for backwards compatibility). The <code class="docutils literal notranslate">__reduce_ex__</code> method will be called with a single integer argument, the protocol version. The ‘object’ class implements both <code class="docutils literal notranslate">__reduce__</code> and <code class="docutils literal notranslate">__reduce_ex__</code>; however, if a subclass overrides <code class="docutils literal notranslate">__reduce__</code> but not <code class="docutils literal notranslate">__reduce_ex__</code>, the <code class="docutils literal notranslate">__reduce_ex__</code> implementation detects this and calls <code class="docutils literal notranslate">__reduce__</code>. </section> <section id="customizing-pickling-absent-a-reduce-implementation"> <h2><a class="toc-backref" href="#customizing-pickling-absent-a-reduce-implementation" role="doc-backlink">Customizing pickling absent a <code class="docutils literal notranslate">__reduce__</code> implementation</a></h2> If no <code class="docutils literal notranslate">__reduce__</code> implementation is available for a particular class, there are three cases that need to be considered separately, because they are handled differently: <ol class="arabic simple"> <li>classic class instances, all protocols</li> <li>new-style class instances, protocols 0 and 1</li> <li>new-style class instances, protocol 2</li> </ol> Types implemented in C are considered new-style classes. However, except for the common built-in types, these need to provide a <code class="docutils literal notranslate">__reduce__</code> implementation in order to be picklable with protocols 0 or 1. Protocol 2 supports built-in types providing <code class="docutils literal notranslate">__getnewargs__</code>, <code class="docutils literal notranslate">__getstate__</code> and <code class="docutils literal notranslate">__setstate__</code> as well. <section id="case-1-pickling-classic-class-instances"> <h3><a class="toc-backref" href="#case-1-pickling-classic-class-instances" role="doc-backlink">Case 1: pickling classic class instances</a></h3> This case is the same for all protocols, and is unchanged from Python 2.1. For classic classes, <code class="docutils literal notranslate">__reduce__</code> is not used. Instead, classic classes can customize their pickling by providing methods named <code class="docutils literal notranslate">__getstate__</code>, <code class="docutils literal notranslate">__setstate__</code> and <code class="docutils literal notranslate">__getinitargs__</code>. Absent these, a default pickling strategy for classic class instances is implemented that works as long as all instance variables are picklable. This default strategy is documented in terms of default implementations of <code class="docutils literal notranslate">__getstate__</code> and <code class="docutils literal notranslate">__setstate__</code>. The primary ways to customize pickling of classic class instances is by specifying <code class="docutils literal notranslate">__getstate__</code> and/or <code class="docutils literal notranslate">__setstate__</code> methods. It is fine if a class implements one of these but not the other, as long as it is compatible with the default version. <section id="the-getstate-method"> <h4><a class="toc-backref" href="#the-getstate-method" role="doc-backlink">The <code class="docutils literal notranslate">__getstate__</code> method</a></h4> The <code class="docutils literal notranslate">__getstate__</code> method should return a picklable value representing the object’s state without referencing the object itself. If no <code class="docutils literal notranslate">__getstate__</code> method exists, a default implementation is used that returns <code class="docutils literal notranslate">self.__dict__</code>. </section> <section id="the-setstate-method"> <h4><a class="toc-backref" href="#the-setstate-method" role="doc-backlink">The <code class="docutils literal notranslate">__setstate__</code> method</a></h4> The <code class="docutils literal notranslate">__setstate__</code> method should take one argument; it will be called with the value returned by <code class="docutils literal notranslate">__getstate__</code> (or its default implementation). If no <code class="docutils literal notranslate">__setstate__</code> method exists, a default implementation is provided that assumes the state is a dictionary mapping instance variable names to values. The default implementation tries two things: <ul class="simple"> <li>First, it tries to call <code class="docutils literal notranslate">self.__dict__.update(state)</code>.</li> <li>If the <code class="docutils literal notranslate">update()</code> call fails with a <code class="docutils literal notranslate">RuntimeError</code> exception, it calls <code class="docutils literal notranslate">setattr(self, key, value)</code> for each <code class="docutils literal notranslate">(key, value)</code> pair in the state dictionary. This only happens when unpickling in restricted execution mode (see the <code class="docutils literal notranslate">rexec</code> standard library module).</li> </ul> </section> <section id="the-getinitargs-method"> <h4><a class="toc-backref" href="#the-getinitargs-method" role="doc-backlink">The <code class="docutils literal notranslate">__getinitargs__</code> method</a></h4> The <code class="docutils literal notranslate">__setstate__</code> method (or its default implementation) requires that a new object already exists so that its <code class="docutils literal notranslate">__setstate__</code> method can be called. The point is to create a new object that isn’t fully initialized; in particular, the class’s <code class="docutils literal notranslate">__init__</code> method should not be called if possible. These are the possibilities: <ul class="simple"> <li>Normally, the following trick is used: create an instance of a trivial classic class (one without any methods or instance variables) and then use <code class="docutils literal notranslate">__class__</code> assignment to change its class to the desired class. This creates an instance of the desired class with an empty <code class="docutils literal notranslate">__dict__</code> whose <code class="docutils literal notranslate">__init__</code> has not been called.</li> <li>However, if the class has a method named <code class="docutils literal notranslate">__getinitargs__</code>, the above trick is not used, and a class instance is created by using the tuple returned by <code class="docutils literal notranslate">__getinitargs__</code> as an argument list to the class constructor. This is done even if <code class="docutils literal notranslate">__getinitargs__</code> returns an empty tuple — a <code class="docutils literal notranslate">__getinitargs__</code> method that returns <code class="docutils literal notranslate">()</code> is not equivalent to not having <code class="docutils literal notranslate">__getinitargs__</code> at all. <code class="docutils literal notranslate">__getinitargs__</code> must return a tuple.</li> <li>In restricted execution mode, the trick from the first bullet doesn’t work; in this case, the class constructor is called with an empty argument list if no <code class="docutils literal notranslate">__getinitargs__</code> method exists. This means that in order for a classic class to be unpicklable in restricted execution mode, it must either implement <code class="docutils literal notranslate">__getinitargs__</code> or its constructor (i.e., its <code class="docutils literal notranslate">__init__</code> method) must be callable without arguments.</li> </ul> </section> </section> <section id="case-2-pickling-new-style-class-instances-using-protocols-0-or-1"> <h3><a class="toc-backref" href="#case-2-pickling-new-style-class-instances-using-protocols-0-or-1" role="doc-backlink">Case 2: pickling new-style class instances using protocols 0 or 1</a></h3> This case is unchanged from Python 2.2. For better pickling of new-style class instances when backwards compatibility is not an issue, protocol 2 should be used; see case 3 below. New-style classes, whether implemented in C or in Python, inherit a default <code class="docutils literal notranslate">__reduce__</code> implementation from the universal base class ‘object’. This default <code class="docutils literal notranslate">__reduce__</code> implementation is not used for those built-in types for which the <code class="docutils literal notranslate">pickle</code> module has built-in support. Here’s a full list of those types: <ul class="simple"> <li>Concrete built-in types: <code class="docutils literal notranslate">NoneType</code>, <code class="docutils literal notranslate">bool</code>, <code class="docutils literal notranslate">int</code>, <code class="docutils literal notranslate">float</code>, <code class="docutils literal notranslate">complex</code>, <code class="docutils literal notranslate">str</code>, <code class="docutils literal notranslate">unicode</code>, <code class="docutils literal notranslate">tuple</code>, <code class="docutils literal notranslate">list</code>, <code class="docutils literal notranslate">dict</code>. (Complex is supported by virtue of a <code class="docutils literal notranslate">__reduce__</code> implementation registered in <code class="docutils literal notranslate">copy_reg</code>.) In Jython, <code class="docutils literal notranslate">PyStringMap</code> is also included in this list.</li> <li>Classic instances.</li> <li>Classic class objects, Python function objects, built-in function and method objects, and new-style type objects (== new-style class objects). These are pickled by name, not by value: at unpickling time, a reference to an object with the same name (the fully qualified module name plus the variable name in that module) is substituted.</li> </ul> The default <code class="docutils literal notranslate">__reduce__</code> implementation will fail at pickling time for built-in types not mentioned above, and for new-style classes implemented in C: if they want to be picklable, they must supply a custom <code class="docutils literal notranslate">__reduce__</code> implementation under protocols 0 and 1. For new-style classes implemented in Python, the default <code class="docutils literal notranslate">__reduce__</code> implementation (<code class="docutils literal notranslate">copy_reg._reduce</code>) works as follows: Let <code class="docutils literal notranslate">D</code> be the class on the object to be pickled. First, find the nearest base class that is implemented in C (either as a built-in type or as a type defined by an extension class). Call this base class <code class="docutils literal notranslate">B</code>, and the class of the object to be pickled <code class="docutils literal notranslate">D</code>. Unless <code class="docutils literal notranslate">B</code> is the class ‘object’, instances of class <code class="docutils literal notranslate">B</code> must be picklable, either by having built-in support (as defined in the above three bullet points), or by having a non-default <code class="docutils literal notranslate">__reduce__</code> implementation. <code class="docutils literal notranslate">B</code> must not be the same class as <code class="docutils literal notranslate">D</code> (if it were, it would mean that <code class="docutils literal notranslate">D</code> is not implemented in Python). The callable produced by the default <code class="docutils literal notranslate">__reduce__</code> is <code class="docutils literal notranslate">copy_reg._reconstructor</code>, and its arguments tuple is <code class="docutils literal notranslate">(D, B, basestate)</code>, where <code class="docutils literal notranslate">basestate</code> is <code class="docutils literal notranslate">None</code> if <code class="docutils literal notranslate">B</code> is the builtin object class, and <code class="docutils literal notranslate">basestate</code> is <div class="highlight-default notranslate"><div class="highlight"><pre>basestate = B(obj) </pre></div> </div> if <code class="docutils literal notranslate">B</code> is not the builtin object class. This is geared toward pickling subclasses of builtin types, where, for example, <code class="docutils literal notranslate">list(some_list_subclass_instance)</code> produces “the list part” of the <code class="docutils literal notranslate">list</code> subclass instance. The object is recreated at unpickling time by <code class="docutils literal notranslate">copy_reg._reconstructor</code>, like so: <div class="highlight-default notranslate"><div class="highlight"><pre>obj = B.__new__(D, basestate) B.__init__(obj, basestate) </pre></div> </div> Objects using the default <code class="docutils literal notranslate">__reduce__</code> implementation can customize it by defining <code class="docutils literal notranslate">__getstate__</code> and/or <code class="docutils literal notranslate">__setstate__</code> methods. These work almost the same as described for classic classes above, except that if <code class="docutils literal notranslate">__getstate__</code> returns an object (of any type) whose value is considered false (e.g. <code class="docutils literal notranslate">None</code>, or a number that is zero, or an empty sequence or mapping), this state is not pickled and <code class="docutils literal notranslate">__setstate__</code> will not be called at all. If <code class="docutils literal notranslate">__getstate__</code> exists and returns a true value, that value becomes the third element of the tuple returned by the default <code class="docutils literal notranslate">__reduce__</code>, and at unpickling time the value is passed to <code class="docutils literal notranslate">__setstate__</code>. If <code class="docutils literal notranslate">__getstate__</code> does not exist, but <code class="docutils literal notranslate">obj.__dict__</code> exists, then <code class="docutils literal notranslate">obj.__dict__</code> becomes the third element of the tuple returned by <code class="docutils literal notranslate">__reduce__</code>, and again at unpickling time the value is passed to <code class="docutils literal notranslate">obj.__setstate__</code>. The default <code class="docutils literal notranslate">__setstate__</code> is the same as that for classic classes, described above. Note that this strategy ignores slots. Instances of new-style classes that have slots but no <code class="docutils literal notranslate">__getstate__</code> method cannot be pickled by protocols 0 and 1; the code explicitly checks for this condition. Note that pickling new-style class instances ignores <code class="docutils literal notranslate">__getinitargs__</code> if it exists (and under all protocols). <code class="docutils literal notranslate">__getinitargs__</code> is useful only for classic classes. </section> <section id="case-3-pickling-new-style-class-instances-using-protocol-2"> <h3><a class="toc-backref" href="#case-3-pickling-new-style-class-instances-using-protocol-2" role="doc-backlink">Case 3: pickling new-style class instances using protocol 2</a></h3> Under protocol 2, the default <code class="docutils literal notranslate">__reduce__</code> implementation inherited from the ‘object’ base class is ignored. Instead, a different default implementation is used, which allows more efficient pickling of new-style class instances than possible with protocols 0 or 1, at the cost of backward incompatibility with Python 2.2 (meaning no more than that a protocol 2 pickle cannot be unpickled before Python 2.3). The customization uses three special methods: <code class="docutils literal notranslate">__getstate__</code>, <code class="docutils literal notranslate">__setstate__</code> and <code class="docutils literal notranslate">__getnewargs__</code> (note that <code class="docutils literal notranslate">__getinitargs__</code> is again ignored). It is fine if a class implements one or more but not all of these, as long as it is compatible with the default implementations. <section id="id1"> <h4><a class="toc-backref" href="#id1" role="doc-backlink">The <code class="docutils literal notranslate">__getstate__</code> method</a></h4> The <code class="docutils literal notranslate">__getstate__</code> method should return a picklable value representing the object’s state without referencing the object itself. If no <code class="docutils literal notranslate">__getstate__</code> method exists, a default implementation is used which is described below. There’s a subtle difference between classic and new-style classes here: if a classic class’s <code class="docutils literal notranslate">__getstate__</code> returns <code class="docutils literal notranslate">None</code>, <code class="docutils literal notranslate">self.__setstate__(None)</code> will be called as part of unpickling. But if a new-style class’s <code class="docutils literal notranslate">__getstate__</code> returns <code class="docutils literal notranslate">None</code>, its <code class="docutils literal notranslate">__setstate__</code> won’t be called at all as part of unpickling. If no <code class="docutils literal notranslate">__getstate__</code> method exists, a default state is computed. There are several cases: <ul class="simple"> <li>For a new-style class that has no instance <code class="docutils literal notranslate">__dict__</code> and no <code class="docutils literal notranslate">__slots__</code>, the default state is <code class="docutils literal notranslate">None</code>.</li> <li>For a new-style class that has an instance <code class="docutils literal notranslate">__dict__</code> and no <code class="docutils literal notranslate">__slots__</code>, the default state is <code class="docutils literal notranslate">self.__dict__</code>.</li> <li>For a new-style class that has an instance <code class="docutils literal notranslate">__dict__</code> and <code class="docutils literal notranslate">__slots__</code>, the default state is a tuple consisting of two dictionaries: <code class="docutils literal notranslate">self.__dict__</code>, and a dictionary mapping slot names to slot values. Only slots that have a value are included in the latter.</li> <li>For a new-style class that has <code class="docutils literal notranslate">__slots__</code> and no instance <code class="docutils literal notranslate">__dict__</code>, the default state is a tuple whose first item is <code class="docutils literal notranslate">None</code> and whose second item is a dictionary mapping slot names to slot values described in the previous bullet.</li> </ul> </section> <section id="id2"> <h4><a class="toc-backref" href="#id2" role="doc-backlink">The <code class="docutils literal notranslate">__setstate__</code> method</a></h4> The <code class="docutils literal notranslate">__setstate__</code> method should take one argument; it will be called with the value returned by <code class="docutils literal notranslate">__getstate__</code> or with the default state described above if no <code class="docutils literal notranslate">__getstate__</code> method is defined. If no <code class="docutils literal notranslate">__setstate__</code> method exists, a default implementation is provided that can handle the state returned by the default <code class="docutils literal notranslate">__getstate__</code>, described above. </section> <section id="the-getnewargs-method"> <h4><a class="toc-backref" href="#the-getnewargs-method" role="doc-backlink">The <code class="docutils literal notranslate">__getnewargs__</code> method</a></h4> Like for classic classes, the <code class="docutils literal notranslate">__setstate__</code> method (or its default implementation) requires that a new object already exists so that its <code class="docutils literal notranslate">__setstate__</code> method can be called. In protocol 2, a new pickling opcode is used that causes a new object to be created as follows: <div class="highlight-default notranslate"><div class="highlight"><pre>obj = C.__new__(C, *args) </pre></div> </div> where <code class="docutils literal notranslate">C</code> is the class of the pickled object, and <code class="docutils literal notranslate">args</code> is either the empty tuple, or the tuple returned by the <code class="docutils literal notranslate">__getnewargs__</code> method, if defined. <code class="docutils literal notranslate">__getnewargs__</code> must return a tuple. The absence of a <code class="docutils literal notranslate">__getnewargs__</code> method is equivalent to the existence of one that returns <code class="docutils literal notranslate">()</code>. </section> </section> </section> <section id="the-newobj-unpickling-function"> <h2><a class="toc-backref" href="#the-newobj-unpickling-function" role="doc-backlink">The <code class="docutils literal notranslate">__newobj__</code> unpickling function</a></h2> When the unpickling function returned by <code class="docutils literal notranslate">__reduce__</code> (the first item of the returned tuple) has the name <code class="docutils literal notranslate">__newobj__</code>, something special happens for pickle protocol 2. An unpickling function named <code class="docutils literal notranslate">__newobj__</code> is assumed to have the following semantics: <div class="highlight-default notranslate"><div class="highlight"><pre>def __newobj__(cls, *args): return cls.__new__(cls, *args) </pre></div> </div> Pickle protocol 2 special-cases an unpickling function with this name, and emits a pickling opcode that, given ‘cls’ and ‘args’, will return <code class="docutils literal notranslate">cls.__new__(cls, *args)</code> without also pickling a reference to <code class="docutils literal notranslate">__newobj__</code> (this is the same pickling opcode used by protocol 2 for a new-style class instance when no <code class="docutils literal notranslate">__reduce__</code> implementation exists). This is the main reason why protocol 2 pickles are much smaller than classic pickles. Of course, the pickling code cannot verify that a function named <code class="docutils literal notranslate">__newobj__</code> actually has the expected semantics. If you use an unpickling function named <code class="docutils literal notranslate">__newobj__</code> that returns something different, you deserve what you get. It is safe to use this feature under Python 2.2; there’s nothing in the recommended implementation of <code class="docutils literal notranslate">__newobj__</code> that depends on Python 2.3. </section> <section id="the-extension-registry"> <h2><a class="toc-backref" href="#the-extension-registry" role="doc-backlink">The extension registry</a></h2> Protocol 2 supports a new mechanism to reduce the size of pickles. When class instances (classic or new-style) are pickled, the full name of the class (module name including package name, and class name) is included in the pickle. Especially for applications that generate many small pickles, this is a lot of overhead that has to be repeated in each pickle. For large pickles, when using protocol 1, repeated references to the same class name are compressed using the “memo” feature; but each class name must be spelled in full at least once per pickle, and this causes a lot of overhead for small pickles. The extension registry allows one to represent the most frequently used names by small integers, which are pickled very efficiently: an extension code in the range 1–255 requires only two bytes including the opcode, one in the range 256–65535 requires only three bytes including the opcode. One of the design goals of the pickle protocol is to make pickles “context-free”: as long as you have installed the modules containing the classes referenced by a pickle, you can unpickle it, without needing to import any of those classes ahead of time. Unbridled use of extension codes could jeopardize this desirable property of pickles. Therefore, the main use of extension codes is reserved for a set of codes to be standardized by some standard-setting body. This being Python, the standard-setting body is the PSF. From time to time, the PSF will decide on a table mapping extension codes to class names (or occasionally names of other global objects; functions are also eligible). This table will be incorporated in the next Python release(s). However, for some applications, like Zope, context-free pickles are not a requirement, and waiting for the PSF to standardize some codes may not be practical. Two solutions are offered for such applications. First, a few ranges of extension codes are reserved for private use. Any application can register codes in these ranges. Two applications exchanging pickles using codes in these ranges need to have some out-of-band mechanism to agree on the mapping between extension codes and names. Second, some large Python projects (e.g. Zope) can be assigned a range of extension codes outside the “private use” range that they can assign as they see fit. The extension registry is defined as a mapping between extension codes and names. When an extension code is unpickled, it ends up producing an object, but this object is gotten by interpreting the name as a module name followed by a class (or function) name. The mapping from names to objects is cached. It is quite possible that certain names cannot be imported; that should not be a problem as long as no pickle containing a reference to such names has to be unpickled. (The same issue already exists for direct references to such names in pickles that use protocols 0 or 1.) Here is the proposed initial assignment of extension code ranges: <table class="docutils align-default"> <thead> <tr class="row-odd"><th class="head">First</th> <th class="head">Last</th> <th class="head">Count</th> <th class="head">Purpose</th> </tr> </thead> <tbody> <tr class="row-even"><td>0</td> <td>0</td> <td>1</td> <td>Reserved — will never be used</td> </tr> <tr class="row-odd"><td>1</td> <td>127</td> <td>127</td> <td>Reserved for Python standard library</td> </tr> <tr class="row-even"><td>128</td> <td>191</td> <td>64</td> <td>Reserved for Zope</td> </tr> <tr class="row-odd"><td>192</td> <td>239</td> <td>48</td> <td>Reserved for 3rd parties</td> </tr> <tr class="row-even"><td>240</td> <td>255</td> <td>16</td> <td>Reserved for private use (will never be assigned)</td> </tr> <tr class="row-odd"><td>256</td> <td>MAX</td> <td>MAX</td> <td>Reserved for future assignment</td> </tr> </tbody> </table> MAX stands for 2147483647, or <code class="docutils literal notranslate">2**31-1</code>. This is a hard limitation of the protocol as currently defined. At the moment, no specific extension codes have been assigned yet. <section id="extension-registry-api"> <h3><a class="toc-backref" href="#extension-registry-api" role="doc-backlink">Extension registry API</a></h3> The extension registry is maintained as private global variables in the <code class="docutils literal notranslate">copy_reg</code> module. The following three functions are defined in this module to manipulate the registry: <dl class="simple"> <dt><code class="docutils literal notranslate">add_extension(module, name, code)</code></dt><dd>Register an extension code. The module and name arguments must be strings; code must be an <code class="docutils literal notranslate">int</code> in the inclusive range 1 through MAX. This must either register a new <code class="docutils literal notranslate">(module, name)</code> pair to a new code, or be a redundant repeat of a previous call that was not canceled by a <code class="docutils literal notranslate">remove_extension()</code> call; a <code class="docutils literal notranslate">(module, name)</code> pair may not be mapped to more than one code, nor may a code be mapped to more than one <code class="docutils literal notranslate">(module, name)</code> pair.</dd> <dt><code class="docutils literal notranslate">remove_extension(module, name, code)</code></dt><dd>Arguments are as for <code class="docutils literal notranslate">add_extension()</code>. Remove a previously registered mapping between <code class="docutils literal notranslate">(module, name)</code> and code.</dd> <dt><code class="docutils literal notranslate">clear_extension_cache()</code></dt><dd>The implementation of extension codes may use a cache to speed up loading objects that are named frequently. This cache can be emptied (removing references to cached objects) by calling this method.</dd> </dl> Note that the API does not enforce the standard range assignments. It is up to applications to respect these. </section> </section> <section id="the-copy-module"> <h2><a class="toc-backref" href="#the-copy-module" role="doc-backlink">The copy module</a></h2> Traditionally, the <code class="docutils literal notranslate">copy</code> module has supported an extended subset of the pickling APIs for customizing the <code class="docutils literal notranslate">copy()</code> and <code class="docutils literal notranslate">deepcopy()</code> operations. In particular, besides checking for a <code class="docutils literal notranslate">__copy__</code> or <code class="docutils literal notranslate">__deepcopy__</code> method, <code class="docutils literal notranslate">copy()</code> and <code class="docutils literal notranslate">deepcopy()</code> have always looked for <code class="docutils literal notranslate">__reduce__</code>, and for classic classes, have looked for <code class="docutils literal notranslate">__getinitargs__</code>, <code class="docutils literal notranslate">__getstate__</code> and <code class="docutils literal notranslate">__setstate__</code>. In Python 2.2, the default <code class="docutils literal notranslate">__reduce__</code> inherited from ‘object’ made copying simple new-style classes possible, but slots and various other special cases were not covered. In Python 2.3, several changes are made to the <code class="docutils literal notranslate">copy</code> module: <ul class="simple"> <li><code class="docutils literal notranslate">__reduce_ex__</code> is supported (and always called with 2 as the protocol version argument).</li> <li>The four- and five-argument return values of <code class="docutils literal notranslate">__reduce__</code> are supported.</li> <li>Before looking for a <code class="docutils literal notranslate">__reduce__</code> method, the <code class="docutils literal notranslate">copy_reg.dispatch_table</code> is consulted, just like for pickling.</li> <li>When the <code class="docutils literal notranslate">__reduce__</code> method is inherited from object, it is (unconditionally) replaced by a better one that uses the same APIs as pickle protocol 2: <code class="docutils literal notranslate">__getnewargs__</code>, <code class="docutils literal notranslate">__getstate__</code>, and <code class="docutils literal notranslate">__setstate__</code>, handling <code class="docutils literal notranslate">list</code> and <code class="docutils literal notranslate">dict</code> subclasses, and handling slots.</li> </ul> As a consequence of the latter change, certain new-style classes that were copyable under Python 2.2 are not copyable under Python 2.3. (These classes are also not picklable using pickle protocol 2.) A minimal example of such a class: <div class="highlight-default notranslate"><div class="highlight"><pre>class C(object): def __new__(cls, a): return object.__new__(cls) </pre></div> </div> The problem only occurs when <code class="docutils literal notranslate">__new__</code> is overridden and has at least one mandatory argument in addition to the class argument. To fix this, a <code class="docutils literal notranslate">__getnewargs__</code> method should be added that returns the appropriate argument tuple (excluding the class). </section> <section id="pickling-python-longs"> <h2><a class="toc-backref" href="#pickling-python-longs" role="doc-backlink">Pickling Python longs</a></h2> Pickling and unpickling Python longs takes time quadratic in the number of digits, in protocols 0 and 1. Under protocol 2, new opcodes support linear-time pickling and unpickling of longs. </section> <section id="pickling-bools"> <h2><a class="toc-backref" href="#pickling-bools" role="doc-backlink">Pickling bools</a></h2> Protocol 2 introduces new opcodes for pickling <code class="docutils literal notranslate">True</code> and <code class="docutils literal notranslate">False</code> directly. Under protocols 0 and 1, bools are pickled as integers, using a trick in the representation of the integer in the pickle so that an unpickler can recognize that a bool was intended. That trick consumed 4 bytes per bool pickled. The new bool opcodes consume 1 byte per bool. </section> <section id="pickling-small-tuples"> <h2><a class="toc-backref" href="#pickling-small-tuples" role="doc-backlink">Pickling small tuples</a></h2> Protocol 2 introduces new opcodes for more-compact pickling of tuples of lengths 1, 2 and 3. Protocol 1 previously introduced an opcode for more-compact pickling of empty tuples. </section> <section id="protocol-identification"> <h2><a class="toc-backref" href="#protocol-identification" role="doc-backlink">Protocol identification</a></h2> Protocol 2 introduces a new opcode, with which all protocol 2 pickles begin, identifying that the pickle is protocol 2. Attempting to unpickle a protocol 2 pickle under older versions of Python will therefore raise an “unknown opcode” exception immediately. </section> <section id="pickling-of-large-lists-and-dicts"> <h2><a class="toc-backref" href="#pickling-of-large-lists-and-dicts" role="doc-backlink">Pickling of large lists and dicts</a></h2> Protocol 1 pickles large lists and dicts “in one piece”, which minimizes pickle size, but requires that unpickling create a temp object as large as the object being unpickled. Part of the protocol 2 changes break large lists and dicts into pieces of no more than 1000 elements each, so that unpickling needn’t create a temp object larger than needed to hold 1000 elements. This isn’t part of protocol 2, however: the opcodes produced are still part of protocol 1. <code class="docutils literal notranslate">__reduce__</code> implementations that return the optional new listitems or dictitems iterators also benefit from this unpickling temp-space optimization. </section> <section id="copyright"> <h2><a class="toc-backref" href="#copyright" role="doc-backlink">Copyright</a></h2> This document has been placed in the public domain. </section> </section> <hr class="docutils" /> Source: <a class="reference external" href="https://github.com/python/peps/blob/main/peps/pep-0307.rst">https://github.com/python/peps/blob/main/peps/pep-0307.rst</a> Last modified: <a class="reference external" href="https://github.com/python/peps/commits/main/peps/pep-0307.rst">2025-02-01 08:59:27 GMT</a> </article> <nav id="pep-sidebar"> <h2>Contents</h2> <ul> <li><a class="reference internal" href="#introduction">Introduction</a></li> <li><a class="reference internal" href="#motivation">Motivation</a></li> <li><a class="reference internal" href="#protocol-versions">Protocol versions</a></li> <li><a class="reference internal" href="#security-issues">Security issues</a></li> <li><a class="reference internal" href="#extended-reduce-api">Extended <code class="docutils literal notranslate">__reduce__</code> API</a></li> <li><a class="reference internal" href="#the-reduce-ex-api">The <code class="docutils literal notranslate">__reduce_ex__</code> API</a></li> <li><a class="reference internal" href="#customizing-pickling-absent-a-reduce-implementation">Customizing pickling absent a <code class="docutils literal notranslate">__reduce__</code> implementation</a><ul> <li><a class="reference internal" href="#case-1-pickling-classic-class-instances">Case 1: pickling classic class instances</a><ul> <li><a class="reference internal" href="#the-getstate-method">The <code class="docutils literal notranslate">__getstate__</code> method</a></li> <li><a class="reference internal" href="#the-setstate-method">The <code class="docutils literal notranslate">__setstate__</code> method</a></li> <li><a class="reference internal" href="#the-getinitargs-method">The <code class="docutils literal notranslate">__getinitargs__</code> method</a></li> </ul> </li> <li><a class="reference internal" href="#case-2-pickling-new-style-class-instances-using-protocols-0-or-1">Case 2: pickling new-style class instances using protocols 0 or 1</a></li> <li><a class="reference internal" href="#case-3-pickling-new-style-class-instances-using-protocol-2">Case 3: pickling new-style class instances using protocol 2</a><ul> <li><a class="reference internal" href="#id1">The <code class="docutils literal notranslate">__getstate__</code> method</a></li> <li><a class="reference internal" href="#id2">The <code class="docutils literal notranslate">__setstate__</code> method</a></li> <li><a class="reference internal" href="#the-getnewargs-method">The <code class="docutils literal notranslate">__getnewargs__</code> method</a></li> </ul> </li> </ul> </li> <li><a class="reference internal" href="#the-newobj-unpickling-function">The <code class="docutils literal notranslate">__newobj__</code> unpickling function</a></li> <li><a class="reference internal" href="#the-extension-registry">The extension registry</a><ul> <li><a class="reference internal" href="#extension-registry-api">Extension registry API</a></li> </ul> </li> <li><a class="reference internal" href="#the-copy-module">The copy module</a></li> <li><a class="reference internal" href="#pickling-python-longs">Pickling Python longs</a></li> <li><a class="reference internal" href="#pickling-bools">Pickling bools</a></li> <li><a class="reference internal" href="#pickling-small-tuples">Pickling small tuples</a></li> <li><a class="reference internal" href="#protocol-identification">Protocol identification</a></li> <li><a class="reference internal" href="#pickling-of-large-lists-and-dicts">Pickling of large lists and dicts</a></li> <li><a class="reference internal" href="#copyright">Copyright</a></li> </ul> <a id="source" href="https://github.com/python/peps/blob/main/peps/pep-0307.rst">Page Source (GitHub)</a> </nav> </section> <script src="../_static/colour_scheme.js"></script> <script src="../_static/wrap_tables.js"></script> <script src="../_static/sticky_banner.js"></script> </body> </html>