CINXE.COM

<!DOCTYPE html> <html lang="en"> <head> <meta charset="utf-8"> <meta name="viewport" content="width=device-width, initial-scale=1.0"> <meta name="color-scheme" content="light dark"> <title>PEP 364 – Transitioning to the Py3K Standard Library | peps.python.org</title> <link rel="shortcut icon" href="../_static/py.png"> <link rel="canonical" href="https://peps.python.org/pep-0364/"> <link rel="stylesheet" href="../_static/style.css" type="text/css"> <link rel="stylesheet" href="../_static/mq.css" type="text/css"> <link rel="stylesheet" href="../_static/pygments.css" type="text/css" media="(prefers-color-scheme: light)" id="pyg-light"> <link rel="stylesheet" href="../_static/pygments_dark.css" type="text/css" media="(prefers-color-scheme: dark)" id="pyg-dark"> <link rel="alternate" type="application/rss+xml" title="Latest PEPs" href="https://peps.python.org/peps.rss"> <meta property="og:title" content='PEP 364 – Transitioning to the Py3K Standard Library | peps.python.org'> <meta property="og:description" content="PEP 3108 describes the reorganization of the Python standard library for the Python 3.0 release. This PEP describes a mechanism for transitioning from the Python 2.x standard library to the Python 3.0 standard library. This transition will allow and e..."> <meta property="og:type" content="website"> <meta property="og:url" content="https://peps.python.org/pep-0364/"> <meta property="og:site_name" content="Python Enhancement Proposals (PEPs)"> <meta property="og:image" content="https://peps.python.org/_static/og-image.png"> <meta property="og:image:alt" content="Python PEPs"> <meta property="og:image:width" content="200"> <meta property="og:image:height" content="200"> <meta name="description" content="PEP 3108 describes the reorganization of the Python standard library for the Python 3.0 release. This PEP describes a mechanism for transitioning from the Python 2.x standard library to the Python 3.0 standard library. This transition will allow and e..."> <meta name="theme-color" content="#3776ab"> </head> <body> <svg xmlns="http://www.w3.org/2000/svg" style="display: none;"> <symbol id="svg-sun-half" viewBox="0 0 24 24" pointer-events="all"> <title>Following system colour scheme</title> <svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"> <circle cx="12" cy="12" r="9"></circle> <path d="M12 3v18m0-12l4.65-4.65M12 14.3l7.37-7.37M12 19.6l8.85-8.85"></path> </svg> </symbol> <symbol id="svg-moon" viewBox="0 0 24 24" pointer-events="all"> <title>Selected dark colour scheme</title> <svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"> <path stroke="none" d="M0 0h24v24H0z" fill="none"></path> <path d="M12 3c.132 0 .263 0 .393 0a7.5 7.5 0 0 0 7.92 12.446a9 9 0 1 1 -8.313 -12.454z"></path> </svg> </symbol> <symbol id="svg-sun" viewBox="0 0 24 24" pointer-events="all"> <title>Selected light colour scheme</title> <svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"> <circle cx="12" cy="12" r="5"></circle> <line x1="12" y1="1" x2="12" y2="3"></line> <line x1="12" y1="21" x2="12" y2="23"></line> <line x1="4.22" y1="4.22" x2="5.64" y2="5.64"></line> <line x1="18.36" y1="18.36" x2="19.78" y2="19.78"></line> <line x1="1" y1="12" x2="3" y2="12"></line> <line x1="21" y1="12" x2="23" y2="12"></line> <line x1="4.22" y1="19.78" x2="5.64" y2="18.36"></line> <line x1="18.36" y1="5.64" x2="19.78" y2="4.22"></line> </svg> </symbol> </svg> <script> document.documentElement.dataset.colour_scheme = localStorage.getItem("colour_scheme") || "auto" </script> <section id="pep-page-section"> <header> <h1>Python Enhancement Proposals</h1> <ul class="breadcrumbs"> <li><a href="https://www.python.org/" title="The Python Programming Language">Python</a> » </li> <li><a href="../pep-0000/">PEP Index</a> » </li> <li>PEP 364</li> </ul> <button id="colour-scheme-cycler" onClick="setColourScheme(nextColourScheme())"> <svg aria-hidden="true" class="colour-scheme-icon-when-auto"><use href="#svg-sun-half"></use></svg> <svg aria-hidden="true" class="colour-scheme-icon-when-dark"><use href="#svg-moon"></use></svg> <svg aria-hidden="true" class="colour-scheme-icon-when-light"><use href="#svg-sun"></use></svg> <span class="visually-hidden">Toggle light / dark / auto colour theme</span> </button> </header> <article> <section id="pep-content"> <h1 class="page-title">PEP 364 – Transitioning to the Py3K Standard Library</h1> <dl class="rfc2822 field-list simple"> <dt class="field-odd">Author<span class="colon">:</span></dt> <dd class="field-odd">Barry Warsaw <barry at python.org></dd> <dt class="field-even">Status<span class="colon">:</span></dt> <dd class="field-even"><abbr title="Removed from consideration by sponsor or authors">Withdrawn</abbr></dd> <dt class="field-odd">Type<span class="colon">:</span></dt> <dd class="field-odd"><abbr title="Normative PEP with a new feature for Python, implementation change for CPython or interoperability standard for the ecosystem">Standards Track</abbr></dd> <dt class="field-even">Created<span class="colon">:</span></dt> <dd class="field-even">01-Mar-2007</dd> <dt class="field-odd">Python-Version<span class="colon">:</span></dt> <dd class="field-odd">2.6</dd> <dt class="field-even">Post-History<span class="colon">:</span></dt> <dd class="field-even"><p></p></dd> </dl> <hr class="docutils" /> <section id="contents"> <details><summary>Table of Contents</summary><ul class="simple"> <li><a class="reference internal" href="#abstract">Abstract</a></li> <li><a class="reference internal" href="#rationale">Rationale</a></li> <li><a class="reference internal" href="#supported-renamings">Supported Renamings</a></li> <li><a class="reference internal" href="#mv-files">.mv files</a></li> <li><a class="reference internal" href="#implementation-specification">Implementation Specification</a></li> <li><a class="reference internal" href="#programmatic-interface">Programmatic Interface</a></li> <li><a class="reference internal" href="#open-issues">Open Issues</a></li> <li><a class="reference internal" href="#reference-implementation">Reference Implementation</a></li> <li><a class="reference internal" href="#references">References</a></li> <li><a class="reference internal" href="#copyright">Copyright</a></li> </ul> </details></section> <section id="abstract"> <h2><a class="toc-backref" href="#abstract" role="doc-backlink">Abstract</a></h2> <p><a class="pep reference internal" href="../pep-3108/" title="PEP 3108 – Standard Library Reorganization">PEP 3108</a> describes the reorganization of the Python standard library for the Python 3.0 release. This PEP describes a mechanism for transitioning from the Python 2.x standard library to the Python 3.0 standard library. This transition will allow and encourage Python programmers to use the new Python 3.0 library names starting with Python 2.6, while maintaining the old names for backward compatibility. In this way, a Python programmer will be able to write forward compatible code without sacrificing interoperability with existing Python programs.</p> </section> <section id="rationale"> <h2><a class="toc-backref" href="#rationale" role="doc-backlink">Rationale</a></h2> <p><a class="pep reference internal" href="../pep-3108/" title="PEP 3108 – Standard Library Reorganization">PEP 3108</a> presents a rationale for Python standard library (stdlib) reorganization. The reader is encouraged to consult that PEP for details about why and how the library will be reorganized. Should <a class="pep reference internal" href="../pep-3108/" title="PEP 3108 – Standard Library Reorganization">PEP 3108</a> be accepted in part or in whole, then it is advantageous to allow Python programmers to begin the transition to the new stdlib module names in Python 2.x, so that they can write forward compatible code starting with Python 2.6.</p> <p>Note that <a class="pep reference internal" href="../pep-3108/" title="PEP 3108 – Standard Library Reorganization">PEP 3108</a> proposes to remove some “silly old stuff”, i.e. modules that are no longer useful or necessary. The PEP you are reading does not address this because there are no forward compatibility issues for modules that are to be removed, except to stop using such modules.</p> <p>This PEP concerns only the mechanism by which mappings from old stdlib names to new stdlib names are maintained. Please consult <a class="pep reference internal" href="../pep-3108/" title="PEP 3108 – Standard Library Reorganization">PEP 3108</a> for all specific module renaming proposals. Specifically see the section titled <code class="docutils literal notranslate"><span class="pre">Modules</span> <span class="pre">to</span> <span class="pre">Rename</span></code> for guidelines on the old name to new name mappings. The few examples in this PEP are given for illustrative purposes only and should not be used for specific renaming recommendations.</p> </section> <section id="supported-renamings"> <h2><a class="toc-backref" href="#supported-renamings" role="doc-backlink">Supported Renamings</a></h2> <p>There are at least 4 use cases explicitly supported by this PEP:</p> <ul class="simple"> <li>Simple top-level package name renamings, such as <code class="docutils literal notranslate"><span class="pre">StringIO</span></code> to <code class="docutils literal notranslate"><span class="pre">stringio</span></code>;</li> <li>Sub-package renamings where the package name may or may not be renamed, such as <code class="docutils literal notranslate"><span class="pre">email.MIMEText</span></code> to <code class="docutils literal notranslate"><span class="pre">email.mime.text</span></code>;</li> <li>Extension module renaming, such as <code class="docutils literal notranslate"><span class="pre">cStringIO</span></code> to <code class="docutils literal notranslate"><span class="pre">cstringio</span></code>;</li> <li>Third party renaming of any of the above.</li> </ul> <p>Two use cases supported by this PEP include renaming simple top-level modules, such as <code class="docutils literal notranslate"><span class="pre">StringIO</span></code>, as well as modules within packages, such as <code class="docutils literal notranslate"><span class="pre">email.MIMEText</span></code>.</p> <p>In the former case, <a class="pep reference internal" href="../pep-3108/" title="PEP 3108 – Standard Library Reorganization">PEP 3108</a> currently recommends <code class="docutils literal notranslate"><span class="pre">StringIO</span></code> be renamed to <code class="docutils literal notranslate"><span class="pre">stringio</span></code>, following <a class="pep reference internal" href="../pep-0008/" title="PEP 8 – Style Guide for Python Code">PEP 8</a> recommendations.</p> <p>In the latter case, the email 4.0 package distributed with Python 2.5 already renamed <code class="docutils literal notranslate"><span class="pre">email.MIMEText</span></code> to <code class="docutils literal notranslate"><span class="pre">email.mime.text</span></code>, although it did so in a one-off, uniquely hackish way inside the email package. The mechanism described in this PEP is general enough to handle all module renamings, obviating the need for the Python 2.5 hack (except for backward compatibility with earlier Python versions).</p> <p>An additional use case is to support the renaming of C extension modules. As long as the new name for the C module is importable, it can be remapped to the new name. E.g. <code class="docutils literal notranslate"><span class="pre">cStringIO</span></code> renamed to <code class="docutils literal notranslate"><span class="pre">cstringio</span></code>.</p> <p>Third party package renaming is also supported, via several public interfaces accessible by any Python module.</p> <p>Remappings are not performed recursively.</p> </section> <section id="mv-files"> <h2><a class="toc-backref" href="#mv-files" role="doc-backlink">.mv files</a></h2> <p>Remapping files are called <code class="docutils literal notranslate"><span class="pre">.mv</span></code> files; the suffix was chosen to be evocative of the Unix mv(1) command. An <code class="docutils literal notranslate"><span class="pre">.mv</span></code> file is a simple line-oriented text file. All blank lines and lines that start with a # are ignored. All other lines must contain two whitespace separated fields. The first field is the old module name, and the second field is the new module name. Both module names must be specified using their full dotted-path names. Here is an example <code class="docutils literal notranslate"><span class="pre">.mv</span></code> file from Python 2.6:</p> <div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="c1"># Map the various string i/o libraries to their new names</span> <span class="n">StringIO</span> <span class="n">stringio</span> <span class="n">cStringIO</span> <span class="n">cstringio</span> </pre></div> </div> <p><code class="docutils literal notranslate"><span class="pre">.mv</span></code> files can appear anywhere in the file system, and there is a programmatic interface provided to parse them, and register the remappings inside them. By default, when Python starts up, all the <code class="docutils literal notranslate"><span class="pre">.mv</span></code> files in the <code class="docutils literal notranslate"><span class="pre">oldlib</span></code> package are read, and their remappings are automatically registered. This is where all the module remappings should be specified for top-level Python 2.x standard library modules.</p> </section> <section id="implementation-specification"> <h2><a class="toc-backref" href="#implementation-specification" role="doc-backlink">Implementation Specification</a></h2> <p>This section provides the full specification for how module renamings in Python 2.x are implemented. The central mechanism relies on various import hooks as described in <a class="pep reference internal" href="../pep-0302/" title="PEP 302 – New Import Hooks">PEP 302</a>. Specifically <code class="docutils literal notranslate"><span class="pre">sys.path_importer_cache</span></code>, <code class="docutils literal notranslate"><span class="pre">sys.path</span></code>, and <code class="docutils literal notranslate"><span class="pre">sys.meta_path</span></code> are all employed to provide the necessary functionality.</p> <p>When Python’s import machinery is initialized, the oldlib package is imported. Inside oldlib there is a class called <code class="docutils literal notranslate"><span class="pre">OldStdlibLoader</span></code>. This class implements the <a class="pep reference internal" href="../pep-0302/" title="PEP 302 – New Import Hooks">PEP 302</a> interface and is automatically instantiated, with zero arguments. The constructor reads all the <code class="docutils literal notranslate"><span class="pre">.mv</span></code> files from the oldlib package directory, automatically registering all the remappings found in those <code class="docutils literal notranslate"><span class="pre">.mv</span></code> files. This is how the Python 2.x standard library is remapped.</p> <p>The OldStdlibLoader class should not be instantiated by other Python modules. Instead, you can access the global OldStdlibLoader instance via the <code class="docutils literal notranslate"><span class="pre">sys.stdlib_remapper</span></code> instance. Use this instance if you want programmatic access to the remapping machinery.</p> <p>One important implementation detail: as needed by the <a class="pep reference internal" href="../pep-0302/" title="PEP 302 – New Import Hooks">PEP 302</a> API, a magic string is added to sys.path, and module __path__ attributes in order to hook in our remapping loader. This magic string is currently <code class="docutils literal notranslate"><span class="pre"><oldlib></span></code> and some changes were necessary to Python’s site.py file in order to treat all sys.path entries starting with <code class="docutils literal notranslate"><span class="pre"><</span></code> as special. Specifically, no attempt is made to make them absolute file names (since they aren’t file names at all).</p> <p>In order for the remapping import hooks to work, the module or package must be physically located under its new name. This is because the import hooks catch only modules that are not already imported, and cannot be imported by Python’s built-in import rules. Thus, if a module has been moved, say from Lib/StringIO.py to Lib/stringio.py, and the former’s <code class="docutils literal notranslate"><span class="pre">.pyc</span></code> file has been removed, then without the remapper, this would fail:</p> <div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="kn">import</span> <span class="nn">StringIO</span> </pre></div> </div> <p>Instead, with the remapper, this failing import will be caught, the old name will be looked up in the registered remappings, and in this case, the new name <code class="docutils literal notranslate"><span class="pre">stringio</span></code> will be found. The remapper then attempts to import the new name, and if that succeeds, it binds the resulting module into sys.modules, under both the old and new names. Thus, the above import will result in entries in sys.modules for ‘StringIO’ and ‘stringio’, and both will point to the exact same module object.</p> <p>Note that no way to disable the remapping machinery is proposed, short of moving all the <code class="docutils literal notranslate"><span class="pre">.mv</span></code> files away or programmatically removing them in some custom start up code. In Python 3.0, the remappings will be eliminated, leaving only the “new” names.</p> </section> <section id="programmatic-interface"> <h2><a class="toc-backref" href="#programmatic-interface" role="doc-backlink">Programmatic Interface</a></h2> <p>Several methods are added to the <code class="docutils literal notranslate"><span class="pre">sys.stdlib_remapper</span></code> object, which third party packages can use to register their own remappings. Note however that in all cases, there is one and only one mapping from an old name to a new name. If two <code class="docutils literal notranslate"><span class="pre">.mv</span></code> files contain different mappings for an old name, or if a programmatic call is made with an old name that is already remapped, the previous mapping is lost. This will not affect any already imported modules.</p> <p>The following methods are available on the <code class="docutils literal notranslate"><span class="pre">sys.stdlib_remapper</span></code> object:</p> <ul class="simple"> <li><code class="docutils literal notranslate"><span class="pre">read_mv_file(filename)</span></code> – Read the given file and register all remappings found in the file.</li> <li><code class="docutils literal notranslate"><span class="pre">read_directory_mv_files(dirname,</span> <span class="pre">suffix='.mv')</span></code> – List the given directory, reading all files in that directory that have the matching suffix (<code class="docutils literal notranslate"><span class="pre">.mv</span></code> by default). For each parsed file, register all the remappings found in that file.</li> <li><code class="docutils literal notranslate"><span class="pre">set_mapping(oldname,</span> <span class="pre">newname)</span></code> – Register a new mapping from an old module name to a new module name. Both must be the full dotted-path name to the module. newname may be <code class="docutils literal notranslate"><span class="pre">None</span></code> in which case any existing mapping for oldname will be removed (it is not an error if there is no existing mapping).</li> <li><code class="docutils literal notranslate"><span class="pre">get_mapping(oldname,</span> <span class="pre">default=None)</span></code> – Return any registered newname for the given oldname. If there is no registered remapping, default is returned.</li> </ul> </section> <section id="open-issues"> <h2><a class="toc-backref" href="#open-issues" role="doc-backlink">Open Issues</a></h2> <ul> <li>Should there be a command line switch and/or environment variable to disable all remappings?</li> <li>Should remappings occur recursively?</li> <li>Should we automatically parse package directories for .mv files when the package’s __init__.py is loaded? This would allow packages to easily include .mv files for their own remappings. Compare what the email package currently has to do if we place its <code class="docutils literal notranslate"><span class="pre">.mv</span></code> file in the email package instead of in the oldlib package:<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="c1"># Expose old names</span> <span class="kn">import</span> <span class="nn">os</span><span class="o">,</span> <span class="nn">sys</span> <span class="n">sys</span><span class="o">.</span><span class="n">stdlib_remapper</span><span class="o">.</span><span class="n">read_directory_mv_files</span><span class="p">(</span><span class="n">os</span><span class="o">.</span><span class="n">path</span><span class="o">.</span><span class="n">dirname</span><span class="p">(</span><span class="vm">__file__</span><span class="p">))</span> </pre></div> </div> <p>I think we should automatically read a package’s directory for any <code class="docutils literal notranslate"><span class="pre">.mv</span></code> files it might contain.</p> </li> </ul> </section> <section id="reference-implementation"> <h2><a class="toc-backref" href="#reference-implementation" role="doc-backlink">Reference Implementation</a></h2> <p>A reference implementation, in the form of a patch against the current (as of this writing) state of the Python 2.6 svn trunk, is available as SourceForge patch #1675334 <a class="footnote-reference brackets" href="#id2" id="id1">[1]</a>. Note that this patch includes a rename of <code class="docutils literal notranslate"><span class="pre">cStringIO</span></code> to <code class="docutils literal notranslate"><span class="pre">cstringio</span></code>, but this is primarily for illustrative and unit testing purposes. Should the patch be accepted, we might want to split this change off into other <a class="pep reference internal" href="../pep-3108/" title="PEP 3108 – Standard Library Reorganization">PEP 3108</a> changes.</p> </section> <section id="references"> <h2><a class="toc-backref" href="#references" role="doc-backlink">References</a></h2> <aside class="footnote-list brackets"> <aside class="footnote brackets" id="id2" role="doc-footnote"> <dt class="label" id="id2">[<a href="#id1">1</a>]</dt> <dd>Reference implementation (<a class="reference external" href="http://bugs.python.org/issue1675334">http://bugs.python.org/issue1675334</a>)</aside> </aside> </section> <section id="copyright"> <h2><a class="toc-backref" href="#copyright" role="doc-backlink">Copyright</a></h2> <p>This document has been placed in the public domain.</p> </section> </section> <hr class="docutils" /> <p>Source: <a class="reference external" href="https://github.com/python/peps/blob/main/peps/pep-0364.rst">https://github.com/python/peps/blob/main/peps/pep-0364.rst</a></p> <p>Last modified: <a class="reference external" href="https://github.com/python/peps/commits/main/peps/pep-0364.rst">2023-09-09 17:39:29 GMT</a></p> </article> <nav id="pep-sidebar"> <h2>Contents</h2> <ul> <li><a class="reference internal" href="#abstract">Abstract</a></li> <li><a class="reference internal" href="#rationale">Rationale</a></li> <li><a class="reference internal" href="#supported-renamings">Supported Renamings</a></li> <li><a class="reference internal" href="#mv-files">.mv files</a></li> <li><a class="reference internal" href="#implementation-specification">Implementation Specification</a></li> <li><a class="reference internal" href="#programmatic-interface">Programmatic Interface</a></li> <li><a class="reference internal" href="#open-issues">Open Issues</a></li> <li><a class="reference internal" href="#reference-implementation">Reference Implementation</a></li> <li><a class="reference internal" href="#references">References</a></li> <li><a class="reference internal" href="#copyright">Copyright</a></li> </ul> <br> <a id="source" href="https://github.com/python/peps/blob/main/peps/pep-0364.rst">Page Source (GitHub)</a> </nav> </section> <script src="../_static/colour_scheme.js"></script> <script src="../_static/wrap_tables.js"></script> <script src="../_static/sticky_banner.js"></script> </body> </html>