CINXE.COM

<!DOCTYPE html> <html lang="en"> <head> <meta charset="utf-8"> <meta name="viewport" content="width=device-width, initial-scale=1.0"> <meta name="color-scheme" content="light dark"> <title>PEP 273 – Import Modules from Zip Archives | peps.python.org</title> <link rel="shortcut icon" href="../_static/py.png"> <link rel="canonical" href="https://peps.python.org/pep-0273/"> <link rel="stylesheet" href="../_static/style.css" type="text/css"> <link rel="stylesheet" href="../_static/mq.css" type="text/css"> <link rel="stylesheet" href="../_static/pygments.css" type="text/css" media="(prefers-color-scheme: light)" id="pyg-light"> <link rel="stylesheet" href="../_static/pygments_dark.css" type="text/css" media="(prefers-color-scheme: dark)" id="pyg-dark"> <link rel="alternate" type="application/rss+xml" title="Latest PEPs" href="https://peps.python.org/peps.rss"> <meta property="og:title" content='PEP 273 – Import Modules from Zip Archives | peps.python.org'> <meta property="og:description" content="This PEP adds the ability to import Python modules *.py, *.py[co] and packages from zip archives. The same code is used to speed up normal directory imports provided os.listdir is available."> <meta property="og:type" content="website"> <meta property="og:url" content="https://peps.python.org/pep-0273/"> <meta property="og:site_name" content="Python Enhancement Proposals (PEPs)"> <meta property="og:image" content="https://peps.python.org/_static/og-image.png"> <meta property="og:image:alt" content="Python PEPs"> <meta property="og:image:width" content="200"> <meta property="og:image:height" content="200"> <meta name="description" content="This PEP adds the ability to import Python modules *.py, *.py[co] and packages from zip archives. The same code is used to speed up normal directory imports provided os.listdir is available."> <meta name="theme-color" content="#3776ab"> </head> <body> <svg xmlns="http://www.w3.org/2000/svg" style="display: none;"> <symbol id="svg-sun-half" viewBox="0 0 24 24" pointer-events="all"> <title>Following system colour scheme</title> <svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"> <circle cx="12" cy="12" r="9"></circle> <path d="M12 3v18m0-12l4.65-4.65M12 14.3l7.37-7.37M12 19.6l8.85-8.85"></path> </svg> </symbol> <symbol id="svg-moon" viewBox="0 0 24 24" pointer-events="all"> <title>Selected dark colour scheme</title> <svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"> <path stroke="none" d="M0 0h24v24H0z" fill="none"></path> <path d="M12 3c.132 0 .263 0 .393 0a7.5 7.5 0 0 0 7.92 12.446a9 9 0 1 1 -8.313 -12.454z"></path> </svg> </symbol> <symbol id="svg-sun" viewBox="0 0 24 24" pointer-events="all"> <title>Selected light colour scheme</title> <svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"> <circle cx="12" cy="12" r="5"></circle> <line x1="12" y1="1" x2="12" y2="3"></line> <line x1="12" y1="21" x2="12" y2="23"></line> <line x1="4.22" y1="4.22" x2="5.64" y2="5.64"></line> <line x1="18.36" y1="18.36" x2="19.78" y2="19.78"></line> <line x1="1" y1="12" x2="3" y2="12"></line> <line x1="21" y1="12" x2="23" y2="12"></line> <line x1="4.22" y1="19.78" x2="5.64" y2="18.36"></line> <line x1="18.36" y1="5.64" x2="19.78" y2="4.22"></line> </svg> </symbol> </svg> <script> document.documentElement.dataset.colour_scheme = localStorage.getItem("colour_scheme") || "auto" </script> <section id="pep-page-section"> <header> <h1>Python Enhancement Proposals</h1> <ul class="breadcrumbs"> <li><a href="https://www.python.org/" title="The Python Programming Language">Python</a> » </li> <li><a href="../pep-0000/">PEP Index</a> » </li> <li>PEP 273</li> </ul> <button id="colour-scheme-cycler" onClick="setColourScheme(nextColourScheme())"> <svg aria-hidden="true" class="colour-scheme-icon-when-auto"><use href="#svg-sun-half"></use></svg> <svg aria-hidden="true" class="colour-scheme-icon-when-dark"><use href="#svg-moon"></use></svg> <svg aria-hidden="true" class="colour-scheme-icon-when-light"><use href="#svg-sun"></use></svg> <span class="visually-hidden">Toggle light / dark / auto colour theme</span> </button> </header> <article> <section id="pep-content"> <h1 class="page-title">PEP 273 – Import Modules from Zip Archives</h1> <dl class="rfc2822 field-list simple"> <dt class="field-odd">Author<span class="colon">:</span></dt> <dd class="field-odd">James C. Ahlstrom <jim at interet.com></dd> <dt class="field-even">Status<span class="colon">:</span></dt> <dd class="field-even"><abbr title="Accepted and implementation complete, or no longer active">Final</abbr></dd> <dt class="field-odd">Type<span class="colon">:</span></dt> <dd class="field-odd"><abbr title="Normative PEP with a new feature for Python, implementation change for CPython or interoperability standard for the ecosystem">Standards Track</abbr></dd> <dt class="field-even">Created<span class="colon">:</span></dt> <dd class="field-even">11-Oct-2001</dd> <dt class="field-odd">Python-Version<span class="colon">:</span></dt> <dd class="field-odd">2.3</dd> <dt class="field-even">Post-History<span class="colon">:</span></dt> <dd class="field-even">26-Oct-2001</dd> </dl> <hr class="docutils" /> <section id="contents"> <details><summary>Table of Contents</summary><ul class="simple"> <li><a class="reference internal" href="#abstract">Abstract</a></li> <li><a class="reference internal" href="#note">Note</a></li> <li><a class="reference internal" href="#specification">Specification</a></li> <li><a class="reference internal" href="#subdirectory-equivalence">Subdirectory Equivalence</a></li> <li><a class="reference internal" href="#efficiency">Efficiency</a></li> <li><a class="reference internal" href="#zlib">zlib</a></li> <li><a class="reference internal" href="#booting">Booting</a></li> <li><a class="reference internal" href="#directory-imports">Directory Imports</a></li> <li><a class="reference internal" href="#benchmarks">Benchmarks</a></li> <li><a class="reference internal" href="#custom-imports">Custom Imports</a></li> <li><a class="reference internal" href="#implementation">Implementation</a></li> <li><a class="reference internal" href="#references">References</a></li> <li><a class="reference internal" href="#copyright">Copyright</a></li> </ul> </details></section> <section id="abstract"> <h2><a class="toc-backref" href="#abstract" role="doc-backlink">Abstract</a></h2> <p>This PEP adds the ability to import Python modules <code class="docutils literal notranslate"><span class="pre">*.py</span></code>, <code class="docutils literal notranslate"><span class="pre">*.py[co]</span></code> and packages from zip archives. The same code is used to speed up normal directory imports provided <code class="docutils literal notranslate"><span class="pre">os.listdir</span></code> is available.</p> </section> <section id="note"> <h2><a class="toc-backref" href="#note" role="doc-backlink">Note</a></h2> <p>Zip imports were added to Python 2.3, but the final implementation uses an approach different from the one described in this PEP. The 2.3 implementation is SourceForge patch #652586 <a class="footnote-reference brackets" href="#id5" id="id1">[1]</a>, which adds new import hooks described in <a class="pep reference internal" href="../pep-0302/" title="PEP 302 – New Import Hooks">PEP 302</a>.</p> <p>The rest of this PEP is therefore only of historical interest.</p> </section> <section id="specification"> <h2><a class="toc-backref" href="#specification" role="doc-backlink">Specification</a></h2> <p>Currently, <code class="docutils literal notranslate"><span class="pre">sys.path</span></code> is a list of directory names as strings. If this PEP is implemented, an item of <code class="docutils literal notranslate"><span class="pre">sys.path</span></code> can be a string naming a zip file archive. The zip archive can contain a subdirectory structure to support package imports. The zip archive satisfies imports exactly as a subdirectory would.</p> <p>The implementation is in C code in the Python core and works on all supported Python platforms.</p> <p>Any files may be present in the zip archive, but only files <code class="docutils literal notranslate"><span class="pre">*.py</span></code> and <code class="docutils literal notranslate"><span class="pre">*.py[co]</span></code> are available for import. Zip import of dynamic modules (<code class="docutils literal notranslate"><span class="pre">*.pyd</span></code>, <code class="docutils literal notranslate"><span class="pre">*.so</span></code>) is disallowed.</p> <p>Just as <code class="docutils literal notranslate"><span class="pre">sys.path</span></code> currently has default directory names, a default zip archive name is added too. Otherwise there is no way to import all Python library files from an archive.</p> </section> <section id="subdirectory-equivalence"> <h2><a class="toc-backref" href="#subdirectory-equivalence" role="doc-backlink">Subdirectory Equivalence</a></h2> <p>The zip archive must be treated exactly as a subdirectory tree so we can support package imports based on current and future rules. All zip data is taken from the Central Directory, the data must be correct, and brain dead zip files are not accommodated.</p> <p>Suppose <code class="docutils literal notranslate"><span class="pre">sys.path</span></code> contains “/A/B/SubDir” and “/C/D/E/Archive.zip”, and we are trying to import <code class="docutils literal notranslate"><span class="pre">modfoo</span></code> from the <code class="docutils literal notranslate"><span class="pre">Q</span></code> package. Then <code class="docutils literal notranslate"><span class="pre">import.c</span></code> will generate a list of paths and extensions and will look for the file. The list of generated paths does not change for zip imports. Suppose <code class="docutils literal notranslate"><span class="pre">import.c</span></code> generates the path “/A/B/SubDir/Q/R/modfoo.pyc”. Then it will also generate the path “/C/D/E/Archive.zip/Q/R/modfoo.pyc”. Finding the SubDir path is exactly equivalent to finding “Q/R/modfoo.pyc” in the archive.</p> <p>Suppose you zip up /A/B/SubDir/* and all its subdirectories. Then your zip file will satisfy imports just as your subdirectory did.</p> <p>Well, not quite. You can’t satisfy dynamic modules from a zip file. Dynamic modules have extensions like <code class="docutils literal notranslate"><span class="pre">.dll</span></code>, <code class="docutils literal notranslate"><span class="pre">.pyd</span></code>, and <code class="docutils literal notranslate"><span class="pre">.so</span></code>. They are operating system dependent, and probably can’t be loaded except from a file. It might be possible to extract the dynamic module from the zip file, write it to a plain file and load it. But that would mean creating temporary files, and dealing with all the <code class="docutils literal notranslate"><span class="pre">dynload_*.c</span></code>, and that’s probably not a good idea.</p> <p>When trying to import <code class="docutils literal notranslate"><span class="pre">*.pyc</span></code>, if it is not available then <code class="docutils literal notranslate"><span class="pre">*.pyo</span></code> will be used instead. And vice versa when looking for <code class="docutils literal notranslate"><span class="pre">*.pyo</span></code>. If neither <code class="docutils literal notranslate"><span class="pre">*.pyc</span></code> nor <code class="docutils literal notranslate"><span class="pre">*.pyo</span></code> is available, or if the magic numbers are invalid, then <code class="docutils literal notranslate"><span class="pre">*.py</span></code> will be compiled and used to satisfy the import, but the compiled file will not be saved. Python would normally write it to the same directory as <code class="docutils literal notranslate"><span class="pre">*.py</span></code>, but surely we don’t want to write to the zip file. We could write to the directory of the zip archive, but that would clutter it up, not good if it is <code class="docutils literal notranslate"><span class="pre">/usr/bin</span></code> for example.</p> <p>Failing to write the compiled files will make zip imports very slow, and the user will probably not figure out what is wrong. So it is best to put <code class="docutils literal notranslate"><span class="pre">*.pyc</span></code> and <code class="docutils literal notranslate"><span class="pre">*.pyo</span></code> in the archive with the <code class="docutils literal notranslate"><span class="pre">*.py</span></code>.</p> </section> <section id="efficiency"> <h2><a class="toc-backref" href="#efficiency" role="doc-backlink">Efficiency</a></h2> <p>The only way to find files in a zip archive is linear search. So for each zip file in <code class="docutils literal notranslate"><span class="pre">sys.path</span></code>, we search for its names once, and put the names plus other relevant data into a static Python dictionary. The key is the archive name from <code class="docutils literal notranslate"><span class="pre">sys.path</span></code> joined with the file name (including any subdirectories) within the archive. This is exactly the name generated by <code class="docutils literal notranslate"><span class="pre">import.c</span></code>, and makes lookup easy.</p> <p>This same mechanism is used to speed up directory (non-zip) imports. See below.</p> </section> <section id="zlib"> <h2><a class="toc-backref" href="#zlib" role="doc-backlink">zlib</a></h2> <p>Compressed zip archives require <code class="docutils literal notranslate"><span class="pre">zlib</span></code> for decompression. Prior to any other imports, we attempt an import of <code class="docutils literal notranslate"><span class="pre">zlib</span></code>. Import of compressed files will fail with a message “missing <code class="docutils literal notranslate"><span class="pre">zlib</span></code>” unless <code class="docutils literal notranslate"><span class="pre">zlib</span></code> is available.</p> </section> <section id="booting"> <h2><a class="toc-backref" href="#booting" role="doc-backlink">Booting</a></h2> <p>Python imports <code class="docutils literal notranslate"><span class="pre">site.py</span></code> itself, and this imports <code class="docutils literal notranslate"><span class="pre">os</span></code>, <code class="docutils literal notranslate"><span class="pre">nt</span></code>, <code class="docutils literal notranslate"><span class="pre">ntpath</span></code>, <code class="docutils literal notranslate"><span class="pre">stat</span></code>, and <code class="docutils literal notranslate"><span class="pre">UserDict</span></code>. It also imports <code class="docutils literal notranslate"><span class="pre">sitecustomize.py</span></code> which may import more modules. Zip imports must be available before <code class="docutils literal notranslate"><span class="pre">site.py</span></code> is imported.</p> <p>Just as there are default directories in <code class="docutils literal notranslate"><span class="pre">sys.path</span></code>, there must be one or more default zip archives too.</p> <p>The problem is what the name should be. The name should be linked with the Python version, so the Python executable can correctly find its corresponding libraries even when there are multiple Python versions on the same machine.</p> <p>We add one name to <code class="docutils literal notranslate"><span class="pre">sys.path</span></code>. On Unix, the directory is <code class="docutils literal notranslate"><span class="pre">sys.prefix</span> <span class="pre">+</span> <span class="pre">"/lib"</span></code>, and the file name is <code class="docutils literal notranslate"><span class="pre">"python%s%s.zip"</span> <span class="pre">%</span> <span class="pre">(sys.version[0],</span> <span class="pre">sys.version[2])</span></code>. So for Python 2.2 and prefix <code class="docutils literal notranslate"><span class="pre">/usr/local</span></code>, the path <code class="docutils literal notranslate"><span class="pre">/usr/local/lib/python2.2/</span></code> is already on <code class="docutils literal notranslate"><span class="pre">sys.path</span></code>, and <code class="docutils literal notranslate"><span class="pre">/usr/local/lib/python22.zip</span></code> would be added. On Windows, the file is the full path to <code class="docutils literal notranslate"><span class="pre">python22.dll</span></code>, with “dll” replaced by “zip”. The zip archive name is always inserted as the second item in <code class="docutils literal notranslate"><span class="pre">sys.path</span></code>. The first is the directory of the <code class="docutils literal notranslate"><span class="pre">main.py</span></code> (thanks Tim).</p> </section> <section id="directory-imports"> <h2><a class="toc-backref" href="#directory-imports" role="doc-backlink">Directory Imports</a></h2> <p>The static Python dictionary used to speed up zip imports can be used to speed up normal directory imports too. For each item in <code class="docutils literal notranslate"><span class="pre">sys.path</span></code> that is not a zip archive, we call <code class="docutils literal notranslate"><span class="pre">os.listdir</span></code>, and add the directory contents to the dictionary. Then instead of calling <code class="docutils literal notranslate"><span class="pre">fopen()</span></code> in a double loop, we just check the dictionary. This greatly speeds up imports. If <code class="docutils literal notranslate"><span class="pre">os.listdir</span></code> doesn’t exist, the dictionary is not used.</p> </section> <section id="benchmarks"> <h2><a class="toc-backref" href="#benchmarks" role="doc-backlink">Benchmarks</a></h2> <table class="docutils align-default"> <thead> <tr class="row-odd"><th class="head">Case</th> <th class="head">Original 2.2a3</th> <th class="head">Using os.listdir</th> <th class="head">Zip Uncomp</th> <th class="head">Zip Compr</th> </tr> </thead> <tbody> <tr class="row-even"><td>1</td> <td>3.2 2.5 3.2->1.02</td> <td>2.3 2.5 2.3->0.87</td> <td>1.66->0.93</td> <td>1.5->1.07</td> </tr> <tr class="row-odd"><td>2</td> <td>2.8 3.9 3.0->1.32</td> <td>Same as Case 1.</td> <td></td> <td></td> </tr> <tr class="row-even"><td>3</td> <td>5.7 5.7 5.7->5.7</td> <td>2.1 2.1 2.1->1.8</td> <td>1.25->0.99</td> <td>1.19->1.13</td> </tr> <tr class="row-odd"><td>4</td> <td>9.4 9.4 9.3->9.35</td> <td>Same as Case 3.</td> <td></td> <td></td> </tr> </tbody> </table> <p>Case 1: Local drive C:, <code class="docutils literal notranslate"><span class="pre">sys.path</span></code> has its default value. Case 2: Local drive C:, directory with files is at the end of <code class="docutils literal notranslate"><span class="pre">sys.path</span></code>. Case 3: Network drive, <code class="docutils literal notranslate"><span class="pre">sys.path</span></code> has its default value. Case 4: Network drive, directory with files is at the end of <code class="docutils literal notranslate"><span class="pre">sys.path</span></code>.</p> <p>Benchmarks were performed on a Pentium 4 clone, 1.4 GHz, 256 Meg. The machine was running Windows 2000 with a Linux/Samba network server. Times are in seconds, and are the time to import about 100 Lib modules. Case 2 and 4 have the “correct” directory moved to the end of <code class="docutils literal notranslate"><span class="pre">sys.path</span></code>. “Uncomp” means uncompressed zip archive, “Compr” means compressed.</p> <p>Initial times are after a re-boot of the system; the time after “->” is the time after repeated runs. Times to import from C: after a re-boot are rather highly variable for the “Original” case, but are more realistic.</p> </section> <section id="custom-imports"> <h2><a class="toc-backref" href="#custom-imports" role="doc-backlink">Custom Imports</a></h2> <p>The logic demonstrates the ability to import using default searching until a needed Python module (in this case, <code class="docutils literal notranslate"><span class="pre">os</span></code>) becomes available. This can be used to bootstrap custom importers. For example, if “<code class="docutils literal notranslate"><span class="pre">importer()</span></code>” in <code class="docutils literal notranslate"><span class="pre">__init__.py</span></code> exists, then it could be used for imports. The “<code class="docutils literal notranslate"><span class="pre">importer()</span></code>” can freely import os and other modules, and these will be satisfied from the default mechanism. This PEP does not define any custom importers, and this note is for information only.</p> </section> <section id="implementation"> <h2><a class="toc-backref" href="#implementation" role="doc-backlink">Implementation</a></h2> <p>A C implementation is available as SourceForge patch 492105. Superseded by patch 652586 and current CVS. <a class="footnote-reference brackets" href="#id6" id="id2">[2]</a></p> <p>A newer version (updated for recent CVS by Paul Moore) is 645650. Superseded by patch 652586 and current CVS. <a class="footnote-reference brackets" href="#id7" id="id3">[3]</a></p> <p>A competing implementation by Just van Rossum is 652586, which is the basis for the final implementation of <a class="pep reference internal" href="../pep-0302/" title="PEP 302 – New Import Hooks">PEP 302</a>. <a class="pep reference internal" href="../pep-0273/" title="PEP 273 – Import Modules from Zip Archives">PEP 273</a> has been implemented using <a class="pep reference internal" href="../pep-0302/" title="PEP 302 – New Import Hooks">PEP 302</a>’s import hooks. <a class="footnote-reference brackets" href="#id5" id="id4">[1]</a></p> </section> <section id="references"> <h2><a class="toc-backref" href="#references" role="doc-backlink">References</a></h2> <aside class="footnote-list brackets"> <aside class="footnote brackets" id="id5" role="doc-footnote"> <dt class="label" id="id5">[1]<em> (<a href='#id1'>1</a>, <a href='#id4'>2</a>) </em></dt> <dd>Just van Rossum, New import hooks + Import from Zip files <a class="reference external" href="https://bugs.python.org/issue652586">https://bugs.python.org/issue652586</a></aside> <aside class="footnote brackets" id="id6" role="doc-footnote"> <dt class="label" id="id6">[<a href="#id2">2</a>]</dt> <dd>Import from Zip archive, James C. Ahlstrom <a class="reference external" href="https://bugs.python.org/issue492105">https://bugs.python.org/issue492105</a></aside> <aside class="footnote brackets" id="id7" role="doc-footnote"> <dt class="label" id="id7">[<a href="#id3">3</a>]</dt> <dd>Import from Zip Archive, Paul Moore <a class="reference external" href="https://bugs.python.org/issue645650">https://bugs.python.org/issue645650</a></aside> </aside> </section> <section id="copyright"> <h2><a class="toc-backref" href="#copyright" role="doc-backlink">Copyright</a></h2> <p>This document has been placed in the public domain.</p> </section> </section> <hr class="docutils" /> <p>Source: <a class="reference external" href="https://github.com/python/peps/blob/main/peps/pep-0273.rst">https://github.com/python/peps/blob/main/peps/pep-0273.rst</a></p> <p>Last modified: <a class="reference external" href="https://github.com/python/peps/commits/main/peps/pep-0273.rst">2023-09-09 17:39:29 GMT</a></p> </article> <nav id="pep-sidebar"> <h2>Contents</h2> <ul> <li><a class="reference internal" href="#abstract">Abstract</a></li> <li><a class="reference internal" href="#note">Note</a></li> <li><a class="reference internal" href="#specification">Specification</a></li> <li><a class="reference internal" href="#subdirectory-equivalence">Subdirectory Equivalence</a></li> <li><a class="reference internal" href="#efficiency">Efficiency</a></li> <li><a class="reference internal" href="#zlib">zlib</a></li> <li><a class="reference internal" href="#booting">Booting</a></li> <li><a class="reference internal" href="#directory-imports">Directory Imports</a></li> <li><a class="reference internal" href="#benchmarks">Benchmarks</a></li> <li><a class="reference internal" href="#custom-imports">Custom Imports</a></li> <li><a class="reference internal" href="#implementation">Implementation</a></li> <li><a class="reference internal" href="#references">References</a></li> <li><a class="reference internal" href="#copyright">Copyright</a></li> </ul> <br> <a id="source" href="https://github.com/python/peps/blob/main/peps/pep-0273.rst">Page Source (GitHub)</a> </nav> </section> <script src="../_static/colour_scheme.js"></script> <script src="../_static/wrap_tables.js"></script> <script src="../_static/sticky_banner.js"></script> </body> </html>