CINXE.COM

<!DOCTYPE html> <html lang="en"> <head> <meta charset="utf-8"> <meta name="viewport" content="width=device-width, initial-scale=1.0"> <meta name="color-scheme" content="light dark"> <title>PEP 376 – Database of Installed Python Distributions | peps.python.org</title> <link rel="shortcut icon" href="../_static/py.png"> <link rel="canonical" href="https://peps.python.org/pep-0376/"> <link rel="stylesheet" href="../_static/style.css" type="text/css"> <link rel="stylesheet" href="../_static/mq.css" type="text/css"> <link rel="stylesheet" href="../_static/pygments.css" type="text/css" media="(prefers-color-scheme: light)" id="pyg-light"> <link rel="stylesheet" href="../_static/pygments_dark.css" type="text/css" media="(prefers-color-scheme: dark)" id="pyg-dark"> <link rel="alternate" type="application/rss+xml" title="Latest PEPs" href="https://peps.python.org/peps.rss"> <meta property="og:title" content='PEP 376 – Database of Installed Python Distributions | peps.python.org'> <meta property="og:description" content="The goal of this PEP is to provide a standard infrastructure to manage project distributions installed on a system, so all tools that are installing or removing projects are interoperable."> <meta property="og:type" content="website"> <meta property="og:url" content="https://peps.python.org/pep-0376/"> <meta property="og:site_name" content="Python Enhancement Proposals (PEPs)"> <meta property="og:image" content="https://peps.python.org/_static/og-image.png"> <meta property="og:image:alt" content="Python PEPs"> <meta property="og:image:width" content="200"> <meta property="og:image:height" content="200"> <meta name="description" content="The goal of this PEP is to provide a standard infrastructure to manage project distributions installed on a system, so all tools that are installing or removing projects are interoperable."> <meta name="theme-color" content="#3776ab"> </head> <body> <svg xmlns="http://www.w3.org/2000/svg" style="display: none;"> <symbol id="svg-sun-half" viewBox="0 0 24 24" pointer-events="all"> <title>Following system colour scheme</title> <svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"> <circle cx="12" cy="12" r="9"></circle> <path d="M12 3v18m0-12l4.65-4.65M12 14.3l7.37-7.37M12 19.6l8.85-8.85"></path> </svg> </symbol> <symbol id="svg-moon" viewBox="0 0 24 24" pointer-events="all"> <title>Selected dark colour scheme</title> <svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"> <path stroke="none" d="M0 0h24v24H0z" fill="none"></path> <path d="M12 3c.132 0 .263 0 .393 0a7.5 7.5 0 0 0 7.92 12.446a9 9 0 1 1 -8.313 -12.454z"></path> </svg> </symbol> <symbol id="svg-sun" viewBox="0 0 24 24" pointer-events="all"> <title>Selected light colour scheme</title> <svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"> <circle cx="12" cy="12" r="5"></circle> <line x1="12" y1="1" x2="12" y2="3"></line> <line x1="12" y1="21" x2="12" y2="23"></line> <line x1="4.22" y1="4.22" x2="5.64" y2="5.64"></line> <line x1="18.36" y1="18.36" x2="19.78" y2="19.78"></line> <line x1="1" y1="12" x2="3" y2="12"></line> <line x1="21" y1="12" x2="23" y2="12"></line> <line x1="4.22" y1="19.78" x2="5.64" y2="18.36"></line> <line x1="18.36" y1="5.64" x2="19.78" y2="4.22"></line> </svg> </symbol> </svg> <script> document.documentElement.dataset.colour_scheme = localStorage.getItem("colour_scheme") || "auto" </script> <section id="pep-page-section"> <header> <h1>Python Enhancement Proposals</h1> <ul class="breadcrumbs"> <li><a href="https://www.python.org/" title="The Python Programming Language">Python</a> » </li> <li><a href="../pep-0000/">PEP Index</a> » </li> <li>PEP 376</li> </ul> <button id="colour-scheme-cycler" onClick="setColourScheme(nextColourScheme())"> <svg aria-hidden="true" class="colour-scheme-icon-when-auto"><use href="#svg-sun-half"></use></svg> <svg aria-hidden="true" class="colour-scheme-icon-when-dark"><use href="#svg-moon"></use></svg> <svg aria-hidden="true" class="colour-scheme-icon-when-light"><use href="#svg-sun"></use></svg> Toggle light / dark / auto colour theme </button> </header> <article> <section id="pep-content"> <h1 class="page-title">PEP 376 – Database of Installed Python Distributions</h1> <dl class="rfc2822 field-list simple"> <dt class="field-odd">Author:</dt> <dd class="field-odd">Tarek Ziadé <tarek at ziade.org></dd> <dt class="field-even">Status:</dt> <dd class="field-even"><abbr title="Accepted and implementation complete, or no longer active">Final</abbr></dd> <dt class="field-odd">Type:</dt> <dd class="field-odd"><abbr title="Normative PEP with a new feature for Python, implementation change for CPython or interoperability standard for the ecosystem">Standards Track</abbr></dd> <dt class="field-even">Topic:</dt> <dd class="field-even"><a class="reference external" href="../topic/packaging/">Packaging</a></dd> <dt class="field-odd">Created:</dt> <dd class="field-odd">22-Feb-2009</dd> <dt class="field-even">Python-Version:</dt> <dd class="field-even">2.7, 3.2</dd> <dt class="field-odd">Post-History:</dt> <dd class="field-odd"><a class="reference external" href="https://mail.python.org/archives/list/python-dev@python.org/thread/ILLTIOZAULMDY5CAS6GOITEYJ4HNFATQ/" title="Python-Dev thread">22-Jun-2009</a></dd> </dl> <hr class="docutils" /> <section id="contents"> <details><summary>Table of Contents</summary><ul class="simple"> <li><a class="reference internal" href="#abstract">Abstract</a></li> <li><a class="reference internal" href="#rationale">Rationale</a><ul> <li><a class="reference internal" href="#how-distributions-are-installed">How distributions are installed</a></li> <li><a class="reference internal" href="#uninstall-information">Uninstall information</a></li> <li><a class="reference internal" href="#what-this-pep-proposes">What this PEP proposes</a></li> </ul> </li> <li><a class="reference internal" href="#one-dist-info-directory-per-installed-distribution">One .dist-info directory per installed distribution</a><ul> <li><a class="reference internal" href="#record">RECORD</a></li> <li><a class="reference internal" href="#installer">INSTALLER</a></li> <li><a class="reference internal" href="#requested">REQUESTED</a></li> </ul> </li> <li><a class="reference internal" href="#implementation-details">Implementation details</a><ul> <li><a class="reference internal" href="#new-functions-and-classes-in-pkgutil">New functions and classes in pkgutil</a><ul> <li><a class="reference internal" href="#functions">Functions</a></li> <li><a class="reference internal" href="#distribution-class">Distribution class</a></li> <li><a class="reference internal" href="#examples">Examples</a></li> </ul> </li> <li><a class="reference internal" href="#new-functions-in-distutils">New functions in Distutils</a><ul> <li><a class="reference internal" href="#filtering">Filtering</a></li> <li><a class="reference internal" href="#installer-marker">Installer marker</a></li> <li><a class="reference internal" href="#adding-an-uninstall-script">Adding an Uninstall script</a></li> </ul> </li> </ul> </li> <li><a class="reference internal" href="#backward-compatibility-and-roadmap">Backward compatibility and roadmap</a></li> <li><a class="reference internal" href="#references">References</a></li> <li><a class="reference internal" href="#acknowledgements">Acknowledgements</a></li> <li><a class="reference internal" href="#copyright">Copyright</a></li> </ul> </details></section> <div class="pep-banner canonical-pypa-spec sticky-banner admonition important"> Important This PEP is a historical document. The up-to-date, canonical spec, <a class="reference external" href="https://packaging.python.org/en/latest/specifications/core-metadata/#core-metadata" title="(in Python Packaging User Guide)">Core metadata specifications</a>, is maintained on the <a class="reference external" href="https://packaging.python.org/en/latest/specifications/">PyPA specs page</a>. × See the <a class="reference external" href="https://www.pypa.io/en/latest/specifications/#handling-fixes-and-other-minor-updates">PyPA specification update process</a> for how to propose changes. </div> <section id="abstract"> <h2><a class="toc-backref" href="#abstract" role="doc-backlink">Abstract</a></h2> The goal of this PEP is to provide a standard infrastructure to manage project distributions installed on a system, so all tools that are installing or removing projects are interoperable. To achieve this goal, the PEP proposes a new format to describe installed distributions on a system. It also describes a reference implementation for the standard library. In the past an attempt was made to create an installation database (see <a class="pep reference internal" href="../pep-0262/" title="PEP 262 – A Database of Installed Python Packages">PEP 262</a>). Combined with <a class="pep reference internal" href="../pep-0345/" title="PEP 345 – Metadata for Python Software Packages 1.2">PEP 345</a>, the current proposal supersedes <a class="pep reference internal" href="../pep-0262/" title="PEP 262 – A Database of Installed Python Packages">PEP 262</a>. Note: the implementation plan didn’t go as expected, so it should be considered informative only for this PEP. </section> <section id="rationale"> <h2><a class="toc-backref" href="#rationale" role="doc-backlink">Rationale</a></h2> There are two problems right now in the way distributions are installed in Python: <ul class="simple"> <li>There are too many ways to do it and this makes interoperation difficult.</li> <li>There is no API to get information on installed distributions.</li> </ul> <section id="how-distributions-are-installed"> <h3><a class="toc-backref" href="#how-distributions-are-installed" role="doc-backlink">How distributions are installed</a></h3> Right now, when a distribution is installed in Python, every element can be installed in a different directory. For instance, <code class="docutils literal notranslate">Distutils</code> installs the pure Python code in the <code class="docutils literal notranslate">purelib</code> directory, which is <code class="docutils literal notranslate">lib/python2.6/site-packages</code> for unix-like systems and Mac OS X, or <code class="docutils literal notranslate">Lib\site-packages</code> under Python’s installation directory for Windows. Additionally, the <code class="docutils literal notranslate">install_egg_info</code> subcommand of the Distutils <code class="docutils literal notranslate">install</code> command adds an <code class="docutils literal notranslate">.egg-info</code> file for the project into the <code class="docutils literal notranslate">purelib</code> directory. For example, for the <code class="docutils literal notranslate">docutils</code> distribution, which contains one package an extra module and executable scripts, three elements are installed in <code class="docutils literal notranslate">site-packages</code>: <ul class="simple"> <li><code class="docutils literal notranslate">docutils</code>: The <code class="docutils literal notranslate">docutils</code> package.</li> <li><code class="docutils literal notranslate">roman.py</code>: An extra module used by <code class="docutils literal notranslate">docutils</code>.</li> <li><code class="docutils literal notranslate">docutils-0.5-py2.6.egg-info</code>: A file containing the distribution metadata as described in <a class="pep reference internal" href="../pep-0314/" title="PEP 314 – Metadata for Python Software Packages 1.1">PEP 314</a>. This file corresponds to the file called <code class="docutils literal notranslate">PKG-INFO</code>, built by the <code class="docutils literal notranslate">sdist</code> command.</li> </ul> Some executable scripts, such as <code class="docutils literal notranslate">rst2html.py</code>, are also added in the <code class="docutils literal notranslate">bin</code> directory of the Python installation. Another project called <code class="docutils literal notranslate">setuptools</code> <a class="footnote-reference brackets" href="#setuptools" id="id1">[1]</a> has two other formats to install distributions, called <code class="docutils literal notranslate">EggFormats</code> <a class="footnote-reference brackets" href="#eggformats" id="id2">[4]</a>: <ul class="simple"> <li>a self-contained <code class="docutils literal notranslate">.egg</code> directory, that contains all the distribution files and the distribution metadata in a file called <code class="docutils literal notranslate">PKG-INFO</code> in a subdirectory called <code class="docutils literal notranslate">EGG-INFO</code>. <code class="docutils literal notranslate">setuptools</code> creates other files in that directory that can be considered as complementary metadata.</li> <li>an <code class="docutils literal notranslate">.egg-info</code> directory installed in <code class="docutils literal notranslate">site-packages</code>, that contains the same files <code class="docutils literal notranslate">EGG-INFO</code> has in the <code class="docutils literal notranslate">.egg</code> format.</li> </ul> The first format is automatically used when you install a distribution that uses the <code class="docutils literal notranslate">setuptools.setup</code> function in its setup.py file, instead of the <code class="docutils literal notranslate">distutils.core.setup</code> one. <code class="docutils literal notranslate">setuptools</code> also add a reference to the distribution into an <code class="docutils literal notranslate">easy-install.pth</code> file. Last, the <code class="docutils literal notranslate">setuptools</code> project provides an executable script called <code class="docutils literal notranslate">easy_install</code> <a class="footnote-reference brackets" href="#easyinstall" id="id3">[2]</a> that installs all distributions, including distutils-based ones in self-contained <code class="docutils literal notranslate">.egg</code> directories. If you want to have standalone <code class="docutils literal notranslate">.egg-info</code> directories for your distributions, e.g. the second <code class="docutils literal notranslate">setuptools</code> format, you have to force it when you work with a setuptools-based distribution or with the <code class="docutils literal notranslate">easy_install</code> script. You can force it by using the <code class="docutils literal notranslate">--single-version-externally-managed</code> option or the <code class="docutils literal notranslate">--root</code> option. This will make the <code class="docutils literal notranslate">setuptools</code> project install the project like distutils does. This option is used by : <ul class="simple"> <li>the <code class="docutils literal notranslate">pip</code> <a class="footnote-reference brackets" href="#pip" id="id4">[3]</a> installer</li> <li>the Fedora packagers <a class="footnote-reference brackets" href="#fedora" id="id5">[5]</a>.</li> <li>the Debian packagers <a class="footnote-reference brackets" href="#debian" id="id6">[6]</a>.</li> </ul> </section> <section id="uninstall-information"> <h3><a class="toc-backref" href="#uninstall-information" role="doc-backlink">Uninstall information</a></h3> Distutils doesn’t provide an <code class="docutils literal notranslate">uninstall</code> command. If you want to uninstall a distribution, you have to be a power user and remove the various elements that were installed, and then look over the <code class="docutils literal notranslate">.pth</code> file to clean them if necessary. And the process differs depending on the tools you have used to install the distribution and if the distribution’s <code class="docutils literal notranslate">setup.py</code> uses Distutils or Setuptools. Under some circumstances, you might not be able to know for sure that you have removed everything, or that you didn’t break another distribution by removing a file that is shared among several distributions. But there’s a common behavior: when you install a distribution, files are copied in your system. And it’s possible to keep track of these files for later removal. Moreover, the Pip project has gained an <code class="docutils literal notranslate">uninstall</code> feature lately. It records all installed files, using the <code class="docutils literal notranslate">record</code> option of the <code class="docutils literal notranslate">install</code> command. </section> <section id="what-this-pep-proposes"> <h3><a class="toc-backref" href="#what-this-pep-proposes" role="doc-backlink">What this PEP proposes</a></h3> To address those issues, this PEP proposes a few changes: <ul class="simple"> <li>A new <code class="docutils literal notranslate">.dist-info</code> structure using a directory, inspired on one format of the <code class="docutils literal notranslate">EggFormats</code> standard from <code class="docutils literal notranslate">setuptools</code>.</li> <li>New APIs in <code class="docutils literal notranslate">pkgutil</code> to be able to query the information of installed distributions.</li> <li>An uninstall function and an uninstall script in Distutils.</li> </ul> </section> </section> <section id="one-dist-info-directory-per-installed-distribution"> <h2><a class="toc-backref" href="#one-dist-info-directory-per-installed-distribution" role="doc-backlink">One .dist-info directory per installed distribution</a></h2> This PEP proposes an installation format inspired by one of the options in the <code class="docutils literal notranslate">EggFormats</code> standard, the one that uses a distinct directory located in the site-packages directory. This distinct directory is named as follows: <div class="highlight-default notranslate"><div class="highlight"><pre>name + '-' + version + '.dist-info' </pre></div> </div> This <code class="docutils literal notranslate">.dist-info</code> directory can contain these files: <ul class="simple"> <li><code class="docutils literal notranslate">METADATA</code>: contains metadata, as described in <a class="pep reference internal" href="../pep-0345/" title="PEP 345 – Metadata for Python Software Packages 1.2">PEP 345</a>, <a class="pep reference internal" href="../pep-0314/" title="PEP 314 – Metadata for Python Software Packages 1.1">PEP 314</a> and <a class="pep reference internal" href="../pep-0241/" title="PEP 241 – Metadata for Python Software Packages">PEP 241</a>.</li> <li><code class="docutils literal notranslate">RECORD</code>: records the list of installed files</li> <li><code class="docutils literal notranslate">INSTALLER</code>: records the name of the tool used to install the project</li> <li><code class="docutils literal notranslate">REQUESTED</code>: the presence of this file indicates that the project installation was explicitly requested (i.e., not installed as a dependency).</li> </ul> The METADATA, RECORD and INSTALLER files are mandatory, while REQUESTED may be missing. This proposal will not impact Python itself because the metadata files are not used anywhere yet in the standard library besides Distutils. It will impact the <code class="docutils literal notranslate">setuptools</code> and <code class="docutils literal notranslate">pip</code> projects but, given the fact that they already work with a directory that contains a <code class="docutils literal notranslate">PKG-INFO</code> file, the change will have no deep consequences. <section id="record"> <h3><a class="toc-backref" href="#record" role="doc-backlink">RECORD</a></h3> A <code class="docutils literal notranslate">RECORD</code> file is added inside the <code class="docutils literal notranslate">.dist-info</code> directory at installation time when installing a source distribution using the <code class="docutils literal notranslate">install</code> command. Notice that when installing a binary distribution created with <code class="docutils literal notranslate">bdist</code> command or a <code class="docutils literal notranslate">bdist</code>-based command, the <code class="docutils literal notranslate">RECORD</code> file will be installed as well since these commands use the <code class="docutils literal notranslate">install</code> command to create binary distributions. The <code class="docutils literal notranslate">RECORD</code> file holds the list of installed files. These correspond to the files listed by the <code class="docutils literal notranslate">record</code> option of the <code class="docutils literal notranslate">install</code> command, and will be generated by default. This allows the implementation of an uninstallation feature, as explained later in this PEP. The <code class="docutils literal notranslate">install</code> command also provides an option to prevent the <code class="docutils literal notranslate">RECORD</code> file from being written and this option should be used when creating system packages. Third-party installation tools also should not overwrite or delete files that are not in a RECORD file without prompting or warning. This RECORD file is inspired from <a class="pep reference internal" href="../pep-0262/" title="PEP 262 – A Database of Installed Python Packages">PEP 262</a> FILES. The <code class="docutils literal notranslate">RECORD</code> file is a CSV file, composed of records, one line per installed file. The <code class="docutils literal notranslate">csv</code> module is used to read the file, with these options: <ul class="simple"> <li>field delimiter : <code class="docutils literal notranslate">,</code></li> <li>quoting char : <code class="docutils literal notranslate">"</code>.</li> <li>line terminator : <code class="docutils literal notranslate">os.linesep</code> (so <code class="docutils literal notranslate">\r\n</code> or <code class="docutils literal notranslate">\n</code>)</li> </ul> When a distribution is installed, files can be installed under: <ul class="simple"> <li>the base location: path defined by the <code class="docutils literal notranslate">--install-lib</code> option, which defaults to the site-packages directory.</li> <li>the installation prefix: path defined by the <code class="docutils literal notranslate">--prefix</code> option, which defaults to <code class="docutils literal notranslate">sys.prefix</code>.</li> <li>any other path on the system.</li> </ul> Each record is composed of three elements: <ul> <li>the file’s path<ul class="simple"> <li>a ‘/’-separated path, relative to the base location, if the file is under the base location.</li> <li>a ‘/’-separated path, relative to the base location, if the file is under the installation prefix AND if the base location is a subpath of the installation prefix.</li> <li>an absolute path, using the local platform separator</li> </ul> </li> <li>a hash of the file’s contents. Notice that <code class="docutils literal notranslate">pyc</code> and <code class="docutils literal notranslate">pyo</code> generated files don’t have any hash because they are automatically produced from <code class="docutils literal notranslate">py</code> files. So checking the hash of the corresponding <code class="docutils literal notranslate">py</code> file is enough to decide if the file and its associated <code class="docutils literal notranslate">pyc</code> or <code class="docutils literal notranslate">pyo</code> files have changed.The hash is either the empty string or the hash algorithm as named in <code class="docutils literal notranslate">hashlib.algorithms_guaranteed</code>, followed by the equals character <code class="docutils literal notranslate">=</code>, followed by the urlsafe-base64-nopad encoding of the digest (<code class="docutils literal notranslate">base64.urlsafe_b64encode(digest)</code> with trailing <code class="docutils literal notranslate">=</code> removed). </li> <li>the file’s size in bytes</li> </ul> The <code class="docutils literal notranslate">csv</code> module is used to generate this file, so the field separator is “,”. Any “,” character found within a field is escaped automatically by <code class="docutils literal notranslate">csv</code>. When the file is read, the <code class="docutils literal notranslate">U</code> option is used so the universal newline support (see <a class="pep reference internal" href="../pep-0278/" title="PEP 278 – Universal Newline Support">PEP 278</a>) is activated, avoiding any trouble reading a file produced on a platform that uses a different new line terminator. Here’s an example of a RECORD file (extract): <div class="highlight-default notranslate"><div class="highlight"><pre>lib/python2.6/site-packages/docutils/__init__.py,md5=nWt-Dge1eug4iAgqLS_uWg,9544 lib/python2.6/site-packages/docutils/__init__.pyc,, lib/python2.6/site-packages/docutils/core.py,md5=X90C_JLIcC78PL74iuhPnA,66188 lib/python2.6/site-packages/docutils/core.pyc,, lib/python2.6/site-packages/roman.py,md5=7YhfNczihNjOY0FXlupwBg,234 lib/python2.6/site-packages/roman.pyc,, /usr/local/bin/rst2html.py,md5=g22D3amDLJP-FhBzCi7EvA,234 /usr/local/bin/rst2html.pyc,, python2.6/site-packages/docutils-0.5.dist-info/METADATA,md5=ovJyUNzXdArGfmVyb0onyA,195 lib/python2.6/site-packages/docutils-0.5.dist-info/RECORD,, </pre></div> </div> Notice that the <code class="docutils literal notranslate">RECORD</code> file can’t contain a hash of itself and is just mentioned here A project that installs a <code class="docutils literal notranslate">config.ini</code> file in <code class="docutils literal notranslate">/etc/myapp</code> will be added like this: <div class="highlight-default notranslate"><div class="highlight"><pre>/etc/myapp/config.ini,md5=gLfd6IANquzGLhOkW4Mfgg,9544 </pre></div> </div> For a windows platform, the drive letter is added for the absolute paths, so a file that is copied in c:MyAppwill be: <div class="highlight-default notranslate"><div class="highlight"><pre>c:\etc\myapp\config.ini,md5=gLfd6IANquzGLhOkW4Mfgg,9544 </pre></div> </div> </section> <section id="installer"> <h3><a class="toc-backref" href="#installer" role="doc-backlink">INSTALLER</a></h3> The <code class="docutils literal notranslate">install</code> command has a new option called <code class="docutils literal notranslate">installer</code>. This option is the name of the tool used to invoke the installation. It’s a normalized lower-case string matching <code class="docutils literal notranslate">[a-z0-9_\-\.]</code>. <blockquote> <div>$ python setup.py install –installer=pkg-system</div></blockquote> It defaults to <code class="docutils literal notranslate">distutils</code> if not provided. When a distribution is installed, the INSTALLER file is generated in the <code class="docutils literal notranslate">.dist-info</code> directory with this value, to keep track of who installed the distribution. The file is a single-line text file. </section> <section id="requested"> <h3><a class="toc-backref" href="#requested" role="doc-backlink">REQUESTED</a></h3> Some install tools automatically detect unfulfilled dependencies and install them. In these cases, it is useful to track which distributions were installed purely as a dependency, so if their dependent distribution is later uninstalled, the user can be alerted of the orphaned dependency. If a distribution is installed by direct user request (the usual case), a file REQUESTED is added to the .dist-info directory of the installed distribution. The REQUESTED file may be empty, or may contain a marker comment line beginning with the “#” character. If an install tool installs a distribution automatically, as a dependency of another distribution, the REQUESTED file should not be created. The <code class="docutils literal notranslate">install</code> command of distutils by default creates the REQUESTED file. It accepts <code class="docutils literal notranslate">--requested</code> and <code class="docutils literal notranslate">--no-requested</code> options to explicitly specify whether the file is created. If a distribution that was already installed on the system as a dependency is later installed by name, the distutils <code class="docutils literal notranslate">install</code> command will create the REQUESTED file in the .dist-info directory of the existing installation. </section> </section> <section id="implementation-details"> <h2><a class="toc-backref" href="#implementation-details" role="doc-backlink">Implementation details</a></h2> Note: this section is non-normative. In the end, this PEP was implemented by third-party libraries and tools, not the standard library. <section id="new-functions-and-classes-in-pkgutil"> <h3><a class="toc-backref" href="#new-functions-and-classes-in-pkgutil" role="doc-backlink">New functions and classes in pkgutil</a></h3> To use the <code class="docutils literal notranslate">.dist-info</code> directory content, we need to add in the standard library a set of APIs. The best place to put these APIs is <code class="docutils literal notranslate">pkgutil</code>. <section id="functions"> <h4><a class="toc-backref" href="#functions" role="doc-backlink">Functions</a></h4> The new functions added in the <code class="docutils literal notranslate">pkgutil</code> module are : <ul> <li><code class="docutils literal notranslate">distinfo_dirname(name, version)</code> -> directory name<blockquote> <div><code class="docutils literal notranslate">name</code> is converted to a standard distribution name by replacing any runs of non-alphanumeric characters with a single ‘-‘.<code class="docutils literal notranslate">version</code> is converted to a standard version string. Spaces become dots, and all other non-alphanumeric characters (except dots) become dashes, with runs of multiple dashes condensed to a single dash. Both attributes are then converted into their filename-escaped form, i.e. any ‘-’ characters are replaced with ‘_’ other than the one in ‘dist-info’ and the one separating the name from the version number. </div></blockquote> </li> <li><code class="docutils literal notranslate">get_distributions()</code> -> iterator of <code class="docutils literal notranslate">Distribution</code> instances.Provides an iterator that looks for <code class="docutils literal notranslate">.dist-info</code> directories in <code class="docutils literal notranslate">sys.path</code> and returns <code class="docutils literal notranslate">Distribution</code> instances for each one of them. </li> <li><code class="docutils literal notranslate">get_distribution(name)</code> -> <code class="docutils literal notranslate">Distribution</code> or None.</li> <li><code class="docutils literal notranslate">obsoletes_distribution(name, version=None)</code> -> iterator of <code class="docutils literal notranslate">Distribution</code> instances.Iterates over all distributions to find which distributions obsolete <code class="docutils literal notranslate">name</code>. If a <code class="docutils literal notranslate">version</code> is provided, it will be used to filter the results. </li> <li><code class="docutils literal notranslate">provides_distribution(name, version=None)</code> -> iterator of <code class="docutils literal notranslate">Distribution</code> instances.Iterates over all distributions to find which distributions provide <code class="docutils literal notranslate">name</code>. If a <code class="docutils literal notranslate">version</code> is provided, it will be used to filter the results. Scans all elements in <code class="docutils literal notranslate">sys.path</code> and looks for all directories ending with <code class="docutils literal notranslate">.dist-info</code>. Returns a <code class="docutils literal notranslate">Distribution</code> corresponding to the <code class="docutils literal notranslate">.dist-info</code> directory that contains a METADATA that matches <code class="docutils literal notranslate">name</code> for the <code class="docutils literal notranslate">name</code> metadata. This function only returns the first result founded, since no more than one values are expected. If the directory is not found, returns None. </li> <li><code class="docutils literal notranslate">get_file_users(path)</code> -> iterator of <code class="docutils literal notranslate">Distribution</code> instances.Iterates over all distributions to find out which distributions uses <code class="docutils literal notranslate">path</code>. <code class="docutils literal notranslate">path</code> can be a local absolute path or a relative ‘/’-separated path. A local absolute path is an absolute path in which occurrences of ‘/’ have been replaced by the system separator given by <code class="docutils literal notranslate">os.sep</code>. </li> </ul> </section> <section id="distribution-class"> <h4><a class="toc-backref" href="#distribution-class" role="doc-backlink">Distribution class</a></h4> A new class called <code class="docutils literal notranslate">Distribution</code> is created with the path of the <code class="docutils literal notranslate">.dist-info</code> directory provided to the constructor. It reads the metadata contained in <code class="docutils literal notranslate">METADATA</code> when it is instantiated. <code class="docutils literal notranslate">Distribution(path)</code> -> instance <blockquote> <div>Creates a <code class="docutils literal notranslate">Distribution</code> instance for the given <code class="docutils literal notranslate">path</code>.</div></blockquote> <code class="docutils literal notranslate">Distribution</code> provides the following attributes: <ul class="simple"> <li><code class="docutils literal notranslate">name</code>: The name of the distribution.</li> <li><code class="docutils literal notranslate">metadata</code>: A <code class="docutils literal notranslate">DistributionMetadata</code> instance loaded with the distribution’s METADATA file.</li> <li><code class="docutils literal notranslate">requested</code>: A boolean that indicates whether the REQUESTED metadata file is present (in other words, whether the distribution was installed by user request).</li> </ul> And following methods: <ul> <li><code class="docutils literal notranslate">get_installed_files(local=False)</code> -> iterator of (path, hash, size)Iterates over the <code class="docutils literal notranslate">RECORD</code> entries and return a tuple <code class="docutils literal notranslate">(path, hash, size)</code> for each line. If <code class="docutils literal notranslate">local</code> is <code class="docutils literal notranslate">True</code>, the path is transformed into a local absolute path. Otherwise the raw value from <code class="docutils literal notranslate">RECORD</code> is returned. A local absolute path is an absolute path in which occurrences of ‘/’ have been replaced by the system separator given by <code class="docutils literal notranslate">os.sep</code>. </li> <li><code class="docutils literal notranslate">uses(path)</code> -> BooleanReturns <code class="docutils literal notranslate">True</code> if <code class="docutils literal notranslate">path</code> is listed in <code class="docutils literal notranslate">RECORD</code>. <code class="docutils literal notranslate">path</code> can be a local absolute path or a relative ‘/’-separated path. </li> <li><code class="docutils literal notranslate">get_distinfo_file(path, binary=False)</code> -> file object<blockquote> <div>Returns a file located under the <code class="docutils literal notranslate">.dist-info</code> directory.Returns a <code class="docutils literal notranslate">file</code> instance for the file pointed by <code class="docutils literal notranslate">path</code>. <code class="docutils literal notranslate">path</code> has to be a ‘/’-separated path relative to the <code class="docutils literal notranslate">.dist-info</code> directory or an absolute path. If <code class="docutils literal notranslate">path</code> is an absolute path and doesn’t start with the <code class="docutils literal notranslate">.dist-info</code> directory path, a <code class="docutils literal notranslate">DistutilsError</code> is raised. If <code class="docutils literal notranslate">binary</code> is <code class="docutils literal notranslate">True</code>, opens the file in read-only binary mode (<code class="docutils literal notranslate">rb</code>), otherwise opens it in read-only mode (<code class="docutils literal notranslate">r</code>). </div></blockquote> </li> <li><code class="docutils literal notranslate">get_distinfo_files(local=False)</code> -> iterator of pathsIterates over the <code class="docutils literal notranslate">RECORD</code> entries and returns paths for each line if the path is pointing to a file located in the <code class="docutils literal notranslate">.dist-info</code> directory or one of its subdirectories. If <code class="docutils literal notranslate">local</code> is <code class="docutils literal notranslate">True</code>, each path is transformed into a local absolute path. Otherwise the raw value from <code class="docutils literal notranslate">RECORD</code> is returned. </li> </ul> Notice that the API is organized in five classes that work with directories and Zip files (so it works with files included in Zip files, see <a class="pep reference internal" href="../pep-0273/" title="PEP 273 – Import Modules from Zip Archives">PEP 273</a> for more details). These classes are described in the documentation of the prototype implementation for interested readers <a class="footnote-reference brackets" href="#prototype" id="id7">[7]</a>. </section> <section id="examples"> <h4><a class="toc-backref" href="#examples" role="doc-backlink">Examples</a></h4> Let’s use some of the new APIs with our <code class="docutils literal notranslate">docutils</code> example: <div class="highlight-default notranslate"><div class="highlight"><pre>>>> from pkgutil import get_distribution, get_file_users, distinfo_dirname >>> dist = get_distribution('docutils') >>> dist.name 'docutils' >>> dist.metadata.version '0.5' >>> distinfo_dirname('docutils', '0.5') 'docutils-0.5.dist-info' >>> distinfo_dirname('python-ldap', '2.5') 'python_ldap-2.5.dist-info' >>> distinfo_dirname('python-ldap', '2.5 a---5') 'python_ldap-2.5.a_5.dist-info' >>> for path, hash, size in dist.get_installed_files():: ... print '%s %s %d' % (path, hash, size) ... python2.6/site-packages/docutils/__init__.py,b690274f621402dda63bf11ba5373bf2,9544 python2.6/site-packages/docutils/core.py,9c4b84aff68aa55f2e9bf70481b94333,66188 python2.6/site-packages/roman.py,a4b84aff68aa55f2e9bf70481b943D3,234 /usr/local/bin/rst2html.py,a4b84aff68aa55f2e9bf70481b943D3,234 python2.6/site-packages/docutils-0.5.dist-info/METADATA,6fe57de576d749536082d8e205b77748,195 python2.6/site-packages/docutils-0.5.dist-info/RECORD >>> dist.uses('docutils/core.py') True >>> dist.uses('/usr/local/bin/rst2html.py') True >>> dist.get_distinfo_file('METADATA') <open file at ...> >>> dist.requested True </pre></div> </div> </section> </section> <section id="new-functions-in-distutils"> <h3><a class="toc-backref" href="#new-functions-in-distutils" role="doc-backlink">New functions in Distutils</a></h3> Distutils already provides a very basic way to install a distribution, which is running the <code class="docutils literal notranslate">install</code> command over the <code class="docutils literal notranslate">setup.py</code> script of the distribution. <a class="pep reference internal" href="../pep-0262/" title="PEP 262 – A Database of Installed Python Packages">Distutils2</a> will provide a very basic <code class="docutils literal notranslate">uninstall</code> function, that is added in <code class="docutils literal notranslate">distutils2.util</code> and takes the name of the distribution to uninstall as its argument. <code class="docutils literal notranslate">uninstall</code> uses the APIs described earlier and remove all unique files, as long as their hash didn’t change. Then it removes empty directories left behind. <code class="docutils literal notranslate">uninstall</code> returns a list of uninstalled files: <div class="highlight-default notranslate"><div class="highlight"><pre>>>> from distutils2.util import uninstall >>> uninstall('docutils') ['/opt/local/lib/python2.6/site-packages/docutils/core.py', ... '/opt/local/lib/python2.6/site-packages/docutils/__init__.py'] </pre></div> </div> If the distribution is not found, a <code class="docutils literal notranslate">DistutilsUninstallError</code> is raised. <section id="filtering"> <h4><a class="toc-backref" href="#filtering" role="doc-backlink">Filtering</a></h4> To make it a reference API for third-party projects that wish to control how <code class="docutils literal notranslate">uninstall</code> works, a second callable argument can be used. It’s called for each file that is removed. If the callable returns <code class="docutils literal notranslate">True</code>, the file is removed. If it returns False, it’s left alone. Examples: <div class="highlight-default notranslate"><div class="highlight"><pre>>>> def _remove_and_log(path): ... logging.info('Removing %s' % path) ... return True ... >>> uninstall('docutils', _remove_and_log) >>> def _dry_run(path): ... logging.info('Removing %s (dry run)' % path) ... return False ... >>> uninstall('docutils', _dry_run) </pre></div> </div> Of course, a third-party tool can use lower-level <code class="docutils literal notranslate">pkgutil</code> APIs to implement its own uninstall feature. </section> <section id="installer-marker"> <h4><a class="toc-backref" href="#installer-marker" role="doc-backlink">Installer marker</a></h4> As explained earlier in this PEP, the <code class="docutils literal notranslate">install</code> command adds an <code class="docutils literal notranslate">INSTALLER</code> file in the <code class="docutils literal notranslate">.dist-info</code> directory with the name of the installer. To avoid removing distributions that were installed by another packaging system, the <code class="docutils literal notranslate">uninstall</code> function takes an extra argument <code class="docutils literal notranslate">installer</code> which defaults to <code class="docutils literal notranslate">distutils2</code>. When called, <code class="docutils literal notranslate">uninstall</code> controls that the <code class="docutils literal notranslate">INSTALLER</code> file matches this argument. If not, it raises a <code class="docutils literal notranslate">DistutilsUninstallError</code>: <div class="highlight-default notranslate"><div class="highlight"><pre>>>> uninstall('docutils') Traceback (most recent call last): ... DistutilsUninstallError: docutils was installed by 'cool-pkg-manager' >>> uninstall('docutils', installer='cool-pkg-manager') </pre></div> </div> This allows a third-party application to use the <code class="docutils literal notranslate">uninstall</code> function and strongly suggest that no other program remove a distribution it has previously installed. This is useful when a third-party program that relies on Distutils APIs does extra steps on the system at installation time, it has to undo at uninstallation time. </section> <section id="adding-an-uninstall-script"> <h4><a class="toc-backref" href="#adding-an-uninstall-script" role="doc-backlink">Adding an Uninstall script</a></h4> An <code class="docutils literal notranslate">uninstall</code> script is added in Distutils2. and is used like this: <div class="highlight-default notranslate"><div class="highlight"><pre>$ python -m distutils2.uninstall projectname </pre></div> </div> Notice that script doesn’t control if the removal of a distribution breaks another distribution. Although it makes sure that all the files it removes are not used by any other distribution, by using the uninstall function. Also note that this uninstall script pays no attention to the REQUESTED metadata; that is provided only for use by external tools to provide more advanced dependency management. </section> </section> </section> <section id="backward-compatibility-and-roadmap"> <h2><a class="toc-backref" href="#backward-compatibility-and-roadmap" role="doc-backlink">Backward compatibility and roadmap</a></h2> These changes don’t introduce any compatibility problems since they will be implemented in: <ul class="simple"> <li>pkgutil in new functions</li> <li>distutils2</li> </ul> The plan is to include the functionality outlined in this PEP in pkgutil for Python 3.2, and in Distutils2. Distutils2 will also contain a backport of the new pgkutil, and can be used for 2.4 onward. Distributions installed using existing, pre-standardization formats do not have the necessary metadata available for the new API, and thus will be ignored. Third-party tools may of course to continue to support previous formats in addition to the new format, in order to ease the transition. </section> <section id="references"> <h2><a class="toc-backref" href="#references" role="doc-backlink">References</a></h2> <aside class="footnote-list brackets"> <aside class="footnote brackets" id="setuptools" role="doc-footnote"> <dt class="label" id="setuptools">[<a href="#id1">1</a>]</dt> <dd><a class="reference external" href="https://peak.telecommunity.com/DevCenter/setuptools">https://peak.telecommunity.com/DevCenter/setuptools</a></aside> <aside class="footnote brackets" id="easyinstall" role="doc-footnote"> <dt class="label" id="easyinstall">[<a href="#id3">2</a>]</dt> <dd><a class="reference external" href="https://peak.telecommunity.com/DevCenter/EasyInstall">https://peak.telecommunity.com/DevCenter/EasyInstall</a></aside> <aside class="footnote brackets" id="pip" role="doc-footnote"> <dt class="label" id="pip">[<a href="#id4">3</a>]</dt> <dd><a class="reference external" href="https://pypi.org/project/pip/">https://pypi.org/project/pip/</a></aside> <aside class="footnote brackets" id="eggformats" role="doc-footnote"> <dt class="label" id="eggformats">[<a href="#id2">4</a>]</dt> <dd><a class="reference external" href="https://peak.telecommunity.com/DevCenter/EggFormats">https://peak.telecommunity.com/DevCenter/EggFormats</a></aside> <aside class="footnote brackets" id="fedora" role="doc-footnote"> <dt class="label" id="fedora">[<a href="#id5">5</a>]</dt> <dd><a class="reference external" href="https://fedoraproject.org/wiki/Packaging/Python/Eggs#Providing_Eggs_using_Setuptools">https://fedoraproject.org/wiki/Packaging/Python/Eggs#Providing_Eggs_using_Setuptools</a></aside> <aside class="footnote brackets" id="debian" role="doc-footnote"> <dt class="label" id="debian">[<a href="#id6">6</a>]</dt> <dd><a class="reference external" href="https://wiki.debian.org/DebianPython/NewPolicy">https://wiki.debian.org/DebianPython/NewPolicy</a></aside> <aside class="footnote brackets" id="prototype" role="doc-footnote"> <dt class="label" id="prototype">[<a href="#id7">7</a>]</dt> <dd><a class="reference external" href="https://web.archive.org/web/20090726092550/http://bitbucket.org/tarek/pep376/">https://web.archive.org/web/20090726092550/http://bitbucket.org/tarek/pep376/</a></aside> </aside> </section> <section id="acknowledgements"> <h2><a class="toc-backref" href="#acknowledgements" role="doc-backlink">Acknowledgements</a></h2> Jim Fulton, Ian Bicking, Phillip Eby, Rafael Villar Burke, and many people at Pycon and Distutils-SIG. </section> <section id="copyright"> <h2><a class="toc-backref" href="#copyright" role="doc-backlink">Copyright</a></h2> This document has been placed in the public domain. </section> </section> <hr class="docutils" /> Source: <a class="reference external" href="https://github.com/python/peps/blob/main/peps/pep-0376.rst">https://github.com/python/peps/blob/main/peps/pep-0376.rst</a> Last modified: <a class="reference external" href="https://github.com/python/peps/commits/main/peps/pep-0376.rst">2024-12-15 20:57:13 GMT</a> </article> <nav id="pep-sidebar"> <h2>Contents</h2> <ul> <li><a class="reference internal" href="#abstract">Abstract</a></li> <li><a class="reference internal" href="#rationale">Rationale</a><ul> <li><a class="reference internal" href="#how-distributions-are-installed">How distributions are installed</a></li> <li><a class="reference internal" href="#uninstall-information">Uninstall information</a></li> <li><a class="reference internal" href="#what-this-pep-proposes">What this PEP proposes</a></li> </ul> </li> <li><a class="reference internal" href="#one-dist-info-directory-per-installed-distribution">One .dist-info directory per installed distribution</a><ul> <li><a class="reference internal" href="#record">RECORD</a></li> <li><a class="reference internal" href="#installer">INSTALLER</a></li> <li><a class="reference internal" href="#requested">REQUESTED</a></li> </ul> </li> <li><a class="reference internal" href="#implementation-details">Implementation details</a><ul> <li><a class="reference internal" href="#new-functions-and-classes-in-pkgutil">New functions and classes in pkgutil</a><ul> <li><a class="reference internal" href="#functions">Functions</a></li> <li><a class="reference internal" href="#distribution-class">Distribution class</a></li> <li><a class="reference internal" href="#examples">Examples</a></li> </ul> </li> <li><a class="reference internal" href="#new-functions-in-distutils">New functions in Distutils</a><ul> <li><a class="reference internal" href="#filtering">Filtering</a></li> <li><a class="reference internal" href="#installer-marker">Installer marker</a></li> <li><a class="reference internal" href="#adding-an-uninstall-script">Adding an Uninstall script</a></li> </ul> </li> </ul> </li> <li><a class="reference internal" href="#backward-compatibility-and-roadmap">Backward compatibility and roadmap</a></li> <li><a class="reference internal" href="#references">References</a></li> <li><a class="reference internal" href="#acknowledgements">Acknowledgements</a></li> <li><a class="reference internal" href="#copyright">Copyright</a></li> </ul> <a id="source" href="https://github.com/python/peps/blob/main/peps/pep-0376.rst">Page Source (GitHub)</a> </nav> </section> <script src="../_static/colour_scheme.js"></script> <script src="../_static/wrap_tables.js"></script> <script src="../_static/sticky_banner.js"></script> </body> </html>