CINXE.COM

<!DOCTYPE html> <html lang="en"> <head> <meta charset="utf-8"> <meta name="viewport" content="width=device-width, initial-scale=1.0"> <meta name="color-scheme" content="light dark"> <title>PEP 759 – External Wheel Hosting | peps.python.org</title> <link rel="shortcut icon" href="../_static/py.png"> <link rel="canonical" href="https://peps.python.org/pep-0759/"> <link rel="stylesheet" href="../_static/style.css" type="text/css"> <link rel="stylesheet" href="../_static/mq.css" type="text/css"> <link rel="stylesheet" href="../_static/pygments.css" type="text/css" media="(prefers-color-scheme: light)" id="pyg-light"> <link rel="stylesheet" href="../_static/pygments_dark.css" type="text/css" media="(prefers-color-scheme: dark)" id="pyg-dark"> <link rel="alternate" type="application/rss+xml" title="Latest PEPs" href="https://peps.python.org/peps.rss"> <meta property="og:title" content='PEP 759 – External Wheel Hosting | peps.python.org'> <meta property="og:description" content="This PEP proposes a mechanism by which projects hosted on pypi.org can safely host wheel artifacts on external sites other than PyPI. This PEP explicitly does not propose external hosting of projects, packages, or their metadata. That functionality is a..."> <meta property="og:type" content="website"> <meta property="og:url" content="https://peps.python.org/pep-0759/"> <meta property="og:site_name" content="Python Enhancement Proposals (PEPs)"> <meta property="og:image" content="https://peps.python.org/_static/og-image.png"> <meta property="og:image:alt" content="Python PEPs"> <meta property="og:image:width" content="200"> <meta property="og:image:height" content="200"> <meta name="description" content="This PEP proposes a mechanism by which projects hosted on pypi.org can safely host wheel artifacts on external sites other than PyPI. This PEP explicitly does not propose external hosting of projects, packages, or their metadata. That functionality is a..."> <meta name="theme-color" content="#3776ab"> </head> <body> <svg xmlns="http://www.w3.org/2000/svg" style="display: none;"> <symbol id="svg-sun-half" viewBox="0 0 24 24" pointer-events="all"> <title>Following system colour scheme</title> <svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"> <circle cx="12" cy="12" r="9"></circle> <path d="M12 3v18m0-12l4.65-4.65M12 14.3l7.37-7.37M12 19.6l8.85-8.85"></path> </svg> </symbol> <symbol id="svg-moon" viewBox="0 0 24 24" pointer-events="all"> <title>Selected dark colour scheme</title> <svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"> <path stroke="none" d="M0 0h24v24H0z" fill="none"></path> <path d="M12 3c.132 0 .263 0 .393 0a7.5 7.5 0 0 0 7.92 12.446a9 9 0 1 1 -8.313 -12.454z"></path> </svg> </symbol> <symbol id="svg-sun" viewBox="0 0 24 24" pointer-events="all"> <title>Selected light colour scheme</title> <svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"> <circle cx="12" cy="12" r="5"></circle> <line x1="12" y1="1" x2="12" y2="3"></line> <line x1="12" y1="21" x2="12" y2="23"></line> <line x1="4.22" y1="4.22" x2="5.64" y2="5.64"></line> <line x1="18.36" y1="18.36" x2="19.78" y2="19.78"></line> <line x1="1" y1="12" x2="3" y2="12"></line> <line x1="21" y1="12" x2="23" y2="12"></line> <line x1="4.22" y1="19.78" x2="5.64" y2="18.36"></line> <line x1="18.36" y1="5.64" x2="19.78" y2="4.22"></line> </svg> </symbol> </svg> <script> document.documentElement.dataset.colour_scheme = localStorage.getItem("colour_scheme") || "auto" </script> <section id="pep-page-section"> <header> <h1>Python Enhancement Proposals</h1> <ul class="breadcrumbs"> <li><a href="https://www.python.org/" title="The Python Programming Language">Python</a> » </li> <li><a href="../pep-0000/">PEP Index</a> » </li> <li>PEP 759</li> </ul> <button id="colour-scheme-cycler" onClick="setColourScheme(nextColourScheme())"> <svg aria-hidden="true" class="colour-scheme-icon-when-auto"><use href="#svg-sun-half"></use></svg> <svg aria-hidden="true" class="colour-scheme-icon-when-dark"><use href="#svg-moon"></use></svg> <svg aria-hidden="true" class="colour-scheme-icon-when-light"><use href="#svg-sun"></use></svg> <span class="visually-hidden">Toggle light / dark / auto colour theme</span> </button> </header> <article> <section id="pep-content"> <h1 class="page-title">PEP 759 – External Wheel Hosting</h1> <dl class="rfc2822 field-list simple"> <dt class="field-odd">Author<span class="colon">:</span></dt> <dd class="field-odd">Barry Warsaw <barry at python.org>, Emma Harper Smith <emma at python.org></dd> <dt class="field-even">PEP-Delegate<span class="colon">:</span></dt> <dd class="field-even">Donald Stufft <donald at python.org></dd> <dt class="field-odd">Discussions-To<span class="colon">:</span></dt> <dd class="field-odd"><a class="reference external" href="https://discuss.python.org/t/pep-759-external-wheel-hosting/66458">Discourse thread</a></dd> <dt class="field-even">Status<span class="colon">:</span></dt> <dd class="field-even"><abbr title="Removed from consideration by sponsor or authors">Withdrawn</abbr></dd> <dt class="field-odd">Type<span class="colon">:</span></dt> <dd class="field-odd"><abbr title="Normative PEP with a new feature for Python, implementation change for CPython or interoperability standard for the ecosystem">Standards Track</abbr></dd> <dt class="field-even">Topic<span class="colon">:</span></dt> <dd class="field-even"><a class="reference external" href="../topic/packaging/">Packaging</a></dd> <dt class="field-odd">Created<span class="colon">:</span></dt> <dd class="field-odd">01-Oct-2024</dd> <dt class="field-even">Post-History<span class="colon">:</span></dt> <dd class="field-even">10-Oct-2024, 31-Jan-2025</dd> <dt class="field-odd">Resolution<span class="colon">:</span></dt> <dd class="field-odd"><a class="reference external" href="https://discuss.python.org/t/pep-759-external-wheel-hosting/66458/48">31-Jan-2025</a></dd> </dl> <hr class="docutils" /> <section id="contents"> <details><summary>Table of Contents</summary><ul class="simple"> <li><a class="reference internal" href="#abstract">Abstract</a></li> <li><a class="reference internal" href="#pep-withdrawn">PEP withdrawn</a></li> <li><a class="reference internal" href="#rationale">Rationale</a><ul> <li><a class="reference internal" href="#historical-context">Historical context</a></li> <li><a class="reference internal" href="#the-problem-with-multiple-indexes">The problem with multiple indexes</a></li> <li><a class="reference internal" href="#addressing-pypi-limits">Addressing PyPI limits</a></li> <li><a class="reference internal" href="#reducing-operational-complexity">Reducing operational complexity</a></li> </ul> </li> <li><a class="reference internal" href="#specification">Specification</a><ul> <li><a class="reference internal" href="#effects-of-the-rim-file">Effects of the RIM file</a></li> <li><a class="reference internal" href="#availability-order">Availability order</a></li> <li><a class="reference internal" href="#wheels-can-override-rims">Wheels can override RIMs</a></li> <li><a class="reference internal" href="#pypi-api-bump-unnecessary">PyPI API bump unnecessary</a></li> </ul> </li> <li><a class="reference internal" href="#external-hosting-resiliency">External hosting resiliency</a></li> <li><a class="reference internal" href="#dismounting-wheels">Dismounting wheels</a></li> <li><a class="reference internal" href="#changes-to-tools">Changes to tools</a></li> <li><a class="reference internal" href="#constraints-for-external-hosting-services">Constraints for external hosting services</a></li> <li><a class="reference internal" href="#security">Security</a></li> <li><a class="reference internal" href="#rejected-ideas">Rejected ideas</a></li> <li><a class="reference internal" href="#footnotes">Footnotes</a></li> <li><a class="reference internal" href="#copyright">Copyright</a></li> </ul> </details></section> <section id="abstract"> <h2><a class="toc-backref" href="#abstract" role="doc-backlink">Abstract</a></h2> <p>This PEP proposes a mechanism by which projects hosted on <a class="reference external" href="https://pypi.org">pypi.org</a> can safely host wheel artifacts on external sites other than PyPI. This PEP explicitly does <em>not</em> propose external hosting of projects, packages, or their metadata. That functionality is already available by externally hosting independent package indexes. Because this PEP only provides a mechanism for projects to customize the download URL for specific released wheel artifacts, dependency resolution as already implemented by common installer tools such as <a class="reference external" href="https://pip.pypa.io/en/stable/">pip</a> and <a class="reference external" href="https://docs.astral.sh/uv/">uv</a> does not need to change.</p> <p>This PEP defines what it means to be “safe” in this context, along with a new package upload file format called a <code class="docutils literal notranslate"><span class="pre">.rim</span></code> file. It defines how <code class="docutils literal notranslate"><span class="pre">.rim</span></code> files affect the metadata returned for a package’s <a class="reference external" href="https://packaging.python.org/en/latest/specifications/simple-repository-api/#simple-repository-api" title="(in Python Packaging User Guide)"><span class="xref std std-ref">Simple Repository API</span></a> in both HTML and JSON formats, and how traditional wheels can easily be turned into <code class="docutils literal notranslate"><span class="pre">.rim</span></code> files.</p> </section> <section id="pep-withdrawn"> <h2><a class="toc-backref" href="#pep-withdrawn" role="doc-backlink">PEP withdrawn</a></h2> <p>This PEP was withdrawn by the authors on 2025-01-31. Our reading of the sentiment in the discussion thread is that, while the problems this PEP attempts to solve are valid, most folks would prefer a different approach. Specifically, our read is that most users would prefer more control over the ability to specify multiple indexes, how those indexes interoperate, and the priority and trust assertions for those indexes. For example, solutions such as <a class="pep reference internal" href="../pep-0766/" title="PEP 766 – Explicit Priority Choices Among Multiple Indexes">PEP 766</a> may provide a better way forward. Existing stop gap measures (e.g. PyPI limit increase requests and the <a class="reference external" href="https://pypi.org/project/wheel-stub/">“wheel-stub”</a> approach) are sufficient – if not ideal – in the meantime. The authors wish to thank everyone who contributed to the constructive discussion, and especially those who showed their support for this PEP, both in public and private.</p> </section> <section id="rationale"> <h2><a class="toc-backref" href="#rationale" role="doc-backlink">Rationale</a></h2> <p>The Python Package Index, hosted at <a class="reference external" href="https://pypi.org">https://pypi.org</a>, imposes <a class="reference external" href="https://pypi.org/help/">default limits</a> on upload artifact file size (100 MiB) and total project size (10 GiB). Most projects can comfortably fit within these limits during the lifetime of the project, through years of uploads. A few projects have encountered these limits, and have been granted both file size and project size exceptions, allowing them to continue uploading new releases without having to take more drastic measures, such as removing files which may potentially still be in use by consumers (e.g. through version pins).</p> <p>A related workaround is the <a class="reference external" href="https://github.com/wheel-next/wheel-stub">“wheel stub”</a> approach, which provides an indirect link between PyPI and an external third party package index, where such limitations can be avoided. Wheel stubs are <a class="reference external" href="https://packaging.python.org/en/latest/specifications/source-distribution-format/#source-distribution-format" title="(in Python Packaging User Guide)"><span class="xref std std-ref">source distributions</span></a> (a.k.a. “sdists”) which utilize a <a class="pep reference internal" href="../pep-0517/" title="PEP 517 – A build-system independent format for source trees">PEP 517</a> build backend that, instead of turning source code into a binary wheel, performs some logic to calculate the URL for an existing, externally hosted wheel to download and install. This approach works, but it obscures the connection between PyPI, the sdist, and the externally hosted wheel, since there is no way to present this information to <code class="docutils literal notranslate"><span class="pre">pip</span></code> or other such tools.</p> <section id="historical-context"> <h3><a class="toc-backref" href="#historical-context" role="doc-backlink">Historical context</a></h3> <p>In 2013, <a class="pep reference internal" href="../pep-0438/" title="PEP 438 – Transitioning to release-file hosting on PyPI">PEP 438</a> proposed a “backward-compatible two-phase transition process” to modify several aspects of release file hosting on PyPI. As this PEP describes, PyPI originally supported only project and release <em>registration</em> without also allowing for artifact file hosting. As such, most projects hosted release file artifacts elsewhere. Artifact hosting was later added, but the mix of externally and PyPI-hosted files led to a wide range of usability and potential security-related problems. PEP 438 was an attempt to provide several facilities to allow external hosting while promoting a PyPI-first hosting preference.</p> <p>PEP 438 was complex, with three different “hosting modes”, <code class="docutils literal notranslate"><span class="pre">rel</span></code> metadata in the simple HTML index pages to signify hosting locations, and a two-phase transition plan affecting PyPI and installer tools. PEP 438 was ultimately retracted in 2015 by <a class="pep reference internal" href="../pep-0470/" title="PEP 470 – Removing External Hosting Support on PyPI">PEP 470</a>, which acknowledges that PEP 438 did succeed in…</p> <blockquote> <div>bringing about more people to utilize PyPI’s repository features, an altogether good thing given the global CDN powering PyPI providing speed ups for a lot of people[…]</div></blockquote> <p>Instead of external hosting, PEP 470 promoted the use of explicit multiple repositories, providing full package indexing and artifact hosting, and enabled through installer tool support, such as <code class="docutils literal notranslate"><span class="pre">pip</span> <span class="pre">install</span> <span class="pre">--extra-index-url</span></code> allowing <code class="docutils literal notranslate"><span class="pre">pip</span></code> to essentially treat multiple repositories as <a class="reference external" href="https://pip.pypa.io/en/stable/cli/pip_install/#cmdoption-extra-index-url">one single global repository</a> for package installation resolution. Because this has been the blessed norm for so many years, all Python package installation tools support querying multiple indexes for dependency resolution.</p> </section> <section id="the-problem-with-multiple-indexes"> <h3><a class="toc-backref" href="#the-problem-with-multiple-indexes" role="doc-backlink">The problem with multiple indexes</a></h3> <p>Why then does this PEP propose to allow a more limited form of external hosting, and how does this proposal avoid the problems documented in PEP 470?</p> <p>One well-known problem that consolidating multiple indexes enables is <a class="reference external" href="https://medium.com/@alex.birsan/dependency-confusion-4a5d60fec610">dependency confusion attacks</a>, to which Python <em>can</em> be particularly vulnerable, due to the algorithm that <code class="docutils literal notranslate"><span class="pre">pip</span> <span class="pre">install</span></code> uses for resolving package dependencies and preferred versions. The <code class="docutils literal notranslate"><span class="pre">uv</span></code> tool addresses this by supporting an additional <a class="reference external" href="https://docs.astral.sh/uv/reference/settings/#index-strategy">index strategy</a> option, whereby users can select between, e.g. a <code class="docutils literal notranslate"><span class="pre">pip</span></code>-compatible strategy, and a more limited strategy that prevents such dependency confusion attacks.</p> <p><a class="pep reference internal" href="../pep-0708/" title="PEP 708 – Extending the Repository API to Mitigate Dependency Confusion Attacks">PEP 708</a> provides additional background about dependency confusion attacks, and takes a different approach to preventing them. At its core, PEP 708 allows repository owners to indicate that projects track across different repositories, which allows installers to determine how to treat the global package namespace when combined across multiple repositories. PEP 708 has been provisionally accepted, pending several required conditions as outlined in PEP 708, some of which may have an indeterminate future. As PEP 708 itself says, this won’t by itself solve dependency confusion attacks, but is one way to provide enough information to installers to help minimize these attacks.</p> <p>While there can still be valid use cases for standing up a totally independent package index (such as providing richer platform support for GPUs until a fully formed <a class="reference external" href="https://discuss.python.org/t/selecting-variant-wheels-according-to-a-semi-static-specification/53446">variant proposal</a> is accepted), this PEP takes a different, simpler approach and doesn’t replace any of the existing, proposed, or approved package index cooperation specifications.</p> <p>This PEP also preserves the core purpose of PyPI, and allows it to remain the traditional, canonical, centralized index of all Python packages.</p> </section> <section id="addressing-pypi-limits"> <h3><a class="toc-backref" href="#addressing-pypi-limits" role="doc-backlink">Addressing PyPI limits</a></h3> <p>This proposal also addresses the problem of size limits imposed by PyPI, where there is a <a class="reference external" href="https://pypi.org/help/#file-size-limit">default artifact size limit</a> of 100 MiB and a default overall <a class="reference external" href="https://pypi.org/help/#project-size-limit">project size limit</a> of 10 GiB. Most packages and artifacts can easily fit in these limits, even for packages containing binary extension modules for a variety of platforms. A small, but important class of packages routinely exceed these limits, requiring them to submit PyPI <a class="reference external" href="https://github.com/pypi/support/issues?q=is%3Aissue+is%3Aclosed+file+limit+request">exception request support tickets</a>. It’s not necessarily difficult to get resolution on such exceptions, but it is a special process that can take some time to resolve, and the criteria for granting such exceptions aren’t well documented.</p> </section> <section id="reducing-operational-complexity"> <h3><a class="toc-backref" href="#reducing-operational-complexity" role="doc-backlink">Reducing operational complexity</a></h3> <p>Setting up and maintaining an entire package index can be a complex operational solution, both time and resource intensive. This is especially true if the main purpose of such an index is just to avoid file size limitations. The external index approach also imposes a tricky UX on consumers of projects on the external index, requiring them to understand how CLI options such as <code class="docutils literal notranslate"><span class="pre">--external-index-url</span></code> work, along with the security implications of such flags. It would be much easier for both producers and consumers of large wheel packages to just set up and maintain a simple web server, capable of serving individual files with no more complex API than <code class="docutils literal notranslate"><span class="pre">HTTP</span> <span class="pre">GET</span></code>. Such an interface is also easily cacheable or placed behind a <a class="reference external" href="https://en.wikipedia.org/wiki/Content_delivery_network">CDN</a>. Simple HTTP servers are also much easier to audit for security purposes, easier to proxy, and usually take much less resources to run, support, and maintain. Even something like <a class="reference external" href="https://aws.amazon.com/s3/">Amazon S3</a> could be used to host external wheels.</p> <p>This PEP proposes an approach that favors such operational simplicity.</p> </section> </section> <section id="specification"> <h2><a class="toc-backref" href="#specification" role="doc-backlink">Specification</a></h2> <p>A new type of uploadable file is defined, called a “RIM” (i.e. <code class="docutils literal notranslate"><span class="pre">.rim</span></code>), or “Remote Installable Metadata” file. The name evokes the image of a wheel with the tire removed, and emphasizes that <code class="docutils literal notranslate"><span class="pre">.rim</span></code> files are easily derived from <code class="docutils literal notranslate"><span class="pre">.whl</span></code> files. The process of turning a <code class="docutils literal notranslate"><span class="pre">.whl</span></code> into a <code class="docutils literal notranslate"><span class="pre">.rim</span></code> is <a class="reference internal" href="#dismounting"><span class="std std-ref">outlined below</span></a>. The file name format exactly matches the <a class="reference external" href="https://packaging.python.org/en/latest/specifications/binary-distribution-format/#wheel-file-name-spec" title="(in Python Packaging User Guide)"><span class="xref std std-ref">wheel file naming format</span></a> specification, except that RIM files use the suffix <code class="docutils literal notranslate"><span class="pre">.rim</span></code>. This means that all the tags used to discriminate <code class="docutils literal notranslate"><span class="pre">.whl</span></code> files also distinguish between different <code class="docutils literal notranslate"><span class="pre">.rim</span></code> files, and thus can be used during dependency resolution steps, exactly as <code class="docutils literal notranslate"><span class="pre">.whl</span></code> files are today. In this respect, <code class="docutils literal notranslate"><span class="pre">.whl</span></code> and <code class="docutils literal notranslate"><span class="pre">.rim</span></code> files are interchangeable.</p> <p>The content of a <code class="docutils literal notranslate"><span class="pre">.rim</span></code> file is <em>nearly</em> identical to <code class="docutils literal notranslate"><span class="pre">.whl</span></code> files, however <code class="docutils literal notranslate"><span class="pre">.rim</span></code> files <strong>MUST</strong> contain only the <code class="docutils literal notranslate"><span class="pre">.dist-info</span></code> directory from a wheel. No other top-level file or directory is allowed in the <code class="docutils literal notranslate"><span class="pre">.rim</span></code> zip file. The <code class="docutils literal notranslate"><span class="pre">.dist-info</span></code> directory <strong>MUST</strong> contain a single additional file in addition to those <a class="reference external" href="https://packaging.python.org/en/latest/specifications/binary-distribution-format/#the-dist-info-directory">allowed</a> in a <code class="docutils literal notranslate"><span class="pre">.whl</span></code> file’s <code class="docutils literal notranslate"><span class="pre">.dist-info</span></code> directory: a file called <code class="docutils literal notranslate"><span class="pre">EXTERNAL-HOSTING.json</span></code>.</p> <p id="file-format">This is a JSON file contains containing the following keys:</p> <dl class="simple"> <dt><code class="docutils literal notranslate"><span class="pre">version</span></code></dt><dd>This is the file format version, which for this PEP <strong>MUST</strong> be <code class="docutils literal notranslate"><span class="pre">1.0</span></code>.</dd> <dt><code class="docutils literal notranslate"><span class="pre">owner</span></code></dt><dd>This <strong>MUST</strong> name the PyPI organization owner of this externally hosted file, for reasons which will be described in <a class="reference internal" href="#resiliency"><span class="std std-ref">detail below</span></a>.</dd> <dt><code class="docutils literal notranslate"><span class="pre">uri</span></code></dt><dd>This is a single URL naming the location of the physical <code class="docutils literal notranslate"><span class="pre">.whl</span></code> file hosted on an external site. This URL <strong>MUST</strong> use the <code class="docutils literal notranslate"><span class="pre">https</span></code> scheme.</dd> <dt><code class="docutils literal notranslate"><span class="pre">size</span></code></dt><dd>This is an integer value describing the size in bytes of the physical <code class="docutils literal notranslate"><span class="pre">.whl</span></code> file on the remote host.</dd> <dt><code class="docutils literal notranslate"><span class="pre">hashes</span></code></dt><dd>This is a dictionary of the format described in <a class="pep reference internal" href="../pep-0694/" title="PEP 694 – Upload 2.0 API for Python Package Indexes">PEP 694</a>, used to capture both the <a class="pep reference internal" href="../pep-0694/#upload-each-file" title="PEP 694 – Upload 2.0 API for Python Package Indexes">PEP 694</a> of the physical <code class="docutils literal notranslate"><span class="pre">.whl</span></code> file, with the same constraints as proposed in that PEP. Since these hashes are immutable once uploaded to PyPI, they serve as a critical validation that the externally hosted wheel hasn’t been corrupted or compromised.</dd> </dl> <section id="effects-of-the-rim-file"> <h3><a class="toc-backref" href="#effects-of-the-rim-file" role="doc-backlink">Effects of the RIM file</a></h3> <p>The only effect of a <code class="docutils literal notranslate"><span class="pre">.rim</span></code> file is to change the download URL for the wheel artifact in both the HTML and JSON interfaces in the <a class="reference external" href="https://packaging.python.org/en/latest/specifications/simple-repository-api/#">simple repository API</a>. In the HTML page for a package release, the <code class="docutils literal notranslate"><span class="pre">href</span></code> attribute <strong>MUST</strong> be the value of the <code class="docutils literal notranslate"><span class="pre">uri</span></code> key, including a <code class="docutils literal notranslate"><span class="pre">#<hashname>=<hashvalue></span></code> fragment. this hash fragment <strong>MUST</strong> be in exactly the same format as described the <a class="pep reference internal" href="../pep-0376/" title="PEP 376 – Database of Installed Python Distributions">PEP 376</a> originated <a class="reference external" href="https://packaging.python.org/en/latest/specifications/binary-distribution-format/#signed-wheel-files">signed wheel file format</a> in the <code class="docutils literal notranslate"><span class="pre">.dist-info/RECORD</span></code> file. The exact same rules for selection of hash algorithm and encoding is used here.</p> <p>Similarly in the <a class="reference external" href="https://packaging.python.org/en/latest/specifications/simple-repository-api/#json-based-simple-api-for-python-package-indexes">JSON response</a> the <code class="docutils literal notranslate"><span class="pre">url</span></code> key pointing to the download file must be the value of the <a class="reference internal" href="#file-format"><span class="std std-ref">uri</span></a> key, and the <code class="docutils literal notranslate"><span class="pre">hashes</span></code> dictionary <strong>MUST</strong> be included with values populated from the <code class="docutils literal notranslate"><span class="pre">hashes</span></code> dictionary provided above.</p> <p>In all other respects, a compliant package index should treat <code class="docutils literal notranslate"><span class="pre">.rim</span></code> files the same as <code class="docutils literal notranslate"><span class="pre">.whl</span></code> files, with some other minor exceptions as outlined below. For example, <code class="docutils literal notranslate"><span class="pre">.rim</span></code> files can be <a class="reference external" href="https://pypi.org/help/#deletion">deleted</a> and yanked (<a class="pep reference internal" href="../pep-0592/" title="PEP 592 – Adding “Yank” Support to the Simple API">PEP 592</a>) just like any <code class="docutils literal notranslate"><span class="pre">.whl</span></code> file, with the exact same semantics (i.e. deletions are permanent). When a <code class="docutils literal notranslate"><span class="pre">.rim</span></code> is deleted, an index <strong>MUST NOT</strong> allow a matching <code class="docutils literal notranslate"><span class="pre">.whl</span></code> or <code class="docutils literal notranslate"><span class="pre">.rim</span></code> file to be (re-)uploaded.</p> </section> <section id="availability-order"> <h3><a class="toc-backref" href="#availability-order" role="doc-backlink">Availability order</a></h3> <p>Externally hosted wheels <strong>MUST</strong> be available before the corresponding <code class="docutils literal notranslate"><span class="pre">.rim</span></code> file is uploaded to PyPI, otherwise a publishing race condition is introduced, although this requirement <strong>MAY</strong> be relaxed for <code class="docutils literal notranslate"><span class="pre">.rim</span></code> files uploaded to a <a class="pep reference internal" href="../pep-0694/" title="PEP 694 – Upload 2.0 API for Python Package Indexes">PEP 694</a> staged release.</p> </section> <section id="wheels-can-override-rims"> <h3><a class="toc-backref" href="#wheels-can-override-rims" role="doc-backlink">Wheels can override RIMs</a></h3> <p>Indexes <strong>MUST</strong> reject <code class="docutils literal notranslate"><span class="pre">.rim</span></code> files if a matching <code class="docutils literal notranslate"><span class="pre">.whl</span></code> file already exists with the exact same file name tags. However, indexes <strong>MAY</strong> accept a <code class="docutils literal notranslate"><span class="pre">.whl</span></code> file if a matching <code class="docutils literal notranslate"><span class="pre">.rim</span></code> file exists, as long as that <code class="docutils literal notranslate"><span class="pre">.rim</span></code> file hasn’t been deleted or yanked. This allows uploaders to replace an externally hosted wheel file with an index hosted wheel file, but the converse is prohibited. Since the default is to host wheels on the same package index that contains the package metadata, it is not allowed to “downgrade” an existing wheel file once uploaded. When a <code class="docutils literal notranslate"><span class="pre">.whl</span></code> replaces a <code class="docutils literal notranslate"><span class="pre">.rim</span></code>, the index <strong>MUST</strong> provide download URLs for the package using its own hosted file service. When uploading the overriding <code class="docutils literal notranslate"><span class="pre">.whl</span></code> file, the package index <strong>MUST</strong> validate the hash from the existing <code class="docutils literal notranslate"><span class="pre">.rim</span></code> file, and these hashes must match or the overriding upload <strong>MUST</strong> be rejected.</p> </section> <section id="pypi-api-bump-unnecessary"> <h3><a class="toc-backref" href="#pypi-api-bump-unnecessary" role="doc-backlink">PyPI API bump unnecessary</a></h3> <p>It’s likely that the changes are backward compatible enough that a bump in the <a class="reference external" href="https://packaging.python.org/en/latest/specifications/simple-repository-api/#versioning-pypi-s-simple-api">PyPI repository version</a> is not necessary. Since <code class="docutils literal notranslate"><span class="pre">.rim</span></code> files are essentially changes only to the upload API, package resolvers and package installers can continue to function with the APIs they’ve always supported.</p> </section> </section> <section id="external-hosting-resiliency"> <span id="resiliency"></span><h2><a class="toc-backref" href="#external-hosting-resiliency" role="doc-backlink">External hosting resiliency</a></h2> <p>One of the key concerns leading to PEP 438’s revocation in PEP 470 was potential user confusion when an external index disappeared. From PEP 470:</p> <blockquote> <div>This confusion comes down to end users of projects not realizing if a project is hosted on PyPI or if it relies on an external service. This often manifests itself when the external service is down but PyPI is not. People will see that PyPI works, and other projects works, but this one specific one does not. They oftentimes do not realize who they need to contact in order to get this fixed or what their remediation steps are.</div></blockquote> <p>While the problem of external wheel hosting service going down is not directly solved by this PEP, several safeguards are in place to greatly reduce the potential burden on PyPI administrators.</p> <p>This PEP thus proposes that:</p> <ul class="simple"> <li>External wheel hosting is only allowed for packages which are owned by <a class="reference external" href="https://docs.pypi.org/organization-accounts/">organization accounts</a>. External hosting is an organization-wide setting.</li> <li>Organization accounts do not automatically gain the ability to externally host wheels; this feature MUST be explicitly enabled by PyPI admins at their discretion. Since this will not be a common request, we don’t expect the overhead to be nearly as burdensome as <a class="pep reference internal" href="../pep-0541/" title="PEP 541 – Package Index Name Retention">PEP 541</a> resolutions, account recovery requests, or even file/project size increase requests. External hosting requests would be handled in the same manner as those requests, i.e. via the <a class="reference external" href="https://github.com/pypi/support">PyPI GitHub support tracker</a>.</li> <li>Organization accounts requesting external wheel hosting <strong>MUST</strong> register their own support contact URI, be it a <code class="docutils literal notranslate"><span class="pre">mailto</span></code> URI for a contact email address, or the URL to the organization’s support tracker. Such a contact URI is optional for organizations which do not avail themselves of external wheel file hosting.</li> </ul> <p>Combined with the <code class="docutils literal notranslate"><span class="pre">EXTERNAL-HOSTING.json</span></code> file’s <code class="docutils literal notranslate"><span class="pre">owner</span></code> key, this allows for installer tools to unambiguously redirect any download errors away from the PyPI support admins and squarely to the organization’s support admins.</p> <p>While the exact mechanics of storing and retrieving this organization support URL will be defined separately, for the sake of example, let’s say a package <code class="docutils literal notranslate"><span class="pre">foo</span></code> externally hosts wheel files on <code class="docutils literal notranslate"><span class="pre">`https://foo.example.com</span></code> <<a class="reference external" href="https://foo.example.com">https://foo.example.com</a>>`__ and that host becomes unreachable. When an installer tool tries to download and install the package <code class="docutils literal notranslate"><span class="pre">foo</span></code> wheel, the download step will fail. The installer would then be able to query PyPI to provide a useful error message to the end user:</p> <ul> <li>The installer downloads the <code class="docutils literal notranslate"><span class="pre">.rim</span></code> file and reads the <code class="docutils literal notranslate"><span class="pre">owner</span></code> key from the <code class="docutils literal notranslate"><span class="pre">EXTERNAL-HOSTING.json</span></code> file inside the <code class="docutils literal notranslate"><span class="pre">.rim</span></code> zip file.</li> <li>The installer queries PyPI for the support URI for the organization owner of the externally hosted wheel.</li> <li>An informative error message would then be displayed, e.g.:<blockquote> <div>The externally hosted wheel file <code class="docutils literal notranslate"><span class="pre">foo-....whl</span></code> could not be downloaded. Please contact <a class="reference external" href="mailto:support%40foo.example.com">support<span>@</span>foo<span>.</span>example<span>.</span>com</a> for help. Do not report this to the PyPI administrators.</div></blockquote> </li> </ul> </section> <section id="dismounting-wheels"> <span id="dismounting"></span><h2><a class="toc-backref" href="#dismounting-wheels" role="doc-backlink">Dismounting wheels</a></h2> <p>It is generally very easy to produce a <code class="docutils literal notranslate"><span class="pre">.rim</span></code> file from an existing <code class="docutils literal notranslate"><span class="pre">.whl</span></code> file. This could be done efficiently by a <a class="pep reference internal" href="../pep-0518/" title="PEP 518 – Specifying Minimum Build System Requirements for Python Projects">PEP 518</a> build backend with an additional command line option, or a separate tool which takes a <code class="docutils literal notranslate"><span class="pre">.whl</span></code> file as input and creates the associated <code class="docutils literal notranslate"><span class="pre">.rim</span></code> file. To complete the analogy, the act of turning a <code class="docutils literal notranslate"><span class="pre">.whl</span></code> into a <code class="docutils literal notranslate"><span class="pre">.rim</span></code> is called “dismounting”. The steps such a tool would take are:</p> <ul class="simple"> <li>Accept as input the source <code class="docutils literal notranslate"><span class="pre">.whl</span></code> file, the organization owner of the package, and URL at which the <code class="docutils literal notranslate"><span class="pre">.whl</span></code> will be hosted, and the support URI to report download problems from. These could in fact be captured in the <code class="docutils literal notranslate"><span class="pre">pyproject.toml</span></code> file, but that specification is out of scope for this PEP.</li> <li>Unzip the <code class="docutils literal notranslate"><span class="pre">.whl</span></code> and create the <code class="docutils literal notranslate"><span class="pre">.rim</span></code> zip archive.</li> <li>Omit from the <code class="docutils literal notranslate"><span class="pre">.rim</span></code> file any path in the <code class="docutils literal notranslate"><span class="pre">.whl</span></code> that <strong>isn’t</strong> rooted at the <code class="docutils literal notranslate"><span class="pre">.dist-info</span></code> directory.</li> <li>Calculate the hash of the source <code class="docutils literal notranslate"><span class="pre">.whl</span></code> file.</li> <li>Add the <code class="docutils literal notranslate"><span class="pre">EXTERNAL-HOSTING.json</span></code> file containing the JSON keys and values as described above, to the <code class="docutils literal notranslate"><span class="pre">.rim</span></code> archive.</li> </ul> </section> <section id="changes-to-tools"> <h2><a class="toc-backref" href="#changes-to-tools" role="doc-backlink">Changes to tools</a></h2> <p>Theoretically, installer tools shouldn’t need any changes, since when they have identified the wheel to download and install, they simply consult the download URLs returned by PyPI’s Simple API. In practice though, tools such as <code class="docutils literal notranslate"><span class="pre">pip</span></code> and <code class="docutils literal notranslate"><span class="pre">uv</span></code> may have constrained lists of hosts they will allow downloads from, such as PyPI’s own <code class="docutils literal notranslate"><span class="pre">pythonhosted.org</span></code> domain.</p> <p>In this case, such tools will need to relax those constraints, but the exact policy for this is left to the installer tools themselves. Any number of approaches could be implemented, such as downloading the <code class="docutils literal notranslate"><span class="pre">.rim</span></code> file and verifying the <code class="docutils literal notranslate"><span class="pre">EXTERNAL-HOSTING.json</span></code> metadata, or simply trusting the external downloads for any wheel with a matching checksum. They could also query PyPI for the project’s organization owner and support URI before trusting the download. They could warn the user when externally hosted wheel files are encountered, and/or require the use of a command line option to enable additional download hosts. Any of these verification policies could be chosen in configuration files.</p> <p>Installer tools should also probably provide better error messages when externally hosted wheels cannot be downloaded, e.g. because a host is unreachable. As described above, such tools could query enough metadata from PyPI to provide clear and distinct error messages pointing users to the package’s external hosting support email or issue tracker.</p> </section> <section id="constraints-for-external-hosting-services"> <h2><a class="toc-backref" href="#constraints-for-external-hosting-services" role="doc-backlink">Constraints for external hosting services</a></h2> <p>The following constraints lead to reliable and compatible external wheel hosting services:</p> <ul class="simple"> <li>External wheels <strong>MUST</strong> be served over HTTPS, with a certificate signed by <a class="reference external" href="https://wiki.mozilla.org/CA">Mozilla’s root certificate store</a>. This ensures compatibility with <a class="reference external" href="https://pip.pypa.io/en/stable/topics/https-certificates/">pip</a> and <a class="reference external" href="https://docs.astral.sh/uv/configuration/authentication/#custom-ca-certificates">uv</a>. At the time of this writing, <code class="docutils literal notranslate"><span class="pre">pip</span></code> 24.2 on Python 3.10 or newer uses the system certificate store in addition to the Mozilla store provided by the third party <a class="reference external" href="https://pypi.org/project/certifi/">certifi</a> Python package. <code class="docutils literal notranslate"><span class="pre">uv</span></code> uses the Mozilla store provided by the <a class="reference external" href="https://github.com/rustls/webpki-roots">webpki-roots</a> crate, but not the system store unless the <code class="docutils literal notranslate"><span class="pre">--native-tls</span></code> flag is given <a class="footnote-reference brackets" href="#fn1" id="id1">[1]</a>. <em>The PyPI administrators may modify this requirement in the future, but compatibility with popular installers will not be compromised.</em></li> <li>External wheel hosts <strong>SHOULD</strong> use a content delivery network (<a class="reference external" href="https://en.wikipedia.org/wiki/Content_delivery_network">CDN</a>), just as PyPI does.</li> <li>External wheel hosts <strong>MUST</strong> commit to a stable URL for all wheels they host.</li> <li>Externally hosted wheels <strong>MUST NOT</strong> be removed from an external wheel host unless the corresponding <code class="docutils literal notranslate"><span class="pre">.rim</span></code> file is deleted from PyPI first, and <strong>MUST NOT</strong> remove external wheels for yanked releases.</li> <li>External wheel hosts <strong>MUST</strong> support <a class="reference external" href="https://http.dev/range-request">HTTP range requests</a>.</li> <li>External wheel hosts <strong>SHOULD</strong> support the <a class="reference external" href="https://http.dev/2">HTTP/2</a> protocol.</li> </ul> </section> <section id="security"> <h2><a class="toc-backref" href="#security" role="doc-backlink">Security</a></h2> <p>Several factors as described in this proposal should mitigate security concerns with externally hosted wheels, such as:</p> <ul class="simple"> <li>Wheel file checksums <strong>MUST</strong> be included in <code class="docutils literal notranslate"><span class="pre">.rim</span></code> files, and once uploaded cannot be changed. Since the checksum stored on PyPI is immutable and required, it is not possible to spoof an external wheel file, even if the owning organization lost control of their hosting domain.</li> <li>Externally hosted wheels <strong>MUST</strong> be served over HTTPS.</li> <li>In order to serve externally hosted wheels, organizations <strong>MUST</strong> be approved by the PyPI admins.</li> </ul> <p>When users identify malware or vulnerabilities in PyPI-hosted projects, they can now report this using the <a class="reference external" href="https://pypi.org/security/">malware reporting facilities</a> on PyPI, as also described in this <a class="reference external" href="https://blog.pypi.org/posts/2024-03-06-malware-reporting-evolved/">blog post</a>. The same process can be used to report security issues in externally hosted wheels, and the same remediation process should be used. In addition, since organizations with external hosting enabled MUST provide a support contact URI, that URI can be used in some cases to report the security issue to the hosting organization. Such organization reporting won’t make sense for malware, but could indeed be a very useful way to report security vulnerabilities in externally hosted wheels.</p> </section> <section id="rejected-ideas"> <h2><a class="toc-backref" href="#rejected-ideas" role="doc-backlink">Rejected ideas</a></h2> <p>Several ideas were considered and rejected.</p> <ul class="simple"> <li>Requiring digital signatures on externally hosted wheel files, either in addition to or other than hashes. We deem this unnecessary since the checksum requirement should be enough to validate that the metadata on PyPI for a wheel exactly matches the downloaded wheel. The added complexity of key management outweighs any additional benefit such digital signatures might convey.</li> <li>Hash verification on <code class="docutils literal notranslate"><span class="pre">.rim</span></code> file uploads. PyPI <em>could</em> verify that the hash in the uploaded <code class="docutils literal notranslate"><span class="pre">.rim</span></code> file matches the externally hosted wheel before it accepts the upload, but this requires downloading the external wheel and performing the checksum, which also implies that the upload of the <code class="docutils literal notranslate"><span class="pre">.rim</span></code> file cannot be accepted until this external <code class="docutils literal notranslate"><span class="pre">.whl</span></code> file is downloaded and verified. This increases PyPI bandwidth and slows down the upload query, although <a class="pep reference internal" href="../pep-0694/" title="PEP 694 – Upload 2.0 API for Python Package Indexes">PEP 694</a> draft uploads could potentially mitigate these concerns. Still, the benefit is not likely worth the additional complexity.</li> <li>Periodic verification of the download URLs by the index. PyPI could try to periodically ensure that the external wheel host or the external <code class="docutils literal notranslate"><span class="pre">.whl</span></code> file itself is still available, e.g. via an <span class="target" id="index-0"></span><a class="rfc reference external" href="https://datatracker.ietf.org/doc/html/rfc9110.html#section-9.3.2"><strong>HTTP HEAD</strong></a> request. This is likely overkill and without also providing the file’s checksum in the response <a class="footnote-reference brackets" href="#fn2" id="id2">[2]</a>, may not provide much additional benefit.</li> <li>This PEP could allow for an organization to provide fallback download hosts, such that a secondary is available if the primary goes down. We believe that DNS-based replication is a much better, well-known technique, and probably much more resilient anyway.</li> <li><code class="docutils literal notranslate"><span class="pre">.rim</span></code> file replacement. While it is allowed for <code class="docutils literal notranslate"><span class="pre">.whl</span></code> files to replace existing <code class="docutils literal notranslate"><span class="pre">.rim</span></code> files, as long as a) the <code class="docutils literal notranslate"><span class="pre">.rim</span></code> file hasn’t been deleted or yanked, b) the checksums match, we do not allow replacing <code class="docutils literal notranslate"><span class="pre">.whl</span></code> files with <code class="docutils literal notranslate"><span class="pre">.rim</span></code> files, nor do we allow a <code class="docutils literal notranslate"><span class="pre">.rim</span></code> file to overwrite an existing <code class="docutils literal notranslate"><span class="pre">.rim</span></code> file. This latter could be a technique to change the hosting URL for an externally hosted <code class="docutils literal notranslate"><span class="pre">.whl</span></code>; however, we do not think this is a good idea. There are other ways to “fix” an external host URL as described above, and we do not want to encourage mass re-uploads of existing <code class="docutils literal notranslate"><span class="pre">.rim</span></code> files.</li> </ul> </section> <section id="footnotes"> <h2><a class="toc-backref" href="#footnotes" role="doc-backlink">Footnotes</a></h2> <aside class="footnote-list brackets"> <aside class="footnote brackets" id="fn1" role="doc-footnote"> <dt class="label" id="fn1">[<a href="#id1">1</a>]</dt> <dd>The <code class="docutils literal notranslate"><span class="pre">uv</span> <span class="pre">--native-tls</span></code> flag <a class="reference external" href="https://github.com/astral-sh/uv/blob/3ce34035c84804fdfb8b78cf11b9ba1b168d0f35/crates/uv-client/src/base_client.rs#L248">replaces</a> the <code class="docutils literal notranslate"><span class="pre">webpki-roots</span></code> store.</aside> <aside class="footnote brackets" id="fn2" role="doc-footnote"> <dt class="label" id="fn2">[<a href="#id2">2</a>]</dt> <dd>There being no standard way to return the file’s checksum in response to an <span class="target" id="index-1"></span><a class="rfc reference external" href="https://datatracker.ietf.org/doc/html/rfc9110.html#section-9.3.2"><strong>HTTP HEAD</strong></a> request.</aside> </aside> </section> <section id="copyright"> <h2><a class="toc-backref" href="#copyright" role="doc-backlink">Copyright</a></h2> <p>This document is placed in the public domain or under the CC0-1.0-Universal license, whichever is more permissive.</p> </section> </section> <hr class="docutils" /> <p>Source: <a class="reference external" href="https://github.com/python/peps/blob/main/peps/pep-0759.rst">https://github.com/python/peps/blob/main/peps/pep-0759.rst</a></p> <p>Last modified: <a class="reference external" href="https://github.com/python/peps/commits/main/peps/pep-0759.rst">2025-01-31 18:51:56 GMT</a></p> </article> <nav id="pep-sidebar"> <h2>Contents</h2> <ul> <li><a class="reference internal" href="#abstract">Abstract</a></li> <li><a class="reference internal" href="#pep-withdrawn">PEP withdrawn</a></li> <li><a class="reference internal" href="#rationale">Rationale</a><ul> <li><a class="reference internal" href="#historical-context">Historical context</a></li> <li><a class="reference internal" href="#the-problem-with-multiple-indexes">The problem with multiple indexes</a></li> <li><a class="reference internal" href="#addressing-pypi-limits">Addressing PyPI limits</a></li> <li><a class="reference internal" href="#reducing-operational-complexity">Reducing operational complexity</a></li> </ul> </li> <li><a class="reference internal" href="#specification">Specification</a><ul> <li><a class="reference internal" href="#effects-of-the-rim-file">Effects of the RIM file</a></li> <li><a class="reference internal" href="#availability-order">Availability order</a></li> <li><a class="reference internal" href="#wheels-can-override-rims">Wheels can override RIMs</a></li> <li><a class="reference internal" href="#pypi-api-bump-unnecessary">PyPI API bump unnecessary</a></li> </ul> </li> <li><a class="reference internal" href="#external-hosting-resiliency">External hosting resiliency</a></li> <li><a class="reference internal" href="#dismounting-wheels">Dismounting wheels</a></li> <li><a class="reference internal" href="#changes-to-tools">Changes to tools</a></li> <li><a class="reference internal" href="#constraints-for-external-hosting-services">Constraints for external hosting services</a></li> <li><a class="reference internal" href="#security">Security</a></li> <li><a class="reference internal" href="#rejected-ideas">Rejected ideas</a></li> <li><a class="reference internal" href="#footnotes">Footnotes</a></li> <li><a class="reference internal" href="#copyright">Copyright</a></li> </ul> <br> <a id="source" href="https://github.com/python/peps/blob/main/peps/pep-0759.rst">Page Source (GitHub)</a> </nav> </section> <script src="../_static/colour_scheme.js"></script> <script src="../_static/wrap_tables.js"></script> <script src="../_static/sticky_banner.js"></script> </body> </html>