CINXE.COM

<!DOCTYPE html> <html lang="en"> <head> <meta charset="utf-8"> <meta name="viewport" content="width=device-width, initial-scale=1.0"> <meta name="color-scheme" content="light dark"> <title>PEP 658 – Serve Distribution Metadata in the Simple Repository API | peps.python.org</title> <link rel="shortcut icon" href="../_static/py.png"> <link rel="canonical" href="https://peps.python.org/pep-0658/"> <link rel="stylesheet" href="../_static/style.css" type="text/css"> <link rel="stylesheet" href="../_static/mq.css" type="text/css"> <link rel="stylesheet" href="../_static/pygments.css" type="text/css" media="(prefers-color-scheme: light)" id="pyg-light"> <link rel="stylesheet" href="../_static/pygments_dark.css" type="text/css" media="(prefers-color-scheme: dark)" id="pyg-dark"> <link rel="alternate" type="application/rss+xml" title="Latest PEPs" href="https://peps.python.org/peps.rss"> <meta property="og:title" content='PEP 658 – Serve Distribution Metadata in the Simple Repository API | peps.python.org'> <meta property="og:description" content="This PEP proposes adding an anchor tag to expose the METADATA file from distributions in the PEP 503 “simple” repository API. A data-dist-info-metadata attribute is introduced to indicate that the file from a given distribution can be independently fetc..."> <meta property="og:type" content="website"> <meta property="og:url" content="https://peps.python.org/pep-0658/"> <meta property="og:site_name" content="Python Enhancement Proposals (PEPs)"> <meta property="og:image" content="https://peps.python.org/_static/og-image.png"> <meta property="og:image:alt" content="Python PEPs"> <meta property="og:image:width" content="200"> <meta property="og:image:height" content="200"> <meta name="description" content="This PEP proposes adding an anchor tag to expose the METADATA file from distributions in the PEP 503 “simple” repository API. A data-dist-info-metadata attribute is introduced to indicate that the file from a given distribution can be independently fetc..."> <meta name="theme-color" content="#3776ab"> </head> <body> <svg xmlns="http://www.w3.org/2000/svg" style="display: none;"> <symbol id="svg-sun-half" viewBox="0 0 24 24" pointer-events="all"> <title>Following system colour scheme</title> <svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"> <circle cx="12" cy="12" r="9"></circle> <path d="M12 3v18m0-12l4.65-4.65M12 14.3l7.37-7.37M12 19.6l8.85-8.85"></path> </svg> </symbol> <symbol id="svg-moon" viewBox="0 0 24 24" pointer-events="all"> <title>Selected dark colour scheme</title> <svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"> <path stroke="none" d="M0 0h24v24H0z" fill="none"></path> <path d="M12 3c.132 0 .263 0 .393 0a7.5 7.5 0 0 0 7.92 12.446a9 9 0 1 1 -8.313 -12.454z"></path> </svg> </symbol> <symbol id="svg-sun" viewBox="0 0 24 24" pointer-events="all"> <title>Selected light colour scheme</title> <svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"> <circle cx="12" cy="12" r="5"></circle> <line x1="12" y1="1" x2="12" y2="3"></line> <line x1="12" y1="21" x2="12" y2="23"></line> <line x1="4.22" y1="4.22" x2="5.64" y2="5.64"></line> <line x1="18.36" y1="18.36" x2="19.78" y2="19.78"></line> <line x1="1" y1="12" x2="3" y2="12"></line> <line x1="21" y1="12" x2="23" y2="12"></line> <line x1="4.22" y1="19.78" x2="5.64" y2="18.36"></line> <line x1="18.36" y1="5.64" x2="19.78" y2="4.22"></line> </svg> </symbol> </svg> <script> document.documentElement.dataset.colour_scheme = localStorage.getItem("colour_scheme") || "auto" </script> <section id="pep-page-section"> <header> <h1>Python Enhancement Proposals</h1> <ul class="breadcrumbs"> <li><a href="https://www.python.org/" title="The Python Programming Language">Python</a> » </li> <li><a href="../pep-0000/">PEP Index</a> » </li> <li>PEP 658</li> </ul> <button id="colour-scheme-cycler" onClick="setColourScheme(nextColourScheme())"> <svg aria-hidden="true" class="colour-scheme-icon-when-auto"><use href="#svg-sun-half"></use></svg> <svg aria-hidden="true" class="colour-scheme-icon-when-dark"><use href="#svg-moon"></use></svg> <svg aria-hidden="true" class="colour-scheme-icon-when-light"><use href="#svg-sun"></use></svg> <span class="visually-hidden">Toggle light / dark / auto colour theme</span> </button> </header> <article> <section id="pep-content"> <h1 class="page-title">PEP 658 – Serve Distribution Metadata in the Simple Repository API</h1> <dl class="rfc2822 field-list simple"> <dt class="field-odd">Author<span class="colon">:</span></dt> <dd class="field-odd">Tzu-ping Chung <uranusjr at gmail.com></dd> <dt class="field-even">Sponsor<span class="colon">:</span></dt> <dd class="field-even">Brett Cannon <brett at python.org></dd> <dt class="field-odd">PEP-Delegate<span class="colon">:</span></dt> <dd class="field-odd">Donald Stufft <donald at stufft.io></dd> <dt class="field-even">Discussions-To<span class="colon">:</span></dt> <dd class="field-even"><a class="reference external" href="https://discuss.python.org/t/8651">Discourse thread</a></dd> <dt class="field-odd">Status<span class="colon">:</span></dt> <dd class="field-odd"><abbr title="Normative proposal accepted for implementation">Accepted</abbr></dd> <dt class="field-even">Type<span class="colon">:</span></dt> <dd class="field-even"><abbr title="Normative PEP with a new feature for Python, implementation change for CPython or interoperability standard for the ecosystem">Standards Track</abbr></dd> <dt class="field-odd">Topic<span class="colon">:</span></dt> <dd class="field-odd"><a class="reference external" href="../topic/packaging/">Packaging</a></dd> <dt class="field-even">Created<span class="colon">:</span></dt> <dd class="field-even">10-May-2021</dd> <dt class="field-odd">Post-History<span class="colon">:</span></dt> <dd class="field-odd">10-May-2021</dd> <dt class="field-even">Resolution<span class="colon">:</span></dt> <dd class="field-even"><a class="reference external" href="https://discuss.python.org/t/8651/48">Discourse message</a></dd> </dl> <hr class="docutils" /> <section id="contents"> <details><summary>Table of Contents</summary><ul class="simple"> <li><a class="reference internal" href="#abstract">Abstract</a></li> <li><a class="reference internal" href="#motivation">Motivation</a></li> <li><a class="reference internal" href="#rationale">Rationale</a></li> <li><a class="reference internal" href="#specification">Specification</a></li> <li><a class="reference internal" href="#backwards-compatibility">Backwards Compatibility</a></li> <li><a class="reference internal" href="#rejected-ideas">Rejected Ideas</a><ul> <li><a class="reference internal" href="#put-metadata-content-on-the-project-page">Put metadata content on the project page</a></li> <li><a class="reference internal" href="#expose-more-files-in-the-distribution">Expose more files in the distribution</a></li> <li><a class="reference internal" href="#explicitly-specify-the-metadata-file-s-url-on-the-project-page">Explicitly specify the metadata file’s URL on the project page</a></li> </ul> </li> <li><a class="reference internal" href="#references">References</a></li> <li><a class="reference internal" href="#copyright">Copyright</a></li> </ul> </details></section> <section id="abstract"> <h2><a class="toc-backref" href="#abstract" role="doc-backlink">Abstract</a></h2> <p>This PEP proposes adding an anchor tag to expose the <code class="docutils literal notranslate"><span class="pre">METADATA</span></code> file from distributions in the <a class="pep reference internal" href="../pep-0503/" title="PEP 503 – Simple Repository API">PEP 503</a> “simple” repository API. A <code class="docutils literal notranslate"><span class="pre">data-dist-info-metadata</span></code> attribute is introduced to indicate that the file from a given distribution can be independently fetched.</p> </section> <section id="motivation"> <h2><a class="toc-backref" href="#motivation" role="doc-backlink">Motivation</a></h2> <p>Package management workflows made popular by recent tooling increase the need to inspect distribution metadata without intending to install the distribution, and download multiple distributions of a project to choose from based on their metadata. This means they end up discarding much downloaded data, which is inefficient and results in a bad user experience.</p> </section> <section id="rationale"> <h2><a class="toc-backref" href="#rationale" role="doc-backlink">Rationale</a></h2> <p>Tools have been exploring methods to reduce the download size by partially downloading wheels with HTTP range requests. This, however, adds additional run-time requirements to the repository server. It also still adds additional overhead, since a separate request is needed to fetch the wheel’s file listing to find the correct offset to fetch the metadata file. It is therefore desired to make the server extract the metadata file in advance, and serve it as an independent file to avoid the need to perform additional requests and ZIP inspection.</p> <p>The metadata file defined by the Core Metadata Specification <a class="reference internal" href="#core-metadata" id="id1"><span>[core-metadata]</span></a> will be served directly by repositories since it contains the necessary information for common use cases. The metadata must only be served for standards-compliant distributions such as wheels <a class="reference internal" href="#wheel" id="id2"><span>[wheel]</span></a> and sdists <a class="reference internal" href="#sdist" id="id3"><span>[sdist]</span></a>, and must be identical to the distribution’s canonical metadata file, such as a wheel’s <code class="docutils literal notranslate"><span class="pre">METADATA</span></code> file in the <code class="docutils literal notranslate"><span class="pre">.dist-info</span></code> directory <a class="reference internal" href="#dist-info" id="id4"><span>[dist-info]</span></a>.</p> <p>An HTML attribute on the distribution file’s anchor link is needed to indicate whether a client is able to choose the separately served metadata file. The attribute is also used to provide the metadata content’s hash for client-side verification. The attribute’s absence indicates that a separate metadata entry is not available for the distribution, either because of the distribution’s content, or lack of repository support.</p> </section> <section id="specification"> <h2><a class="toc-backref" href="#specification" role="doc-backlink">Specification</a></h2> <p>In a simple repository’s project page, each anchor tag pointing to a distribution <strong>MAY</strong> have a <code class="docutils literal notranslate"><span class="pre">data-dist-info-metadata</span></code> attribute. The presence of the attribute indicates the distribution represented by the anchor tag <strong>MUST</strong> contain a Core Metadata file that will not be modified when the distribution is processed and/or installed.</p> <p>If a <code class="docutils literal notranslate"><span class="pre">data-dist-info-metadata</span></code> attribute is present, the repository <strong>MUST</strong> serve the distribution’s Core Metadata file alongside the distribution with a <code class="docutils literal notranslate"><span class="pre">.metadata</span></code> appended to the distribution’s file name. For example, the Core Metadata of a distribution served at <code class="docutils literal notranslate"><span class="pre">/files/distribution-1.0-py3.none.any.whl</span></code> would be located at <code class="docutils literal notranslate"><span class="pre">/files/distribution-1.0-py3.none.any.whl.metadata</span></code>. This is similar to how <a class="pep reference internal" href="../pep-0503/" title="PEP 503 – Simple Repository API">PEP 503</a> specifies the GPG signature file’s location.</p> <p>The repository <strong>SHOULD</strong> provide the hash of the Core Metadata file as the <code class="docutils literal notranslate"><span class="pre">data-dist-info-metadata</span></code> attribute’s value using the syntax <code class="docutils literal notranslate"><span class="pre"><hashname>=<hashvalue></span></code>, where <code class="docutils literal notranslate"><span class="pre"><hashname></span></code> is the lower cased name of the hash function used, and <code class="docutils literal notranslate"><span class="pre"><hashvalue></span></code> is the hex encoded digest. The repository <strong>MAY</strong> use <code class="docutils literal notranslate"><span class="pre">true</span></code> as the attribute’s value if a hash is unavailable.</p> </section> <section id="backwards-compatibility"> <h2><a class="toc-backref" href="#backwards-compatibility" role="doc-backlink">Backwards Compatibility</a></h2> <p>If an anchor tag lacks the <code class="docutils literal notranslate"><span class="pre">data-dist-info-metadata</span></code> attribute, tools are expected to revert to their current behaviour of downloading the distribution to inspect the metadata.</p> <p>Older tools not supporting the new <code class="docutils literal notranslate"><span class="pre">data-dist-info-metadata</span></code> attribute are expected to ignore the attribute and maintain their current behaviour of downloading the distribution to inspect the metadata. This is similar to how prior <code class="docutils literal notranslate"><span class="pre">data-</span></code> attribute additions expect existing tools to operate.</p> </section> <section id="rejected-ideas"> <h2><a class="toc-backref" href="#rejected-ideas" role="doc-backlink">Rejected Ideas</a></h2> <section id="put-metadata-content-on-the-project-page"> <h3><a class="toc-backref" href="#put-metadata-content-on-the-project-page" role="doc-backlink">Put metadata content on the project page</a></h3> <p>Since tools generally only need dependency information from a distribution in addition to what’s already available on the project page, it was proposed that repositories may directly include the information on the project page, like the <code class="docutils literal notranslate"><span class="pre">data-requires-python</span></code> attribute specified in <a class="pep reference internal" href="../pep-0503/" title="PEP 503 – Simple Repository API">PEP 503</a>.</p> <p>This approach was abandoned since a distribution may contain arbitrarily long lists of dependencies (including required and optional), and it is unclear whether including the information for every distribution in a project would result in net savings since the information for most distributions generally ends up unneeded. By serving the metadata separately, performance can be better estimated since data usage will be more proportional to the number of distributions inspected.</p> </section> <section id="expose-more-files-in-the-distribution"> <h3><a class="toc-backref" href="#expose-more-files-in-the-distribution" role="doc-backlink">Expose more files in the distribution</a></h3> <p>It was proposed to provide the entire <code class="docutils literal notranslate"><span class="pre">.dist-info</span></code> directory as a separate part, instead of only the metadata file. However, serving multiple files in one entity through HTTP requires re-archiving them separately after they are extracted from the original distribution by the repository server, and there are no current use cases for files other than <code class="docutils literal notranslate"><span class="pre">METADATA</span></code> when the distribution itself is not going to be installed.</p> <p>It should also be noted that the approach taken here does not preclude other files from being introduced in the future, whether we want to serve them together or individually.</p> </section> <section id="explicitly-specify-the-metadata-file-s-url-on-the-project-page"> <h3><a class="toc-backref" href="#explicitly-specify-the-metadata-file-s-url-on-the-project-page" role="doc-backlink">Explicitly specify the metadata file’s URL on the project page</a></h3> <p>An early version of this draft proposed putting the metadata file’s URL in the <code class="docutils literal notranslate"><span class="pre">data-dist-info-metadata</span></code> attribute. But people feel it is better for discoverability to require the repository to serve the metadata file at a determined location instead. The current approach also has an additional benefit of making the project page smaller.</p> </section> </section> <section id="references"> <h2><a class="toc-backref" href="#references" role="doc-backlink">References</a></h2> <div role="list" class="citation-list"> <div class="citation" id="core-metadata" role="doc-biblioentry"> <dt class="label" id="core-metadata">[<a href="#id1">core-metadata</a>]</dt> <dd><a class="reference external" href="https://packaging.python.org/specifications/core-metadata/">https://packaging.python.org/specifications/core-metadata/</a></div> <div class="citation" id="dist-info" role="doc-biblioentry"> <dt class="label" id="dist-info">[<a href="#id4">dist-info</a>]</dt> <dd><a class="reference external" href="https://packaging.python.org/specifications/recording-installed-packages/">https://packaging.python.org/specifications/recording-installed-packages/</a></div> <div class="citation" id="wheel" role="doc-biblioentry"> <dt class="label" id="wheel">[<a href="#id2">wheel</a>]</dt> <dd><a class="reference external" href="https://packaging.python.org/specifications/binary-distribution-format/">https://packaging.python.org/specifications/binary-distribution-format/</a></div> <div class="citation" id="sdist" role="doc-biblioentry"> <dt class="label" id="sdist">[<a href="#id3">sdist</a>]</dt> <dd><a class="reference external" href="https://packaging.python.org/specifications/source-distribution-format/">https://packaging.python.org/specifications/source-distribution-format/</a></div> </div> </section> <section id="copyright"> <h2><a class="toc-backref" href="#copyright" role="doc-backlink">Copyright</a></h2> <p>This document is placed in the public domain or under the CC0-1.0-Universal license, whichever is more permissive.</p> </section> </section> <hr class="docutils" /> <p>Source: <a class="reference external" href="https://github.com/python/peps/blob/main/peps/pep-0658.rst">https://github.com/python/peps/blob/main/peps/pep-0658.rst</a></p> <p>Last modified: <a class="reference external" href="https://github.com/python/peps/commits/main/peps/pep-0658.rst">2023-10-13 05:15:59 GMT</a></p> </article> <nav id="pep-sidebar"> <h2>Contents</h2> <ul> <li><a class="reference internal" href="#abstract">Abstract</a></li> <li><a class="reference internal" href="#motivation">Motivation</a></li> <li><a class="reference internal" href="#rationale">Rationale</a></li> <li><a class="reference internal" href="#specification">Specification</a></li> <li><a class="reference internal" href="#backwards-compatibility">Backwards Compatibility</a></li> <li><a class="reference internal" href="#rejected-ideas">Rejected Ideas</a><ul> <li><a class="reference internal" href="#put-metadata-content-on-the-project-page">Put metadata content on the project page</a></li> <li><a class="reference internal" href="#expose-more-files-in-the-distribution">Expose more files in the distribution</a></li> <li><a class="reference internal" href="#explicitly-specify-the-metadata-file-s-url-on-the-project-page">Explicitly specify the metadata file’s URL on the project page</a></li> </ul> </li> <li><a class="reference internal" href="#references">References</a></li> <li><a class="reference internal" href="#copyright">Copyright</a></li> </ul> <br> <a id="source" href="https://github.com/python/peps/blob/main/peps/pep-0658.rst">Page Source (GitHub)</a> </nav> </section> <script src="../_static/colour_scheme.js"></script> <script src="../_static/wrap_tables.js"></script> <script src="../_static/sticky_banner.js"></script> </body> </html>