CINXE.COM

<!DOCTYPE html> <html lang="en"> <head> <meta charset="utf-8"> <meta name="viewport" content="width=device-width, initial-scale=1.0"> <meta name="color-scheme" content="light dark"> <title>PEP 723 – Inline script metadata | peps.python.org</title> <link rel="shortcut icon" href="../_static/py.png"> <link rel="canonical" href="https://peps.python.org/pep-0723/"> <link rel="stylesheet" href="../_static/style.css" type="text/css"> <link rel="stylesheet" href="../_static/mq.css" type="text/css"> <link rel="stylesheet" href="../_static/pygments.css" type="text/css" media="(prefers-color-scheme: light)" id="pyg-light"> <link rel="stylesheet" href="../_static/pygments_dark.css" type="text/css" media="(prefers-color-scheme: dark)" id="pyg-dark"> <link rel="alternate" type="application/rss+xml" title="Latest PEPs" href="https://peps.python.org/peps.rss"> <meta property="og:title" content='PEP 723 – Inline script metadata | peps.python.org'> <meta property="og:description" content="This PEP specifies a metadata format that can be embedded in single-file Python scripts to assist launchers, IDEs and other external tools which may need to interact with such scripts."> <meta property="og:type" content="website"> <meta property="og:url" content="https://peps.python.org/pep-0723/"> <meta property="og:site_name" content="Python Enhancement Proposals (PEPs)"> <meta property="og:image" content="https://peps.python.org/_static/og-image.png"> <meta property="og:image:alt" content="Python PEPs"> <meta property="og:image:width" content="200"> <meta property="og:image:height" content="200"> <meta name="description" content="This PEP specifies a metadata format that can be embedded in single-file Python scripts to assist launchers, IDEs and other external tools which may need to interact with such scripts."> <meta name="theme-color" content="#3776ab"> </head> <body> <svg xmlns="http://www.w3.org/2000/svg" style="display: none;"> <symbol id="svg-sun-half" viewBox="0 0 24 24" pointer-events="all"> <title>Following system colour scheme</title> <svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"> <circle cx="12" cy="12" r="9"></circle> <path d="M12 3v18m0-12l4.65-4.65M12 14.3l7.37-7.37M12 19.6l8.85-8.85"></path> </svg> </symbol> <symbol id="svg-moon" viewBox="0 0 24 24" pointer-events="all"> <title>Selected dark colour scheme</title> <svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"> <path stroke="none" d="M0 0h24v24H0z" fill="none"></path> <path d="M12 3c.132 0 .263 0 .393 0a7.5 7.5 0 0 0 7.92 12.446a9 9 0 1 1 -8.313 -12.454z"></path> </svg> </symbol> <symbol id="svg-sun" viewBox="0 0 24 24" pointer-events="all"> <title>Selected light colour scheme</title> <svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"> <circle cx="12" cy="12" r="5"></circle> <line x1="12" y1="1" x2="12" y2="3"></line> <line x1="12" y1="21" x2="12" y2="23"></line> <line x1="4.22" y1="4.22" x2="5.64" y2="5.64"></line> <line x1="18.36" y1="18.36" x2="19.78" y2="19.78"></line> <line x1="1" y1="12" x2="3" y2="12"></line> <line x1="21" y1="12" x2="23" y2="12"></line> <line x1="4.22" y1="19.78" x2="5.64" y2="18.36"></line> <line x1="18.36" y1="5.64" x2="19.78" y2="4.22"></line> </svg> </symbol> </svg> <script> document.documentElement.dataset.colour_scheme = localStorage.getItem("colour_scheme") || "auto" </script> <section id="pep-page-section"> <header> <h1>Python Enhancement Proposals</h1> <ul class="breadcrumbs"> <li><a href="https://www.python.org/" title="The Python Programming Language">Python</a> » </li> <li><a href="../pep-0000/">PEP Index</a> » </li> <li>PEP 723</li> </ul> <button id="colour-scheme-cycler" onClick="setColourScheme(nextColourScheme())"> <svg aria-hidden="true" class="colour-scheme-icon-when-auto"><use href="#svg-sun-half"></use></svg> <svg aria-hidden="true" class="colour-scheme-icon-when-dark"><use href="#svg-moon"></use></svg> <svg aria-hidden="true" class="colour-scheme-icon-when-light"><use href="#svg-sun"></use></svg> Toggle light / dark / auto colour theme </button> </header> <article> <section id="pep-content"> <h1 class="page-title">PEP 723 – Inline script metadata</h1> <dl class="rfc2822 field-list simple"> <dt class="field-odd">Author:</dt> <dd class="field-odd">Ofek Lev <ofekmeister at gmail.com></dd> <dt class="field-even">Sponsor:</dt> <dd class="field-even">Adam Turner <python at quite.org.uk></dd> <dt class="field-odd">PEP-Delegate:</dt> <dd class="field-odd">Brett Cannon <brett at python.org></dd> <dt class="field-even">Discussions-To:</dt> <dd class="field-even"><a class="reference external" href="https://discuss.python.org/t/31151">Discourse thread</a></dd> <dt class="field-odd">Status:</dt> <dd class="field-odd"><abbr title="Accepted and implementation complete, or no longer active">Final</abbr></dd> <dt class="field-even">Type:</dt> <dd class="field-even"><abbr title="Normative PEP with a new feature for Python, implementation change for CPython or interoperability standard for the ecosystem">Standards Track</abbr></dd> <dt class="field-odd">Topic:</dt> <dd class="field-odd"><a class="reference external" href="../topic/packaging/">Packaging</a></dd> <dt class="field-even">Created:</dt> <dd class="field-even">04-Aug-2023</dd> <dt class="field-odd">Post-History:</dt> <dd class="field-odd"><a class="reference external" href="https://discuss.python.org/t/30979" title="Discourse thread">04-Aug-2023</a>, <a class="reference external" href="https://discuss.python.org/t/31151" title="Discourse thread">06-Aug-2023</a>, <a class="reference external" href="https://discuss.python.org/t/32149" title="Discourse thread">23-Aug-2023</a>, <a class="reference external" href="https://discuss.python.org/t/40418" title="Discourse thread">06-Dec-2023</a></dd> <dt class="field-even">Replaces:</dt> <dd class="field-even"><a class="reference external" href="../pep-0722/">722</a></dd> <dt class="field-odd">Resolution:</dt> <dd class="field-odd"><a class="reference external" href="https://discuss.python.org/t/40418/82">08-Jan-2024</a></dd> </dl> <hr class="docutils" /> <section id="contents"> <details><summary>Table of Contents</summary><ul class="simple"> <li><a class="reference internal" href="#abstract">Abstract</a></li> <li><a class="reference internal" href="#motivation">Motivation</a></li> <li><a class="reference internal" href="#rationale">Rationale</a></li> <li><a class="reference internal" href="#specification">Specification</a><ul> <li><a class="reference internal" href="#script-type">script type</a></li> <li><a class="reference internal" href="#example">Example</a></li> </ul> </li> <li><a class="reference internal" href="#reference-implementation">Reference Implementation</a></li> <li><a class="reference internal" href="#backwards-compatibility">Backwards Compatibility</a></li> <li><a class="reference internal" href="#security-implications">Security Implications</a></li> <li><a class="reference internal" href="#how-to-teach-this">How to Teach This</a></li> <li><a class="reference internal" href="#recommendations">Recommendations</a></li> <li><a class="reference internal" href="#tooling-buy-in">Tooling buy-in</a></li> <li><a class="reference internal" href="#rejected-ideas">Rejected Ideas</a><ul> <li><a class="reference internal" href="#why-not-use-a-comment-block-resembling-requirements-txt">Why not use a comment block resembling requirements.txt?</a></li> <li><a class="reference internal" href="#why-not-use-a-multi-line-string">Why not use a multi-line string?</a></li> <li><a class="reference internal" href="#why-not-reuse-core-metadata-fields">Why not reuse core metadata fields?</a></li> <li><a class="reference internal" href="#why-not-limit-to-specific-metadata-fields">Why not limit to specific metadata fields?</a></li> <li><a class="reference internal" href="#why-not-limit-tool-configuration">Why not limit tool configuration?</a></li> <li><a class="reference internal" href="#why-not-limit-tool-behavior">Why not limit tool behavior?</a></li> <li><a class="reference internal" href="#why-not-just-set-up-a-python-project-with-a-pyproject-toml">Why not just set up a Python project with a <code class="docutils literal notranslate">pyproject.toml</code>?</a></li> <li><a class="reference internal" href="#why-not-infer-the-requirements-from-import-statements">Why not infer the requirements from import statements?</a></li> <li><a class="reference internal" href="#why-not-use-a-requirements-file-for-dependencies">Why not use a requirements file for dependencies?</a></li> <li><a class="reference internal" href="#why-not-use-possibly-restricted-python-syntax">Why not use (possibly restricted) Python syntax?</a></li> <li><a class="reference internal" href="#what-about-local-dependencies">What about local dependencies?</a></li> </ul> </li> <li><a class="reference internal" href="#open-issues">Open Issues</a></li> <li><a class="reference internal" href="#footnotes">Footnotes</a></li> <li><a class="reference internal" href="#copyright">Copyright</a></li> </ul> </details></section> <div class="pep-banner canonical-pypa-spec sticky-banner admonition important"> Important This PEP is a historical document. The up-to-date, canonical spec, <a class="reference external" href="https://packaging.python.org/en/latest/specifications/inline-script-metadata/#inline-script-metadata" title="(in Python Packaging User Guide)">Inline script metadata</a>, is maintained on the <a class="reference external" href="https://packaging.python.org/en/latest/specifications/">PyPA specs page</a>. × See the <a class="reference external" href="https://www.pypa.io/en/latest/specifications/#handling-fixes-and-other-minor-updates">PyPA specification update process</a> for how to propose changes. </div> <section id="abstract"> <h2><a class="toc-backref" href="#abstract" role="doc-backlink">Abstract</a></h2> This PEP specifies a metadata format that can be embedded in single-file Python scripts to assist launchers, IDEs and other external tools which may need to interact with such scripts. </section> <section id="motivation"> <h2><a class="toc-backref" href="#motivation" role="doc-backlink">Motivation</a></h2> Python is routinely used as a scripting language, with Python scripts as a (better) alternative to shell scripts, batch files, etc. When Python code is structured as a script, it is usually stored as a single file and does not expect the availability of any other local code that may be used for imports. As such, it is possible to share with others over arbitrary text-based means such as email, a URL to the script, or even a chat window. Code that is structured like this may live as a single file forever, never becoming a full-fledged project with its own directory and <code class="docutils literal notranslate">pyproject.toml</code> file. An issue that users encounter with this approach is that there is no standard mechanism to define metadata for tools whose job it is to execute such scripts. For example, a tool that runs a script may need to know which dependencies are required or the supported version(s) of Python. There is currently no standard tool that addresses this issue, and this PEP does not attempt to define one. However, any tool that does address this issue will need to know what the runtime requirements of scripts are. By defining a standard format for storing such metadata, existing tools, as well as any future tools, will be able to obtain that information without requiring users to include tool-specific metadata in their scripts. </section> <section id="rationale"> <h2><a class="toc-backref" href="#rationale" role="doc-backlink">Rationale</a></h2> This PEP defines a mechanism for embedding metadata within the script itself, and not in an external file. The metadata format is designed to be similar to the layout of data in the <code class="docutils literal notranslate">pyproject.toml</code> file of a Python project directory, to provide a familiar experience for users who have experience writing Python projects. By using a similar format, we avoid unnecessary inconsistency between packaging tools, a common frustration expressed by users in the recent <a class="reference external" href="https://discuss.python.org/t/22420">packaging survey</a>. The following are some of the use cases that this PEP wishes to support: <ul class="simple"> <li>A user facing CLI that is capable of executing scripts. If we take Hatch as an example, the interface would be simply <code class="docutils literal notranslate">hatch run /path/to/script.py [args]</code> and Hatch will manage the environment for that script. Such tools could be used as shebang lines on non-Windows systems e.g. <code class="docutils literal notranslate">#!/usr/bin/env hatch run</code>.</li> <li>A script that desires to transition to a directory-type project. A user may be rapidly prototyping locally or in a remote REPL environment and then decide to transition to a more formal project layout if their idea works out. Being able to define dependencies in the script would be very useful to have fully reproducible bug reports.</li> <li>Users that wish to avoid manual dependency management. For example, package managers that have commands to add/remove dependencies or dependency update automation in CI that triggers based on new versions or in response to CVEs <a class="footnote-reference brackets" href="#id11" id="id1">[1]</a>.</li> </ul> </section> <section id="specification"> <h2><a class="toc-backref" href="#specification" role="doc-backlink">Specification</a></h2> This PEP defines a metadata comment block format loosely inspired <a class="footnote-reference brackets" href="#id12" id="id2">[2]</a> by <a class="reference external" href="https://docutils.sourceforge.io/docs/ref/rst/directives.html">reStructuredText Directives</a>. Any Python script may have top-level comment blocks that MUST start with the line <code class="docutils literal notranslate"># /// TYPE</code> where <code class="docutils literal notranslate">TYPE</code> determines how to process the content. That is: a single <code class="docutils literal notranslate">#</code>, followed by a single space, followed by three forward slashes, followed by a single space, followed by the type of metadata. Block MUST end with the line <code class="docutils literal notranslate"># ///</code>. That is: a single <code class="docutils literal notranslate">#</code>, followed by a single space, followed by three forward slashes. The <code class="docutils literal notranslate">TYPE</code> MUST only consist of ASCII letters, numbers and hyphens. Every line between these two lines (<code class="docutils literal notranslate"># /// TYPE</code> and <code class="docutils literal notranslate"># ///</code>) MUST be a comment starting with <code class="docutils literal notranslate">#</code>. If there are characters after the <code class="docutils literal notranslate">#</code> then the first character MUST be a space. The embedded content is formed by taking away the first two characters of each line if the second character is a space, otherwise just the first character (which means the line consists of only a single <code class="docutils literal notranslate">#</code>). Precedence for an ending line <code class="docutils literal notranslate"># ///</code> is given when the next line is not a valid embedded content line as described above. For example, the following is a single fully valid block: <div class="highlight-python notranslate"><div class="highlight"><pre># /// some-toml # embedded-csharp = """ # /// <summary> # /// text # /// # /// </summary> # public class MyClass { } # """ # /// </pre></div> </div> A starting line MUST NOT be placed between another starting line and its ending line. In such cases tools MAY produce an error. Unclosed blocks MUST be ignored. When there are multiple comment blocks of the same <code class="docutils literal notranslate">TYPE</code> defined, tools MUST produce an error. Tools reading embedded metadata MAY respect the standard Python encoding declaration. If they choose not to do so, they MUST process the file as UTF-8. This is the canonical regular expression that MAY be used to parse the metadata: <div class="highlight-text notranslate"><div class="highlight"><pre>(?m)^# /// (?P<type>[a-zA-Z0-9-]+)$\s(?P<content>(^#(| .*)$\s)+)^# ///$ </pre></div> </div> In circumstances where there is a discrepancy between the text specification and the regular expression, the text specification takes precedence. Tools MUST NOT read from metadata blocks with types that have not been standardized by this PEP or future ones. <section id="script-type"> <h3><a class="toc-backref" href="#script-type" role="doc-backlink">script type</a></h3> The first type of metadata block is named <code class="docutils literal notranslate">script</code> which contains script metadata (dependency data and tool configuration). This document MAY include top-level fields <code class="docutils literal notranslate">dependencies</code> and <code class="docutils literal notranslate">requires-python</code>, and MAY optionally include a <code class="docutils literal notranslate">[tool]</code> table. The <code class="docutils literal notranslate">[tool]</code> table MAY be used by any tool, script runner or otherwise, to configure behavior. It has the same semantics as the <a class="pep reference internal" href="../pep-0518/#tool-table" title="PEP 518 – Specifying Minimum Build System Requirements for Python Projects § tool table">tool table</a> in <code class="docutils literal notranslate">pyproject.toml</code>. The top-level fields are: <ul class="simple"> <li><code class="docutils literal notranslate">dependencies</code>: A list of strings that specifies the runtime dependencies of the script. Each entry MUST be a valid <a class="pep reference internal" href="../pep-0508/" title="PEP 508 – Dependency specification for Python Software Packages">PEP 508</a> dependency.</li> <li><code class="docutils literal notranslate">requires-python</code>: A string that specifies the Python version(s) with which the script is compatible. The value of this field MUST be a valid <a class="pep reference internal" href="../pep-0440/#version-specifiers" title="PEP 440 – Version Identification and Dependency Specification § Version specifiers">version specifier</a>.</li> </ul> Script runners MUST error if the specified <code class="docutils literal notranslate">dependencies</code> cannot be provided. Script runners SHOULD error if no version of Python that satisfies the specified <code class="docutils literal notranslate">requires-python</code> can be provided. </section> <section id="example"> <h3><a class="toc-backref" href="#example" role="doc-backlink">Example</a></h3> The following is an example of a script with embedded metadata: <div class="highlight-python notranslate"><div class="highlight"><pre># /// script # requires-python = ">=3.11" # dependencies = [ # "requests<3", # "rich", # ] # /// import requests from rich.pretty import pprint resp = requests.get("https://peps.python.org/api/peps.json") data = resp.json() pprint([(k, v["title"]) for k, v in data.items()][:10]) </pre></div> </div> </section> </section> <section id="reference-implementation"> <h2><a class="toc-backref" href="#reference-implementation" role="doc-backlink">Reference Implementation</a></h2> The following is an example of how to read the metadata on Python 3.11 or higher. <div class="highlight-python notranslate"><div class="highlight"><pre>import re import tomllib REGEX = r'(?m)^# /// (?P<type>[a-zA-Z0-9-]+)$\s(?P<content>(^#(| .*)$\s)+)^# ///$' def read(script: str) -> dict | None: name = 'script' matches = list( filter(lambda m: m.group('type') == name, re.finditer(REGEX, script)) ) if len(matches) > 1: raise ValueError(f'Multiple {name} blocks found') elif len(matches) == 1: content = ''.join( line[2:] if line.startswith('# ') else line[1:] for line in matches[0].group('content').splitlines(keepends=True) ) return tomllib.loads(content) else: return None </pre></div> </div> Often tools will edit dependencies like package managers or dependency update automation in CI. The following is a crude example of modifying the content using the <code class="docutils literal notranslate">tomlkit</code> <a class="reference external" href="https://tomlkit.readthedocs.io/en/latest/">library</a>. <div class="highlight-python notranslate"><div class="highlight"><pre>import re import tomlkit REGEX = r'(?m)^# /// (?P<type>[a-zA-Z0-9-]+)$\s(?P<content>(^#(| .*)$\s)+)^# ///$' def add(script: str, dependency: str) -> str: match = re.search(REGEX, script) content = ''.join( line[2:] if line.startswith('# ') else line[1:] for line in match.group('content').splitlines(keepends=True) ) config = tomlkit.parse(content) config['dependencies'].append(dependency) new_content = ''.join( f'# {line}' if line.strip() else f'#{line}' for line in tomlkit.dumps(config).splitlines(keepends=True) ) start, end = match.span('content') return script[:start] + new_content + script[end:] </pre></div> </div> Note that this example used a library that preserves TOML formatting. This is not a requirement for editing by any means but rather is a “nice to have” feature. The following is an example of how to read a stream of arbitrary metadata blocks. <div class="highlight-python notranslate"><div class="highlight"><pre>import re from typing import Iterator REGEX = r'(?m)^# /// (?P<type>[a-zA-Z0-9-]+)$\s(?P<content>(^#(| .*)$\s)+)^# ///$' def stream(script: str) -> Iterator[tuple[str, str]]: for match in re.finditer(REGEX, script): yield match.group('type'), ''.join( line[2:] if line.startswith('# ') else line[1:] for line in match.group('content').splitlines(keepends=True) ) </pre></div> </div> </section> <section id="backwards-compatibility"> <h2><a class="toc-backref" href="#backwards-compatibility" role="doc-backlink">Backwards Compatibility</a></h2> At the time of writing, the <code class="docutils literal notranslate"># /// script</code> block comment starter does not appear in any Python files <a class="reference external" href="https://github.com/search?q=%22%23+%2F%2F%2F+script%22&type=code">on GitHub</a>. Therefore, there is little risk of existing scripts being broken by this PEP. </section> <section id="security-implications"> <h2><a class="toc-backref" href="#security-implications" role="doc-backlink">Security Implications</a></h2> If a script containing embedded metadata is run using a tool that automatically installs dependencies, this could cause arbitrary code to be downloaded and installed in the user’s environment. The risk here is part of the functionality of the tool being used to run the script, and as such should already be addressed by the tool itself. The only additional risk introduced by this PEP is if an untrusted script with embedded metadata is run, when a potentially malicious dependency or transitive dependency might be installed. This risk is addressed by the normal good practice of reviewing code before running it. Additionally, tools may be able to provide <a class="reference internal" href="#tool-configuration">locking functionality</a> to ameliorate this risk. </section> <section id="how-to-teach-this"> <h2><a class="toc-backref" href="#how-to-teach-this" role="doc-backlink">How to Teach This</a></h2> To embed metadata in a script, define a comment block that starts with the line <code class="docutils literal notranslate"># /// script</code> and ends with the line <code class="docutils literal notranslate"># ///</code>. Every line between those two lines must be a comment and the full content is derived by removing the first two characters. <div class="highlight-python notranslate"><div class="highlight"><pre># /// script # dependencies = [ # "requests<3", # "rich", # ] # requires-python = ">=3.11" # /// </pre></div> </div> The allowed fields are described in the following table: <table class="docutils align-default"> <tbody> <tr class="row-odd"><td>Field</td> <td>Description</td> <td>Tool behavior</td> </tr> <tr class="row-even"><td><code class="docutils literal notranslate">dependencies</code></td> <td>A list of strings that specifies the runtime dependencies of the script. Each entry must be a valid <a class="pep reference internal" href="../pep-0508/" title="PEP 508 – Dependency specification for Python Software Packages">PEP 508</a> dependency.</td> <td>Tools will error if the specified dependencies cannot be provided.</td> </tr> <tr class="row-odd"><td><code class="docutils literal notranslate">requires-python</code></td> <td>A string that specifies the Python version(s) with which the script is compatible. The value of this field must be a valid <a class="pep reference internal" href="../pep-0440/#version-specifiers" title="PEP 440 – Version Identification and Dependency Specification § Version specifiers">version specifier</a>.</td> <td>Tools might error if no version of Python that satisfies the constraint can be executed.</td> </tr> </tbody> </table> In addition, a <code class="docutils literal notranslate">[tool]</code> table is allowed. Details of what is permitted are similar to what is permitted in <code class="docutils literal notranslate">pyproject.toml</code>, but precise information must be included in the documentation of the relevant tool. It is up to individual tools whether or not their behavior is altered based on the embedded metadata. For example, every script runner may not be able to provide an environment for specific Python versions as defined by the <code class="docutils literal notranslate">requires-python</code> field. The <a class="pep reference internal" href="../pep-0518/#tool-table" title="PEP 518 – Specifying Minimum Build System Requirements for Python Projects § tool table">tool table</a> may be used by any tool, script runner or otherwise, to configure behavior. </section> <section id="recommendations"> <h2><a class="toc-backref" href="#recommendations" role="doc-backlink">Recommendations</a></h2> Tools that support managing different versions of Python should attempt to use the highest available version of Python that is compatible with the script’s <code class="docutils literal notranslate">requires-python</code> metadata, if defined. </section> <section id="tooling-buy-in"> <h2><a class="toc-backref" href="#tooling-buy-in" role="doc-backlink">Tooling buy-in</a></h2> The following is a list of tools that have expressed support for this PEP or have committed to implementing support should it be accepted: <ul class="simple"> <li><a class="reference external" href="https://discuss.python.org/t/31151/15">Pantsbuild and Pex</a>: expressed support for any way to define dependencies and also features that this PEP considers as valid use cases such as building packages from scripts and embedding tool configuration</li> <li><a class="reference external" href="https://discuss.python.org/t/31151/16">Mypy</a> and <a class="reference external" href="https://discuss.python.org/t/31151/42">Ruff</a>: strongly expressed support for embedding tool configuration as it would solve existing pain points for users</li> <li><a class="reference external" href="https://discuss.python.org/t/31151/53">Hatch</a>: (author of this PEP) expressed support for all aspects of this PEP, and will be one of the first tools to support running scripts with specifically configured Python versions</li> </ul> </section> <section id="rejected-ideas"> <h2><a class="toc-backref" href="#rejected-ideas" role="doc-backlink">Rejected Ideas</a></h2> <section id="why-not-use-a-comment-block-resembling-requirements-txt"> <h3><a class="toc-backref" href="#why-not-use-a-comment-block-resembling-requirements-txt" role="doc-backlink">Why not use a comment block resembling requirements.txt?</a></h3> This PEP considers there to be different types of users for whom Python code would live as single-file scripts: <ul class="simple"> <li>Non-programmers who are just using Python as a scripting language to achieve a specific task. These users are unlikely to be familiar with concepts of operating systems like shebang lines or the <code class="docutils literal notranslate">PATH</code> environment variable. Some examples:<ul> <li>The average person, perhaps at a workplace, who wants to write a script to automate something for efficiency or to reduce tedium</li> <li>Someone doing data science or machine learning in industry or academia who wants to write a script to analyze some data or for research purposes. These users are special in that, although they have limited programming knowledge, they learn from sources like StackOverflow and blogs that have a programming bent and are increasingly likely to be part of communities that share knowledge and code. Therefore, a non-trivial number of these users will have some familiarity with things like Git(Hub), Jupyter, HuggingFace, etc.</li> </ul> </li> <li>Non-programmers who manage operating systems e.g. a sysadmin. These users are able to set up <code class="docutils literal notranslate">PATH</code>, for example, but are unlikely to be familiar with Python concepts like virtual environments. These users often operate in isolation and have limited need to gain exposure to tools intended for sharing like Git.</li> <li>Programmers who manage operating systems/infrastructure e.g. SREs. These users are not very likely to be familiar with Python concepts like virtual environments, but are likely to be familiar with Git and most often use it to version control everything required to manage infrastructure like Python scripts and Kubernetes config.</li> <li>Programmers who write scripts primarily for themselves. These users over time accumulate a great number of scripts in various languages that they use to automate their workflow and often store them in a single directory, that is potentially version controlled for persistence. Non-Windows users may set up each Python script with a shebang line pointing to the desired Python executable or script runner.</li> </ul> This PEP argues that the proposed TOML-based metadata format is the best for each category of user and that the requirements-like block comment is only approachable for those who have familiarity with <code class="docutils literal notranslate">requirements.txt</code>, which represents a small subset of users. <ul> <li>For the average person automating a task or the data scientist, they are already starting with zero context and are unlikely to be familiar with TOML nor <code class="docutils literal notranslate">requirements.txt</code>. These users will very likely rely on snippets found online via a search engine or utilize AI in the form of a chat bot or direct code completion software. The similarity with dependency information stored in <code class="docutils literal notranslate">pyproject.toml</code> will provide useful search results relatively quickly, and while the <code class="docutils literal notranslate">pyproject.toml</code> format and the script metadata format are not the same, any resulting discrepancies are unlikely to be difficult for the intended users to resolve.Additionally, these users are most susceptible to formatting quirks and syntax errors. TOML is a well-defined format with existing online validators that features assignment that is compatible with Python expressions and has no strict indenting rules. The block comment format on the other hand could be easily malformed by forgetting the colon, for example, and debugging why it’s not working with a search engine would be a difficult task for such a user. </li> <li>For the sysadmin types, they are equally unlikely as the previously described users to be familiar with TOML or <code class="docutils literal notranslate">requirements.txt</code>. For either format they would have to read documentation. They would likely be more comfortable with TOML since they are used to structured data formats and there would be less perceived magic in their systems.Additionally, for maintenance of their systems <code class="docutils literal notranslate">/// script</code> would be much easier to search for from a shell than a block comment with potentially numerous extensions over time. </li> <li>For the SRE types, they are likely to be familiar with TOML already from other projects that they might have to work with like configuring the <a class="reference external" href="https://docs.gitlab.com/runner/configuration/advanced-configuration.html">GitLab Runner</a> or <a class="reference external" href="https://buildpacks.io/docs/reference/config/">Cloud Native Buildpacks</a>.These users are responsible for the security of their systems and most likely have security scanners set up to automatically open PRs to update versions of dependencies. Such automated tools like Dependabot would have a much easier time using existing TOML libraries than writing their own custom parser for a block comment format. </li> <li>For the programmer types, they are more likely to be familiar with TOML than they have ever seen a <code class="docutils literal notranslate">requirements.txt</code> file, unless they are a Python programmer who has had previous experience with writing applications. In the case of experience with the requirements format, it necessarily means that they are at least somewhat familiar with the ecosystem and therefore it is safe to assume they know what TOML is.Another benefit of this PEP to these users is that their IDEs like Visual Studio Code would be able to provide TOML syntax highlighting much more easily than each writing custom logic for this feature. </li> </ul> Additionally, since the original block comment alternative format (double <code class="docutils literal notranslate">#</code>) went against the recommendation of <a class="pep reference internal" href="../pep-0008/" title="PEP 8 – Style Guide for Python Code">PEP 8</a> and as a result linters and IDE auto-formatters that respected the recommendation would <a class="reference external" href="https://discuss.python.org/t/29905/247">fail by default</a>, the final proposal uses standard comments starting with a single <code class="docutils literal notranslate">#</code> character without any obvious start nor end sequence. The concept of regular comments that do not appear to be intended for machines (i.e. <a class="reference external" href="https://docs.python.org/3/reference/lexical_analysis.html#encoding-declarations">encoding declarations</a>) affecting behavior would not be customary to users of Python and goes directly against the “explicit is better than implicit” foundational principle. Users typing what to them looks like prose could alter runtime behavior. This PEP takes the view that the possibility of that happening, even when a tool has been set up as such (maybe by a sysadmin), is unfriendly to users. Finally, and critically, the alternatives to this PEP like <a class="pep reference internal" href="../pep-0722/" title="PEP 722 – Dependency specification for single-file scripts">PEP 722</a> do not satisfy the use cases enumerated herein, such as setting the supported Python versions, the eventual building of scripts into packages, and the ability to have machines edit metadata on behalf of users. It is very likely that the requests for such features persist and conceivable that another PEP in the future would allow for the embedding of such metadata. At that point there would be multiple ways to achieve the same thing which goes against our foundational principle of “there should be one - and preferably only one - obvious way to do it”. </section> <section id="why-not-use-a-multi-line-string"> <h3><a class="toc-backref" href="#why-not-use-a-multi-line-string" role="doc-backlink">Why not use a multi-line string?</a></h3> A previous version of this PEP proposed that the metadata be stored as follows: <div class="highlight-python notranslate"><div class="highlight"><pre>__pyproject__ = """ ... """ </pre></div> </div> The most significant problem with this proposal is that the embedded TOML would be limited in the following ways: <ul class="simple"> <li>It would not be possible to use multi-line double-quoted strings in the TOML as that would conflict with the Python string containing the document. Many TOML writers do not preserve style and may potentially produce output that would be malformed.</li> <li>The way in which character escaping works in Python strings is not quite the way it works in TOML strings. It would be possible to preserve a one-to-one character mapping by enforcing raw strings, but this <code class="docutils literal notranslate">r</code> prefix requirement may be potentially confusing to users.</li> </ul> </section> <section id="why-not-reuse-core-metadata-fields"> <h3><a class="toc-backref" href="#why-not-reuse-core-metadata-fields" role="doc-backlink">Why not reuse core metadata fields?</a></h3> A previous version of this PEP proposed to reuse the existing <a class="reference external" href="https://packaging.python.org/en/latest/specifications/declaring-project-metadata/">metadata standard</a> that is used to describe projects. There are two significant problems with this proposal: <ul class="simple"> <li>The <code class="docutils literal notranslate">name</code> and <code class="docutils literal notranslate">version</code> fields are required and changing that would require its own PEP</li> <li>Reusing the data is <a class="reference external" href="https://snarky.ca/differentiating-between-writing-down-dependencies-to-use-packages-and-for-packages-themselves/">fundamentally a misuse of it</a></li> </ul> </section> <section id="why-not-limit-to-specific-metadata-fields"> <h3><a class="toc-backref" href="#why-not-limit-to-specific-metadata-fields" role="doc-backlink">Why not limit to specific metadata fields?</a></h3> By limiting the metadata to just <code class="docutils literal notranslate">dependencies</code>, we would prevent the known use case of tools that support managing Python installations, which would allows users to target specific versions of Python for new syntax or standard library functionality. </section> <section id="why-not-limit-tool-configuration"> <h3><a class="toc-backref" href="#why-not-limit-tool-configuration" role="doc-backlink">Why not limit tool configuration?</a></h3> By not allowing the <code class="docutils literal notranslate">[tool]</code> table, we would prevent known functionality that would benefit users. For example: <ul> <li>A script runner may support injecting of dependency resolution data for an embedded lock file (this is what Go’s <code class="docutils literal notranslate">gorun</code> can do).</li> <li>A script runner may support configuration instructing to run scripts in containers for situations in which there is no cross-platform support for a dependency or if the setup is too complex for the average user like when requiring Nvidia drivers. Situations like this would allow users to proceed with what they want to do whereas otherwise they may stop at that point altogether.</li> <li>Tools may wish to experiment with features to ease development burden for users such as the building of single-file scripts into packages. We received <a class="reference external" href="https://discuss.python.org/t/31151/9">feedback</a> stating that there are already tools that exist in the wild that build wheels and source distributions from single files.The author of the Rust RFC for embedding metadata <a class="reference external" href="https://discuss.python.org/t/29905/179">mentioned to us</a> that they are actively looking into that as well based on user feedback saying that there is unnecessary friction with managing small projects. There has been <a class="reference external" href="https://discuss.python.org/t/31151/15">a commitment</a> to support this by at least one major build system. </li> </ul> </section> <section id="why-not-limit-tool-behavior"> <h3><a class="toc-backref" href="#why-not-limit-tool-behavior" role="doc-backlink">Why not limit tool behavior?</a></h3> A previous version of this PEP proposed that non-script running tools SHOULD NOT modify their behavior when the script is not the sole input to the tool. For example, if a linter is invoked with the path to a directory, it SHOULD behave the same as if zero files had embedded metadata. This was done as a precaution to avoid tool behavior confusion and generating various feature requests for tools to support this PEP. However, during discussion we received <a class="reference external" href="https://discuss.python.org/t/31151/16">feedback</a> from maintainers of tools that this would be undesirable and potentially confusing to users. Additionally, this may allow for a universally easier way to configure tools in certain circumstances and solve existing issues. </section> <section id="why-not-just-set-up-a-python-project-with-a-pyproject-toml"> <h3><a class="toc-backref" href="#why-not-just-set-up-a-python-project-with-a-pyproject-toml" role="doc-backlink">Why not just set up a Python project with a <code class="docutils literal notranslate">pyproject.toml</code>?</a></h3> Again, a key issue here is that the target audience for this proposal is people writing scripts which aren’t intended for distribution. Sometimes scripts will be “shared”, but this is far more informal than “distribution” - it typically involves sending a script via an email with some written instructions on how to run it, or passing someone a link to a GitHub gist. Expecting such users to learn the complexities of Python packaging is a significant step up in complexity, and would almost certainly give the impression that “Python is too hard for scripts”. In addition, if the expectation here is that the <code class="docutils literal notranslate">pyproject.toml</code> will somehow be designed for running scripts in place, that’s a new feature of the standard that doesn’t currently exist. At a minimum, this isn’t a reasonable suggestion until the <a class="reference external" href="https://discuss.python.org/t/projects-that-arent-meant-to-generate-a-wheel-and-pyproject-toml/29684">current discussion on Discourse</a> about using <code class="docutils literal notranslate">pyproject.toml</code> for projects that won’t be distributed as wheels is resolved. And even then, it doesn’t address the “sending someone a script in a gist or email” use case. </section> <section id="why-not-infer-the-requirements-from-import-statements"> <h3><a class="toc-backref" href="#why-not-infer-the-requirements-from-import-statements" role="doc-backlink">Why not infer the requirements from import statements?</a></h3> The idea would be to automatically recognize <code class="docutils literal notranslate">import</code> statements in the source file and turn them into a list of requirements. However, this is infeasible for several reasons. First, the points above about the necessity to keep the syntax easily parsable, for all Python versions, also by tools written in other languages, apply equally here. Second, PyPI and other package repositories conforming to the Simple Repository API do not provide a mechanism to resolve package names from the module names that are imported (see also <a class="reference external" href="https://discuss.python.org/t/record-the-top-level-names-of-a-wheel-in-metadata/29494">this related discussion</a>). Third, even if repositories did offer this information, the same import name may correspond to several packages on PyPI. One might object that disambiguating which package is wanted would only be needed if there are several projects providing the same import name. However, this would make it easy for anyone to unintentionally or malevolently break working scripts, by uploading a package to PyPI providing an import name that is the same as an existing project. The alternative where, among the candidates, the first package to have been registered on the index is chosen, would be confusing in case a popular package is developed with the same import name as an existing obscure package, and even harmful if the existing package is malware intentionally uploaded with a sufficiently generic import name that has a high probability of being reused. A related idea would be to attach the requirements as comments to the import statements instead of gathering them in a block, with a syntax such as: <div class="highlight-default notranslate"><div class="highlight"><pre>import numpy as np # requires: numpy import rich # requires: rich </pre></div> </div> This still suffers from parsing difficulties. Also, where to place the comment in the case of multiline imports is ambiguous and may look ugly: <div class="highlight-default notranslate"><div class="highlight"><pre>from PyQt5.QtWidgets import ( QCheckBox, QComboBox, QDialog, QDialogButtonBox, QGridLayout, QLabel, QSpinBox, QTextEdit ) # requires: PyQt5 </pre></div> </div> Furthermore, this syntax cannot behave as might be intuitively expected in all situations. Consider: <div class="highlight-default notranslate"><div class="highlight"><pre>import platform if platform.system() == "Windows": import pywin32 # requires: pywin32 </pre></div> </div> Here, the user’s intent is that the package is only required on Windows, but this cannot be understood by the script runner (the correct way to write it would be <code class="docutils literal notranslate">requires: pywin32 ; sys_platform == 'win32'</code>). (Thanks to Jean Abou-Samra for the clear discussion of this point) </section> <section id="why-not-use-a-requirements-file-for-dependencies"> <h3><a class="toc-backref" href="#why-not-use-a-requirements-file-for-dependencies" role="doc-backlink">Why not use a requirements file for dependencies?</a></h3> Putting your requirements in a requirements file, doesn’t require a PEP. You can do that right now, and in fact it’s quite likely that many adhoc solutions do this. However, without a standard, there’s no way of knowing how to locate a script’s dependency data. And furthermore, the requirements file format is pip-specific, so tools relying on it are depending on a pip implementation detail. So in order to make a standard, two things would be required: <ol class="arabic simple"> <li>A standardised replacement for the requirements file format.</li> <li>A standard for how to locate the requirements file for a given script.</li> </ol> The first item is a significant undertaking. It has been discussed on a number of occasions, but so far no-one has attempted to actually do it. The most likely approach would be for standards to be developed for individual use cases currently addressed with requirements files. One option here would be for this PEP to simply define a new file format which is simply a text file containing <a class="pep reference internal" href="../pep-0508/" title="PEP 508 – Dependency specification for Python Software Packages">PEP 508</a> requirements, one per line. That would just leave the question of how to locate that file. The “obvious” solution here would be to do something like name the file the same as the script, but with a <code class="docutils literal notranslate">.reqs</code> extension (or something similar). However, this still requires two files, where currently only a single file is needed, and as such, does not match the “better batch file” model (shell scripts and batch files are typically self-contained). It requires the developer to remember to keep the two files together, and this may not always be possible. For example, system administration policies may require that all files in a certain directory are executable (the Linux filesystem standards require this of <code class="docutils literal notranslate">/usr/bin</code>, for example). And some methods of sharing a script (for example, publishing it on a text file sharing service like Github’s gist, or a corporate intranet) may not allow for deriving the location of an associated requirements file from the script’s location (tools like <code class="docutils literal notranslate">pipx</code> support running a script directly from a URL, so “download and unpack a zip of the script and its dependencies” may not be an appropriate requirement). Essentially, though, the issue here is that there is an explicitly stated requirement that the format supports storing dependency data in the script file itself. Solutions that don’t do that are simply ignoring that requirement. </section> <section id="why-not-use-possibly-restricted-python-syntax"> <h3><a class="toc-backref" href="#why-not-use-possibly-restricted-python-syntax" role="doc-backlink">Why not use (possibly restricted) Python syntax?</a></h3> This would typically involve storing metadata as multiple special variables, such as the following. <div class="highlight-python notranslate"><div class="highlight"><pre>__requires_python__ = ">=3.11" __dependencies__ = [ "requests", "click", ] </pre></div> </div> The most significant problem with this proposal is that it requires all consumers of the dependency data to implement a Python parser. Even if the syntax is restricted, the rest of the script will use the full Python syntax, and trying to define a syntax which can be successfully parsed in isolation from the surrounding code is likely to be extremely difficult and error-prone. Furthermore, Python’s syntax changes in every release. If extracting dependency data needs a Python parser, the parser will need to know which version of Python the script is written for, and the overhead for a generic tool of having a parser that can handle multiple versions of Python is unsustainable. With this approach there is the potential to clutter scripts with many variables as new extensions get added. Additionally, intuiting which metadata fields correspond to which variable names would cause confusion for users. It is worth noting, though, that the <code class="docutils literal notranslate">pip-run</code> utility does implement (an extended form of) this approach. <a class="reference external" href="https://github.com/jaraco/pip-run/issues/44">Further discussion</a> of the <code class="docutils literal notranslate">pip-run</code> design is available on the project’s issue tracker. </section> <section id="what-about-local-dependencies"> <h3><a class="toc-backref" href="#what-about-local-dependencies" role="doc-backlink">What about local dependencies?</a></h3> These can be handled without needing special metadata and tooling, simply by adding the location of the dependencies to <code class="docutils literal notranslate">sys.path</code>. This PEP simply isn’t needed for this case. If, on the other hand, the “local dependencies” are actual distributions which are published locally, they can be specified as usual with a <a class="pep reference internal" href="../pep-0508/" title="PEP 508 – Dependency specification for Python Software Packages">PEP 508</a> requirement, and the local package index specified when running a tool by using the tool’s UI for that. </section> </section> <section id="open-issues"> <h2><a class="toc-backref" href="#open-issues" role="doc-backlink">Open Issues</a></h2> None at this point. </section> <section id="footnotes"> <h2><a class="toc-backref" href="#footnotes" role="doc-backlink">Footnotes</a></h2> <aside class="footnote-list brackets"> <aside class="footnote brackets" id="id11" role="doc-footnote"> <dt class="label" id="id11">[<a href="#id1">1</a>]</dt> <dd>A large number of users use scripts that are version controlled. For example, <a class="reference internal" href="#comment-block">the SREs that were mentioned</a> or projects that require special maintenance like the <a class="reference external" href="https://github.com/aws/aws-cli/tree/4393dcdf044a5275000c9c193d1933c07a08fdf1/scripts">AWS CLI</a> or <a class="reference external" href="https://github.com/kovidgoyal/calibre/tree/master/setup">Calibre</a>.</aside> <aside class="footnote brackets" id="id12" role="doc-footnote"> <dt class="label" id="id12">[<a href="#id2">2</a>]</dt> <dd>The syntax is taken directly from the final resolution of the <a class="reference external" href="https://github.com/facelessuser/pymdown-extensions/discussions/1973">Blocks extension</a> to <a class="reference external" href="https://github.com/Python-Markdown/markdown">Python Markdown</a>.</aside> </aside> </section> <section id="copyright"> <h2><a class="toc-backref" href="#copyright" role="doc-backlink">Copyright</a></h2> This document is placed in the public domain or under the CC0-1.0-Universal license, whichever is more permissive. </section> </section> <hr class="docutils" /> Source: <a class="reference external" href="https://github.com/python/peps/blob/main/peps/pep-0723.rst">https://github.com/python/peps/blob/main/peps/pep-0723.rst</a> Last modified: <a class="reference external" href="https://github.com/python/peps/commits/main/peps/pep-0723.rst">2024-10-17 12:49:39 GMT</a> </article> <nav id="pep-sidebar"> <h2>Contents</h2> <ul> <li><a class="reference internal" href="#abstract">Abstract</a></li> <li><a class="reference internal" href="#motivation">Motivation</a></li> <li><a class="reference internal" href="#rationale">Rationale</a></li> <li><a class="reference internal" href="#specification">Specification</a><ul> <li><a class="reference internal" href="#script-type">script type</a></li> <li><a class="reference internal" href="#example">Example</a></li> </ul> </li> <li><a class="reference internal" href="#reference-implementation">Reference Implementation</a></li> <li><a class="reference internal" href="#backwards-compatibility">Backwards Compatibility</a></li> <li><a class="reference internal" href="#security-implications">Security Implications</a></li> <li><a class="reference internal" href="#how-to-teach-this">How to Teach This</a></li> <li><a class="reference internal" href="#recommendations">Recommendations</a></li> <li><a class="reference internal" href="#tooling-buy-in">Tooling buy-in</a></li> <li><a class="reference internal" href="#rejected-ideas">Rejected Ideas</a><ul> <li><a class="reference internal" href="#why-not-use-a-comment-block-resembling-requirements-txt">Why not use a comment block resembling requirements.txt?</a></li> <li><a class="reference internal" href="#why-not-use-a-multi-line-string">Why not use a multi-line string?</a></li> <li><a class="reference internal" href="#why-not-reuse-core-metadata-fields">Why not reuse core metadata fields?</a></li> <li><a class="reference internal" href="#why-not-limit-to-specific-metadata-fields">Why not limit to specific metadata fields?</a></li> <li><a class="reference internal" href="#why-not-limit-tool-configuration">Why not limit tool configuration?</a></li> <li><a class="reference internal" href="#why-not-limit-tool-behavior">Why not limit tool behavior?</a></li> <li><a class="reference internal" href="#why-not-just-set-up-a-python-project-with-a-pyproject-toml">Why not just set up a Python project with a <code class="docutils literal notranslate">pyproject.toml</code>?</a></li> <li><a class="reference internal" href="#why-not-infer-the-requirements-from-import-statements">Why not infer the requirements from import statements?</a></li> <li><a class="reference internal" href="#why-not-use-a-requirements-file-for-dependencies">Why not use a requirements file for dependencies?</a></li> <li><a class="reference internal" href="#why-not-use-possibly-restricted-python-syntax">Why not use (possibly restricted) Python syntax?</a></li> <li><a class="reference internal" href="#what-about-local-dependencies">What about local dependencies?</a></li> </ul> </li> <li><a class="reference internal" href="#open-issues">Open Issues</a></li> <li><a class="reference internal" href="#footnotes">Footnotes</a></li> <li><a class="reference internal" href="#copyright">Copyright</a></li> </ul> <a id="source" href="https://github.com/python/peps/blob/main/peps/pep-0723.rst">Page Source (GitHub)</a> </nav> </section> <script src="../_static/colour_scheme.js"></script> <script src="../_static/wrap_tables.js"></script> <script src="../_static/sticky_banner.js"></script> </body> </html>