CINXE.COM

<!DOCTYPE html> <html lang="en"> <head> <meta charset="utf-8"> <meta name="viewport" content="width=device-width, initial-scale=1.0"> <meta name="color-scheme" content="light dark"> <title>PEP 680 – tomllib: Support for Parsing TOML in the Standard Library | peps.python.org</title> <link rel="shortcut icon" href="../_static/py.png"> <link rel="canonical" href="https://peps.python.org/pep-0680/"> <link rel="stylesheet" href="../_static/style.css" type="text/css"> <link rel="stylesheet" href="../_static/mq.css" type="text/css"> <link rel="stylesheet" href="../_static/pygments.css" type="text/css" media="(prefers-color-scheme: light)" id="pyg-light"> <link rel="stylesheet" href="../_static/pygments_dark.css" type="text/css" media="(prefers-color-scheme: dark)" id="pyg-dark"> <link rel="alternate" type="application/rss+xml" title="Latest PEPs" href="https://peps.python.org/peps.rss"> <meta property="og:title" content='PEP 680 – tomllib: Support for Parsing TOML in the Standard Library | peps.python.org'> <meta property="og:description" content="This PEP proposes adding the tomllib module to the standard library for parsing TOML (Tom’s Obvious Minimal Language, https://toml.io)."> <meta property="og:type" content="website"> <meta property="og:url" content="https://peps.python.org/pep-0680/"> <meta property="og:site_name" content="Python Enhancement Proposals (PEPs)"> <meta property="og:image" content="https://peps.python.org/_static/og-image.png"> <meta property="og:image:alt" content="Python PEPs"> <meta property="og:image:width" content="200"> <meta property="og:image:height" content="200"> <meta name="description" content="This PEP proposes adding the tomllib module to the standard library for parsing TOML (Tom’s Obvious Minimal Language, https://toml.io)."> <meta name="theme-color" content="#3776ab"> </head> <body> <svg xmlns="http://www.w3.org/2000/svg" style="display: none;"> <symbol id="svg-sun-half" viewBox="0 0 24 24" pointer-events="all"> <title>Following system colour scheme</title> <svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"> <circle cx="12" cy="12" r="9"></circle> <path d="M12 3v18m0-12l4.65-4.65M12 14.3l7.37-7.37M12 19.6l8.85-8.85"></path> </svg> </symbol> <symbol id="svg-moon" viewBox="0 0 24 24" pointer-events="all"> <title>Selected dark colour scheme</title> <svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"> <path stroke="none" d="M0 0h24v24H0z" fill="none"></path> <path d="M12 3c.132 0 .263 0 .393 0a7.5 7.5 0 0 0 7.92 12.446a9 9 0 1 1 -8.313 -12.454z"></path> </svg> </symbol> <symbol id="svg-sun" viewBox="0 0 24 24" pointer-events="all"> <title>Selected light colour scheme</title> <svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"> <circle cx="12" cy="12" r="5"></circle> <line x1="12" y1="1" x2="12" y2="3"></line> <line x1="12" y1="21" x2="12" y2="23"></line> <line x1="4.22" y1="4.22" x2="5.64" y2="5.64"></line> <line x1="18.36" y1="18.36" x2="19.78" y2="19.78"></line> <line x1="1" y1="12" x2="3" y2="12"></line> <line x1="21" y1="12" x2="23" y2="12"></line> <line x1="4.22" y1="19.78" x2="5.64" y2="18.36"></line> <line x1="18.36" y1="5.64" x2="19.78" y2="4.22"></line> </svg> </symbol> </svg> <script> document.documentElement.dataset.colour_scheme = localStorage.getItem("colour_scheme") || "auto" </script> <section id="pep-page-section"> <header> <h1>Python Enhancement Proposals</h1> <ul class="breadcrumbs"> <li><a href="https://www.python.org/" title="The Python Programming Language">Python</a> » </li> <li><a href="../pep-0000/">PEP Index</a> » </li> <li>PEP 680</li> </ul> <button id="colour-scheme-cycler" onClick="setColourScheme(nextColourScheme())"> <svg aria-hidden="true" class="colour-scheme-icon-when-auto"><use href="#svg-sun-half"></use></svg> <svg aria-hidden="true" class="colour-scheme-icon-when-dark"><use href="#svg-moon"></use></svg> <svg aria-hidden="true" class="colour-scheme-icon-when-light"><use href="#svg-sun"></use></svg> Toggle light / dark / auto colour theme </button> </header> <article> <section id="pep-content"> <h1 class="page-title">PEP 680 – tomllib: Support for Parsing TOML in the Standard Library</h1> <dl class="rfc2822 field-list simple"> <dt class="field-odd">Author:</dt> <dd class="field-odd">Taneli Hukkinen, Shantanu Jain <hauntsaninja at gmail.com></dd> <dt class="field-even">Sponsor:</dt> <dd class="field-even">Petr Viktorin <encukou at gmail.com></dd> <dt class="field-odd">Discussions-To:</dt> <dd class="field-odd"><a class="reference external" href="https://discuss.python.org/t/13040">Discourse thread</a></dd> <dt class="field-even">Status:</dt> <dd class="field-even"><abbr title="Accepted and implementation complete, or no longer active">Final</abbr></dd> <dt class="field-odd">Type:</dt> <dd class="field-odd"><abbr title="Normative PEP with a new feature for Python, implementation change for CPython or interoperability standard for the ecosystem">Standards Track</abbr></dd> <dt class="field-even">Created:</dt> <dd class="field-even">01-Jan-2022</dd> <dt class="field-odd">Python-Version:</dt> <dd class="field-odd">3.11</dd> <dt class="field-even">Post-History:</dt> <dd class="field-even"><a class="reference external" href="https://mail.python.org/archives/list/python-ideas@python.org/thread/IWJ3I32A4TY6CIVQ6ONPEBPWP4TOV2V7/" title="Python-Ideas thread">09-Dec-2021</a>, <a class="reference external" href="https://discuss.python.org/t/pep-680-tomllib-support-for-parsing-toml-in-the-standard-library/13040" title="Discourse thread">27-Jan-2022</a></dd> <dt class="field-odd">Resolution:</dt> <dd class="field-odd"><a class="reference external" href="https://mail.python.org/archives/list/python-dev@python.org/thread/3AHGWYY562HHO55L4Z2OVYUFZP5W73IS/">Python-Dev thread</a></dd> </dl> <hr class="docutils" /> <section id="contents"> <details><summary>Table of Contents</summary><ul class="simple"> <li><a class="reference internal" href="#abstract">Abstract</a></li> <li><a class="reference internal" href="#motivation">Motivation</a></li> <li><a class="reference internal" href="#rationale">Rationale</a></li> <li><a class="reference internal" href="#specification">Specification</a></li> <li><a class="reference internal" href="#maintenance-implications">Maintenance Implications</a><ul> <li><a class="reference internal" href="#stability-of-toml">Stability of TOML</a></li> <li><a class="reference internal" href="#maintainability-of-proposed-implementation">Maintainability of proposed implementation</a></li> <li><a class="reference internal" href="#toml-support-a-slippery-slope-for-other-things">TOML support a slippery slope for other things</a></li> </ul> </li> <li><a class="reference internal" href="#backwards-compatibility">Backwards Compatibility</a></li> <li><a class="reference internal" href="#security-implications">Security Implications</a></li> <li><a class="reference internal" href="#how-to-teach-this">How to Teach This</a></li> <li><a class="reference internal" href="#reference-implementation">Reference Implementation</a></li> <li><a class="reference internal" href="#rejected-ideas">Rejected Ideas</a><ul> <li><a class="reference internal" href="#basing-on-another-toml-implementation">Basing on another TOML implementation</a></li> <li><a class="reference internal" href="#including-an-api-for-writing-toml">Including an API for writing TOML</a></li> <li><a class="reference internal" href="#assorted-api-details">Assorted API details</a><ul> <li><a class="reference internal" href="#types-accepted-as-the-first-argument-of-tomllib-load">Types accepted as the first argument of <code class="docutils literal notranslate">tomllib.load</code></a></li> <li><a class="reference internal" href="#type-accepted-as-the-first-argument-of-tomllib-loads">Type accepted as the first argument of <code class="docutils literal notranslate">tomllib.loads</code></a></li> </ul> </li> <li><a class="reference internal" href="#controlling-the-type-of-mappings-returned-by-tomllib-load-s">Controlling the type of mappings returned by <code class="docutils literal notranslate">tomllib.load[s]</code></a></li> <li><a class="reference internal" href="#removing-support-for-parse-float-in-tomllib-load-s">Removing support for <code class="docutils literal notranslate">parse_float</code> in <code class="docutils literal notranslate">tomllib.load[s]</code></a></li> <li><a class="reference internal" href="#alternative-names-for-the-module">Alternative names for the module</a></li> </ul> </li> <li><a class="reference internal" href="#previous-discussion">Previous Discussion</a></li> <li><a class="reference internal" href="#appendix-a-differences-between-proposed-api-and-toml">Appendix A: Differences between proposed API and <code class="docutils literal notranslate">toml</code></a></li> <li><a class="reference internal" href="#copyright">Copyright</a></li> </ul> </details></section> <div class="pep-banner canonical-doc sticky-banner admonition important"> Important This PEP is a historical document. The up-to-date, canonical documentation can now be found at <a class="reference external" href="https://docs.python.org/3/library/tomllib.html#module-tomllib" title="(in Python v3.13)"><code class="docutils literal notranslate">tomllib</code></a>. × See <a class="pep reference internal" href="../pep-0001/" title="PEP 1 – PEP Purpose and Guidelines">PEP 1</a> for how to propose changes. </div> <section id="abstract"> <h2><a class="toc-backref" href="#abstract" role="doc-backlink">Abstract</a></h2> This PEP proposes adding the <code class="docutils literal notranslate">tomllib</code> module to the standard library for parsing TOML (Tom’s Obvious Minimal Language, <a class="reference external" href="https://toml.io/en/">https://toml.io</a>). </section> <section id="motivation"> <h2><a class="toc-backref" href="#motivation" role="doc-backlink">Motivation</a></h2> TOML is the format of choice for Python packaging, as evidenced by <a class="pep reference internal" href="../pep-0517/" title="PEP 517 – A build-system independent format for source trees">PEP 517</a>, <a class="pep reference internal" href="../pep-0518/" title="PEP 518 – Specifying Minimum Build System Requirements for Python Projects">PEP 518</a> and <a class="pep reference internal" href="../pep-0621/" title="PEP 621 – Storing project metadata in pyproject.toml">PEP 621</a>. This creates a bootstrapping problem for Python build tools, forcing them to vendor a TOML parsing package or employ other undesirable workarounds, and causes serious issues for repackagers and other downstream consumers. Including TOML support in the standard library would neatly solve all of these issues. Further, many Python tools are now configurable via TOML, such as <code class="docutils literal notranslate">black</code>, <code class="docutils literal notranslate">mypy</code>, <code class="docutils literal notranslate">pytest</code>, <code class="docutils literal notranslate">tox</code>, <code class="docutils literal notranslate">pylint</code> and <code class="docutils literal notranslate">isort</code>. Many that are not, such as <code class="docutils literal notranslate">flake8</code>, cite the lack of standard library support as a <a class="reference external" href="https://github.com/PyCQA/flake8/issues/234#issuecomment-812800722">main reason why</a>. Given the special place TOML already has in the Python ecosystem, it makes sense for it to be an included battery. Finally, TOML as a format is increasingly popular (for the reasons outlined in <a class="pep reference internal" href="../pep-0518/" title="PEP 518 – Specifying Minimum Build System Requirements for Python Projects">PEP 518</a>), with various Python TOML libraries having about 2000 reverse dependencies on PyPI (for comparison, <code class="docutils literal notranslate">requests</code> has about 28000 reverse dependencies). Hence, this is likely to be a generally useful addition, even looking beyond the needs of Python packaging and related tools. </section> <section id="rationale"> <h2><a class="toc-backref" href="#rationale" role="doc-backlink">Rationale</a></h2> This PEP proposes basing the standard library support for reading TOML on the third-party library <code class="docutils literal notranslate">tomli</code> (<a class="reference external" href="https://github.com/hukkin/tomli">github.com/hukkin/tomli</a>). Many projects have recently switched to using <code class="docutils literal notranslate">tomli</code>, such as <code class="docutils literal notranslate">pip</code>, <code class="docutils literal notranslate">build</code>, <code class="docutils literal notranslate">pytest</code>, <code class="docutils literal notranslate">mypy</code>, <code class="docutils literal notranslate">black</code>, <code class="docutils literal notranslate">flit</code>, <code class="docutils literal notranslate">coverage</code>, <code class="docutils literal notranslate">setuptools-scm</code> and <code class="docutils literal notranslate">cibuildwheel</code>. <code class="docutils literal notranslate">tomli</code> is actively maintained and well-tested. It is about 800 lines of code with 100% test coverage, and passes all tests in the <a class="reference external" href="https://github.com/toml-lang/compliance/pull/8">proposed official TOML compliance test suite</a>, as well as <a class="reference external" href="https://github.com/BurntSushi/toml-test">the more established BurntSushi/toml-test suite</a>. </section> <section id="specification"> <h2><a class="toc-backref" href="#specification" role="doc-backlink">Specification</a></h2> A new module <code class="docutils literal notranslate">tomllib</code> will be added to the Python standard library, exposing the following public functions: <div class="highlight-default notranslate"><div class="highlight"><pre>def load( fp: SupportsRead[bytes], /, *, parse_float: Callable[[str], Any] = ..., ) -> dict[str, Any]: ... def loads( s: str, /, *, parse_float: Callable[[str], Any] = ..., ) -> dict[str, Any]: ... </pre></div> </div> <code class="docutils literal notranslate">tomllib.load</code> deserializes a binary file-like object containing a TOML document to a Python <code class="docutils literal notranslate">dict</code>. The <code class="docutils literal notranslate">fp</code> argument must have a <code class="docutils literal notranslate">read()</code> method with the same API as <code class="docutils literal notranslate">io.RawIOBase.read()</code>. <code class="docutils literal notranslate">tomllib.loads</code> deserializes a <code class="docutils literal notranslate">str</code> instance containing a TOML document to a Python <code class="docutils literal notranslate">dict</code>. The <code class="docutils literal notranslate">parse_float</code> argument is a callable object that takes as input the original string representation of a TOML float, and returns a corresponding Python object (similar to <code class="docutils literal notranslate">parse_float</code> in <code class="docutils literal notranslate">json.load</code>). For example, the user may pass a function returning a <code class="docutils literal notranslate">decimal.Decimal</code>, for use cases where exact precision is important. By default, TOML floats are parsed as instances of the Python <code class="docutils literal notranslate">float</code> type. The returned object contains only basic Python objects (<code class="docutils literal notranslate">str</code>, <code class="docutils literal notranslate">int</code>, <code class="docutils literal notranslate">bool</code>, <code class="docutils literal notranslate">float</code>, <code class="docutils literal notranslate">datetime.{datetime,date,time}</code>, <code class="docutils literal notranslate">list</code>, <code class="docutils literal notranslate">dict</code> with string keys), and the results of <code class="docutils literal notranslate">parse_float</code>. <code class="docutils literal notranslate">tomllib.TOMLDecodeError</code> is raised in the case of invalid TOML. Note that this PEP does not propose <code class="docutils literal notranslate">tomllib.dump</code> or <code class="docutils literal notranslate">tomllib.dumps</code> functions; see <a class="reference internal" href="#including-an-api-for-writing-toml">Including an API for writing TOML</a> for details. </section> <section id="maintenance-implications"> <h2><a class="toc-backref" href="#maintenance-implications" role="doc-backlink">Maintenance Implications</a></h2> <section id="stability-of-toml"> <h3><a class="toc-backref" href="#stability-of-toml" role="doc-backlink">Stability of TOML</a></h3> The release of TOML 1.0.0 in January 2021 indicates the TOML format should now be officially considered stable. Empirically, TOML has proven to be a stable format even prior to the release of TOML 1.0.0. From the <a class="reference external" href="https://github.com/toml-lang/toml/blob/master/CHANGELOG.md">changelog</a>, we can see that TOML has had no major changes since April 2020, and has had two releases in the past five years (2017-2021). In the event of changes to the TOML specification, we can treat minor revisions as bug fixes and update the implementation in place. In the event of major breaking changes, we should preserve support for TOML 1.x. </section> <section id="maintainability-of-proposed-implementation"> <h3><a class="toc-backref" href="#maintainability-of-proposed-implementation" role="doc-backlink">Maintainability of proposed implementation</a></h3> The proposed implementation (<code class="docutils literal notranslate">tomli</code>) is pure Python, well tested and weighs in at under 1000 lines of code. It is minimalist, offering a smaller API surface area than other TOML implementations. The author of <code class="docutils literal notranslate">tomli</code> is willing to help integrate <code class="docutils literal notranslate">tomli</code> into the standard library and help maintain it, <a class="reference external" href="https://github.com/hukkin/tomli/issues/141#issuecomment-998018972">as per this post</a>. Furthermore, Python core developer Petr Viktorin has indicated a willingness to maintain a read API, <a class="reference external" href="https://discuss.python.org/t/adopting-recommending-a-toml-parser/4068/88">as per this post</a>. Rewriting the parser in C is not deemed necessary at this time. It is rare for TOML parsing to be a bottleneck in applications, and users with higher performance needs can use a third-party library (as is already often the case with JSON, despite Python offering a standard library C-extension module). </section> <section id="toml-support-a-slippery-slope-for-other-things"> <h3><a class="toc-backref" href="#toml-support-a-slippery-slope-for-other-things" role="doc-backlink">TOML support a slippery slope for other things</a></h3> As discussed in the <a class="reference internal" href="#motivation">Motivation</a> section, TOML holds a special place in the Python ecosystem, for reading <a class="pep reference internal" href="../pep-0518/" title="PEP 518 – Specifying Minimum Build System Requirements for Python Projects">PEP 518</a> <code class="docutils literal notranslate">pyproject.toml</code> packaging and tool configuration files. This chief reason to include TOML in the standard library does not apply to other formats, such as YAML or MessagePack. In addition, the simplicity of TOML distinguishes it from other formats like YAML, which are highly complicated to construct and parse. An API for writing TOML may, however, be added in a future PEP. </section> </section> <section id="backwards-compatibility"> <h2><a class="toc-backref" href="#backwards-compatibility" role="doc-backlink">Backwards Compatibility</a></h2> This proposal has no backwards compatibility issues within the standard library, as it describes a new module. Any existing third-party module named <code class="docutils literal notranslate">tomllib</code> will break, as <code class="docutils literal notranslate">import tomllib</code> will import the standard library module. However, <code class="docutils literal notranslate">tomllib</code> is not registered on PyPI, so it is unlikely that any module with this name is widely used. Note that we avoid using the more straightforward name <code class="docutils literal notranslate">toml</code> to avoid backwards compatibility implications for users who have pinned versions of the current <code class="docutils literal notranslate">toml</code> PyPI package. For more details, see the <a class="reference internal" href="#alternative-names-for-the-module">Alternative names for the module</a> section. </section> <section id="security-implications"> <h2><a class="toc-backref" href="#security-implications" role="doc-backlink">Security Implications</a></h2> Errors in the implementation could cause potential security issues. However, the parser’s output is limited to simple data types; inability to load arbitrary classes avoids security issues common in more “powerful” formats like pickle and YAML. Also, the implementation will be in pure Python, which reduces security issues endemic to C, such as buffer overflows. </section> <section id="how-to-teach-this"> <h2><a class="toc-backref" href="#how-to-teach-this" role="doc-backlink">How to Teach This</a></h2> The API of <code class="docutils literal notranslate">tomllib</code> mimics that of other well-established file format libraries, such as <code class="docutils literal notranslate">json</code> and <code class="docutils literal notranslate">pickle</code>. The lack of a <code class="docutils literal notranslate">dump</code> function will be explained in the documentation, with a link to relevant third-party libraries (e.g. <code class="docutils literal notranslate">tomlkit</code>, <code class="docutils literal notranslate">tomli-w</code>, <code class="docutils literal notranslate">pytomlpp</code>). </section> <section id="reference-implementation"> <h2><a class="toc-backref" href="#reference-implementation" role="doc-backlink">Reference Implementation</a></h2> The proposed implementation can be found at <a class="reference external" href="https://github.com/hukkin/tomli">https://github.com/hukkin/tomli</a> </section> <section id="rejected-ideas"> <h2><a class="toc-backref" href="#rejected-ideas" role="doc-backlink">Rejected Ideas</a></h2> <section id="basing-on-another-toml-implementation"> <h3><a class="toc-backref" href="#basing-on-another-toml-implementation" role="doc-backlink">Basing on another TOML implementation</a></h3> Several potential alternative implementations exist: <ul class="simple"> <li><code class="docutils literal notranslate">tomlkit</code> is well established, actively maintained and supports TOML 1.0.0. An important difference is that <code class="docutils literal notranslate">tomlkit</code> supports style roundtripping. As a result, it has a more complex API and implementation (about 5x as much code as <code class="docutils literal notranslate">tomli</code>). Its author does not believe that <code class="docutils literal notranslate">tomlkit</code> is a good choice for the standard library.</li> <li><code class="docutils literal notranslate">toml</code> is a very widely used library. However, it is not actively maintained, does not support TOML 1.0.0 and has a number of known bugs. Its API is more complex than that of <code class="docutils literal notranslate">tomli</code>. It allows customising output style through a complicated encoder API, and some very limited and mostly unused functionality to preserve input style through an undocumented decoder API. For more details on its API differences from this PEP, refer to <a class="reference internal" href="#pep-680-appendix-a">Appendix A</a>.</li> <li><code class="docutils literal notranslate">pytomlpp</code> is a Python wrapper for the C++ project <code class="docutils literal notranslate">toml++</code>. Pure Python libraries are easier to maintain than extension modules.</li> <li><code class="docutils literal notranslate">rtoml</code> is a Python wrapper for the Rust project <code class="docutils literal notranslate">toml-rs</code> and hence has similar shortcomings to <code class="docutils literal notranslate">pytomlpp</code>. In addition, it does not support TOML 1.0.0.</li> <li>Writing an implementation from scratch. It’s unclear what we would get from this; <code class="docutils literal notranslate">tomli</code> meets our needs and the author is willing to help with its inclusion in the standard library.</li> </ul> </section> <section id="including-an-api-for-writing-toml"> <h3><a class="toc-backref" href="#including-an-api-for-writing-toml" role="doc-backlink">Including an API for writing TOML</a></h3> There are several reasons to not include an API for writing TOML. The ability to write TOML is not needed for the use cases that motivate this PEP: core Python packaging tools, and projects that need to read TOML configuration files. Use cases that involve editing an existing TOML file (as opposed to writing a brand new one) are better served by a style preserving library. TOML is intended as a human-readable and -editable configuration format, so it’s important to preserve comments, formatting and other markup. This requires a parser whose output includes style-related metadata, making it impractical to output plain Python types like <code class="docutils literal notranslate">str</code> and <code class="docutils literal notranslate">dict</code>. Furthermore, it substantially complicates the design of the API. Even without considering style preservation, there are too many degrees of freedom in how to design a write API. For example, what default style (indentation, vertical and horizontal spacing, quotes, etc) should the library use for the output, and how much control should users be given over it? How should the library handle input and output validation? Should it support serialization of custom types, and if so, how? While there are reasonable options for resolving these issues, the nature of the standard library is such that we only get “one chance to get it right”. Currently, no CPython core developers have expressed willingness to maintain a write API, or sponsor a PEP that includes one. Since it is hard to change or remove something in the standard library, it is safer to err on the side of exclusion for now, and potentially revisit this later. Therefore, writing TOML is left to third-party libraries. If a good API and relevant use cases for it are found later, write support can be added in a future PEP. </section> <section id="assorted-api-details"> <h3><a class="toc-backref" href="#assorted-api-details" role="doc-backlink">Assorted API details</a></h3> <section id="types-accepted-as-the-first-argument-of-tomllib-load"> <h4><a class="toc-backref" href="#types-accepted-as-the-first-argument-of-tomllib-load" role="doc-backlink">Types accepted as the first argument of <code class="docutils literal notranslate">tomllib.load</code></a></h4> The <code class="docutils literal notranslate">toml</code> library on PyPI allows passing paths (and lists of path-like objects, ignoring missing files and merging the documents into a single object) to its <code class="docutils literal notranslate">load</code> function. However, allowing this here would be inconsistent with the behavior of <code class="docutils literal notranslate">json.load</code>, <code class="docutils literal notranslate">pickle.load</code> and other standard library functions. If we agree that consistency here is desirable, allowing paths is out of scope for this PEP. This can easily and explicitly be worked around in user code, or by using a third-party library. The proposed API takes a binary file, while <code class="docutils literal notranslate">toml.load</code> takes a text file and <code class="docutils literal notranslate">json.load</code> takes either. Using a binary file allows us to ensure UTF-8 is the encoding used (ensuring correct parsing on platforms with other default encodings, such as Windows), and avoid incorrectly parsing files containing single carriage returns as valid TOML due to universal newlines in text mode. </section> <section id="type-accepted-as-the-first-argument-of-tomllib-loads"> <h4><a class="toc-backref" href="#type-accepted-as-the-first-argument-of-tomllib-loads" role="doc-backlink">Type accepted as the first argument of <code class="docutils literal notranslate">tomllib.loads</code></a></h4> While <code class="docutils literal notranslate">tomllib.load</code> takes a binary file, <code class="docutils literal notranslate">tomllib.loads</code> takes a text string. This may seem inconsistent at first. Quoting the <a class="reference external" href="https://toml.io/en/v1.0.0#spec">TOML v1.0.0 specification</a>: <blockquote> <div>A TOML file must be a valid UTF-8 encoded Unicode document.</div></blockquote> <code class="docutils literal notranslate">tomllib.loads</code> does not intend to load a TOML file, but rather the document that the file stores. The most natural representation of a Unicode document in Python is <code class="docutils literal notranslate">str</code>, not <code class="docutils literal notranslate">bytes</code>. It is possible to add <code class="docutils literal notranslate">bytes</code> support in the future if needed, but we are not aware of any use cases for it. </section> </section> <section id="controlling-the-type-of-mappings-returned-by-tomllib-load-s"> <h3><a class="toc-backref" href="#controlling-the-type-of-mappings-returned-by-tomllib-load-s" role="doc-backlink">Controlling the type of mappings returned by <code class="docutils literal notranslate">tomllib.load[s]</code></a></h3> The <code class="docutils literal notranslate">toml</code> library on PyPI accepts a <code class="docutils literal notranslate">_dict</code> argument in its <code class="docutils literal notranslate">load[s]</code> functions, which works similarly to the <code class="docutils literal notranslate">object_hook</code> argument in <code class="docutils literal notranslate">json.load[s]</code>. There are several uses of <code class="docutils literal notranslate">_dict</code> found on <a class="reference external" href="https://grep.app">https://grep.app</a>; however, almost all of them are passing <code class="docutils literal notranslate">_dict=OrderedDict</code>, which should be unnecessary as of Python 3.7. We found two instances of relevant use: in one case, a custom class was passed for friendlier KeyErrors; in the other, the custom class had several additional lookup and mutation methods (e.g. to help resolve dotted keys). Such a parameter is not necessary for the core use cases outlined in the <a class="reference internal" href="#motivation">Motivation</a> section. The absence of this can be pretty easily worked around using a wrapper class, transformer function, or a third-party library. Finally, support could be added later in a backward-compatible way. </section> <section id="removing-support-for-parse-float-in-tomllib-load-s"> <h3><a class="toc-backref" href="#removing-support-for-parse-float-in-tomllib-load-s" role="doc-backlink">Removing support for <code class="docutils literal notranslate">parse_float</code> in <code class="docutils literal notranslate">tomllib.load[s]</code></a></h3> This option is not strictly necessary, since TOML floats should be implemented as “IEEE 754 binary64 values”, which is equivalent to a Python <code class="docutils literal notranslate">float</code> on most architectures. The TOML specification uses the word “SHOULD”, however, implying a recommendation that can be ignored for valid reasons. Parsing floats differently, such as to <code class="docutils literal notranslate">decimal.Decimal</code>, allows users extra precision beyond that promised by the TOML format. In the author of <code class="docutils literal notranslate">tomli</code>’s experience, this is particularly useful in scientific and financial applications. This is also useful for other cases that need greater precision, or where end-users include non-developers who may not be aware of the limits of binary64 floats. There are also niche architectures where the Python <code class="docutils literal notranslate">float</code> is not a IEEE 754 binary64 value. The <code class="docutils literal notranslate">parse_float</code> argument allows users to achieve correct TOML semantics even on such architectures. </section> <section id="alternative-names-for-the-module"> <h3><a class="toc-backref" href="#alternative-names-for-the-module" role="doc-backlink">Alternative names for the module</a></h3> Ideally, we would be able to use the <code class="docutils literal notranslate">toml</code> module name. However, the <code class="docutils literal notranslate">toml</code> package on PyPI is widely used, so there are backward compatibility concerns. Since the standard library takes precedence over third party packages, libraries and applications who current depend on the <code class="docutils literal notranslate">toml</code> package would likely break when upgrading Python versions due to the many API incompatibilities listed in <a class="reference internal" href="#pep-680-appendix-a">Appendix A</a>, even if they pin their dependency versions. To further clarify, applications with pinned dependencies are of greatest concern here. Even if we were able to obtain control of the <code class="docutils literal notranslate">toml</code> PyPI package name and repurpose it for a backport of the proposed new module, we would still break users on new Python versions that included it in the standard library, regardless of whether they have pinned an older version of the existing <code class="docutils literal notranslate">toml</code> package. This is unfortunate, since pinning would likely be a common response to breaking changes introduced by repurposing the <code class="docutils literal notranslate">toml</code> package as a backport (that is incompatible with today’s <code class="docutils literal notranslate">toml</code>). Finally, the <code class="docutils literal notranslate">toml</code> package on PyPI is not actively maintained, but as of yet, efforts to request that the author add other maintainers <a class="reference external" href="https://github.com/uiri/toml/issues/361">have been unsuccessful</a>, so action here would likely have to be taken without the author’s consent. Instead, this PEP proposes the name <code class="docutils literal notranslate">tomllib</code>. This mirrors <code class="docutils literal notranslate">plistlib</code> and <code class="docutils literal notranslate">xdrlib</code>, two other file format modules in the standard library, as well as other modules, such as <code class="docutils literal notranslate">pathlib</code>, <code class="docutils literal notranslate">contextlib</code> and <code class="docutils literal notranslate">graphlib</code>. Other names considered but rejected include: <ul class="simple"> <li><code class="docutils literal notranslate">tomlparser</code>. This mirrors <code class="docutils literal notranslate">configparser</code>, but is perhaps somewhat less appropriate if we include a write API in the future.</li> <li><code class="docutils literal notranslate">tomli</code>. This assumes we use <code class="docutils literal notranslate">tomli</code> as the basis for implementation.</li> <li><code class="docutils literal notranslate">toml</code> under some namespace, such as <code class="docutils literal notranslate">parser.toml</code>. However, this is awkward, especially so since existing parsing libraries like <code class="docutils literal notranslate">json</code>, <code class="docutils literal notranslate">pickle</code>, <code class="docutils literal notranslate">xml</code>, <code class="docutils literal notranslate">html</code> etc. would not be included in the namespace.</li> </ul> </section> </section> <section id="previous-discussion"> <h2><a class="toc-backref" href="#previous-discussion" role="doc-backlink">Previous Discussion</a></h2> <ul class="simple"> <li><a class="reference external" href="https://bugs.python.org/issue40059">bpo-40059: Provide a toml module in the standard library</a></li> <li><a class="reference external" href="https://mail.python.org/pipermail/python-dev/2019-May/157405.html">[Python-Dev] Adding a toml module to the standard lib?</a></li> <li><a class="reference external" href="https://mail.python.org/archives/list/python-ideas@python.org/thread/IWJ3I32A4TY6CIVQ6ONPEBPWP4TOV2V7/">[Python-Ideas] Python standard library TOML module</a></li> <li><a class="reference external" href="https://discuss.python.org/t/adopting-recommending-a-toml-parser/4068">[Packaging] Adopting/recommending a toml parser?</a></li> <li><a class="reference external" href="https://github.com/hukkin/tomli/issues/141">hukkin/tomli#141: Please consider pushing tomli into the stdlib</a></li> </ul> </section> <section id="appendix-a-differences-between-proposed-api-and-toml"> <h2><a class="toc-backref" href="#appendix-a-differences-between-proposed-api-and-toml" role="doc-backlink">Appendix A: Differences between proposed API and <code class="docutils literal notranslate">toml</code></a></h2> This appendix covers the differences between the API proposed in this PEP and that of the third-party package <code class="docutils literal notranslate">toml</code>. These differences are relevant to understanding the amount of breakage we could expect if we used the <code class="docutils literal notranslate">toml</code> name for the standard library module, as well as to better understand the design space. Note that this list might not be exhaustive. <ol class="arabic"> <li>No proposed inclusion of a write API (no <code class="docutils literal notranslate">toml.dump[s]</code>)This PEP currently proposes not including a write API; that is, there will be no equivalent of <code class="docutils literal notranslate">toml.dump</code> or <code class="docutils literal notranslate">toml.dumps</code>, as discussed at <a class="reference internal" href="#including-an-api-for-writing-toml">Including an API for writing TOML</a>. If we included a write API, it would be relatively straightforward to convert most code that uses <code class="docutils literal notranslate">toml</code> to the new standard library module (acknowledging that this is very different from a compatible API, as it would still require code changes). A significant fraction of <code class="docutils literal notranslate">toml</code> users rely on this, based on comparing <a class="reference external" href="https://grep.app/search?q=toml.load&filter[lang][0]=Python">occurrences of “toml.load”</a> to <a class="reference external" href="https://grep.app/search?q=toml.dump&filter[lang][0]=Python">occurrences of “toml.dump”</a>. </li> <li>Different first argument of <code class="docutils literal notranslate">toml.load</code><code class="docutils literal notranslate">toml.load</code> has the following signature: <div class="highlight-default notranslate"><div class="highlight"><pre>def load( f: Union[SupportsRead[str], str, bytes, list[PathLike | str | bytes]], _dict: Type[MutableMapping[str, Any]] = ..., decoder: TomlDecoder = ..., ) -> MutableMapping[str, Any]: ... </pre></div> </div> This is quite different from the first argument proposed in this PEP: <code class="docutils literal notranslate">SupportsRead[bytes]</code>. Recapping the reasons for this, previously mentioned at <a class="reference internal" href="#types-accepted-as-the-first-argument-of-tomllib-load">Types accepted as the first argument of tomllib.load</a>: <ul class="simple"> <li>Allowing paths (and even lists of paths) as arguments is inconsistent with other similar functions in the standard library.</li> <li>Using <code class="docutils literal notranslate">SupportsRead[bytes]</code> allows us to ensure UTF-8 is the encoding used, and avoid incorrectly parsing single carriage returns as valid TOML.</li> </ul> A significant fraction of <code class="docutils literal notranslate">toml</code> users rely on this, based on manual inspection of <a class="reference external" href="https://grep.app/search?q=toml.load&filter[lang][0]=Python">occurrences of “toml.load”</a>. </li> <li>Errors<code class="docutils literal notranslate">toml</code> raises <code class="docutils literal notranslate">TomlDecodeError</code>, vs. the proposed <a class="pep reference internal" href="../pep-0008/" title="PEP 8 – Style Guide for Python Code">PEP 8</a>-compliant <code class="docutils literal notranslate">TOMLDecodeError</code>. A significant fraction of <code class="docutils literal notranslate">toml</code> users rely on this, based on <a class="reference external" href="https://grep.app/search?q=TomlDecodeError&case=true&filter[lang][0]=Python">occurrences of “TomlDecodeError”</a>. </li> <li><code class="docutils literal notranslate">toml.load[s]</code> accepts a <code class="docutils literal notranslate">_dict</code> argumentDiscussed at <a class="reference internal" href="#controlling-the-type-of-mappings-returned-by-tomllib-load-s">Controlling the type of mappings returned by tomllib.load[s]</a>. As mentioned there, almost all usage consists of <code class="docutils literal notranslate">_dict=OrderedDict</code>, which is not necessary in Python 3.7 and later. </li> <li><code class="docutils literal notranslate">toml.load[s]</code> support an undocumented <code class="docutils literal notranslate">decoder</code> argumentIt seems the intended use case is for an implementation of comment preservation. The information recorded is not sufficient to roundtrip the TOML document preserving style, the implementation has known bugs, the feature is undocumented and we could only find one instance of its use on <a class="reference external" href="https://grep.app">https://grep.app</a>. The <a class="reference external" href="https://github.com/uiri/toml/blob/3f637dba5f68db63d4b30967fedda51c82459471/toml/decoder.pyi#L36">toml.TomlDecoder interface</a> exposed is far from simple, containing nine methods. Users are likely better served by a more complete implementation of style-preserving parsing and writing. </li> <li><code class="docutils literal notranslate">toml.dump[s]</code> support an <code class="docutils literal notranslate">encoder</code> argumentNote that we currently propose to not include a write API; however, if that were to change, these differences would likely become relevant. The <code class="docutils literal notranslate">encoder</code> argument enables two use cases: <ul class="simple"> <li>control over how custom types should be serialized, and</li> <li>control over how output should be formatted.</li> </ul> The first is reasonable; however, we could only find two instances of this on <a class="reference external" href="https://grep.app">https://grep.app</a>. One of these two used this ability to add support for dumping <code class="docutils literal notranslate">decimal.Decimal</code>, which a potential standard library implementation would support out of the box. If needed for other types, this use case could be well served by the equivalent of the <code class="docutils literal notranslate">default</code> argument in <code class="docutils literal notranslate">json.dump</code>. The second use case is enabled by allowing users to specify subclasses of <a class="reference external" href="https://github.com/uiri/toml/blob/3f637dba5f68db63d4b30967fedda51c82459471/toml/encoder.pyi#L9">toml.TomlEncoder</a> and overriding methods to specify parts of the TOML writing process. The API consists of five methods and exposes substantial implementation detail. There is some usage of the <code class="docutils literal notranslate">encoder</code> API on <a class="reference external" href="https://grep.app">https://grep.app</a>; however, it appears to account for a tiny fraction of the overall usage of <code class="docutils literal notranslate">toml</code>. </li> <li>Timezones<code class="docutils literal notranslate">toml</code> uses and exposes custom <code class="docutils literal notranslate">toml.tz.TomlTz</code> timezone objects. The proposed implementation uses <code class="docutils literal notranslate">datetime.timezone</code> objects from the standard library. </li> </ol> </section> <section id="copyright"> <h2><a class="toc-backref" href="#copyright" role="doc-backlink">Copyright</a></h2> This document is placed in the public domain or under the CC0-1.0-Universal license, whichever is more permissive. </section> </section> <hr class="docutils" /> Source: <a class="reference external" href="https://github.com/python/peps/blob/main/peps/pep-0680.rst">https://github.com/python/peps/blob/main/peps/pep-0680.rst</a> Last modified: <a class="reference external" href="https://github.com/python/peps/commits/main/peps/pep-0680.rst">2025-02-01 08:55:40 GMT</a> </article> <nav id="pep-sidebar"> <h2>Contents</h2> <ul> <li><a class="reference internal" href="#abstract">Abstract</a></li> <li><a class="reference internal" href="#motivation">Motivation</a></li> <li><a class="reference internal" href="#rationale">Rationale</a></li> <li><a class="reference internal" href="#specification">Specification</a></li> <li><a class="reference internal" href="#maintenance-implications">Maintenance Implications</a><ul> <li><a class="reference internal" href="#stability-of-toml">Stability of TOML</a></li> <li><a class="reference internal" href="#maintainability-of-proposed-implementation">Maintainability of proposed implementation</a></li> <li><a class="reference internal" href="#toml-support-a-slippery-slope-for-other-things">TOML support a slippery slope for other things</a></li> </ul> </li> <li><a class="reference internal" href="#backwards-compatibility">Backwards Compatibility</a></li> <li><a class="reference internal" href="#security-implications">Security Implications</a></li> <li><a class="reference internal" href="#how-to-teach-this">How to Teach This</a></li> <li><a class="reference internal" href="#reference-implementation">Reference Implementation</a></li> <li><a class="reference internal" href="#rejected-ideas">Rejected Ideas</a><ul> <li><a class="reference internal" href="#basing-on-another-toml-implementation">Basing on another TOML implementation</a></li> <li><a class="reference internal" href="#including-an-api-for-writing-toml">Including an API for writing TOML</a></li> <li><a class="reference internal" href="#assorted-api-details">Assorted API details</a><ul> <li><a class="reference internal" href="#types-accepted-as-the-first-argument-of-tomllib-load">Types accepted as the first argument of <code class="docutils literal notranslate">tomllib.load</code></a></li> <li><a class="reference internal" href="#type-accepted-as-the-first-argument-of-tomllib-loads">Type accepted as the first argument of <code class="docutils literal notranslate">tomllib.loads</code></a></li> </ul> </li> <li><a class="reference internal" href="#controlling-the-type-of-mappings-returned-by-tomllib-load-s">Controlling the type of mappings returned by <code class="docutils literal notranslate">tomllib.load[s]</code></a></li> <li><a class="reference internal" href="#removing-support-for-parse-float-in-tomllib-load-s">Removing support for <code class="docutils literal notranslate">parse_float</code> in <code class="docutils literal notranslate">tomllib.load[s]</code></a></li> <li><a class="reference internal" href="#alternative-names-for-the-module">Alternative names for the module</a></li> </ul> </li> <li><a class="reference internal" href="#previous-discussion">Previous Discussion</a></li> <li><a class="reference internal" href="#appendix-a-differences-between-proposed-api-and-toml">Appendix A: Differences between proposed API and <code class="docutils literal notranslate">toml</code></a></li> <li><a class="reference internal" href="#copyright">Copyright</a></li> </ul> <a id="source" href="https://github.com/python/peps/blob/main/peps/pep-0680.rst">Page Source (GitHub)</a> </nav> </section> <script src="../_static/colour_scheme.js"></script> <script src="../_static/wrap_tables.js"></script> <script src="../_static/sticky_banner.js"></script> </body> </html>