CINXE.COM

<!DOCTYPE html> <html lang="en"> <head> <meta charset="utf-8"> <meta name="viewport" content="width=device-width, initial-scale=1.0"> <meta name="color-scheme" content="light dark"> <title>PEP 460 – Add binary interpolation and formatting | peps.python.org</title> <link rel="shortcut icon" href="../_static/py.png"> <link rel="canonical" href="https://peps.python.org/pep-0460/"> <link rel="stylesheet" href="../_static/style.css" type="text/css"> <link rel="stylesheet" href="../_static/mq.css" type="text/css"> <link rel="stylesheet" href="../_static/pygments.css" type="text/css" media="(prefers-color-scheme: light)" id="pyg-light"> <link rel="stylesheet" href="../_static/pygments_dark.css" type="text/css" media="(prefers-color-scheme: dark)" id="pyg-dark"> <link rel="alternate" type="application/rss+xml" title="Latest PEPs" href="https://peps.python.org/peps.rss"> <meta property="og:title" content='PEP 460 – Add binary interpolation and formatting | peps.python.org'> <meta property="og:description" content="This PEP proposes to add minimal formatting operations to bytes and bytearray objects. The proposed additions are:"> <meta property="og:type" content="website"> <meta property="og:url" content="https://peps.python.org/pep-0460/"> <meta property="og:site_name" content="Python Enhancement Proposals (PEPs)"> <meta property="og:image" content="https://peps.python.org/_static/og-image.png"> <meta property="og:image:alt" content="Python PEPs"> <meta property="og:image:width" content="200"> <meta property="og:image:height" content="200"> <meta name="description" content="This PEP proposes to add minimal formatting operations to bytes and bytearray objects. The proposed additions are:"> <meta name="theme-color" content="#3776ab"> </head> <body> <svg xmlns="http://www.w3.org/2000/svg" style="display: none;"> <symbol id="svg-sun-half" viewBox="0 0 24 24" pointer-events="all"> <title>Following system colour scheme</title> <svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"> <circle cx="12" cy="12" r="9"></circle> <path d="M12 3v18m0-12l4.65-4.65M12 14.3l7.37-7.37M12 19.6l8.85-8.85"></path> </svg> </symbol> <symbol id="svg-moon" viewBox="0 0 24 24" pointer-events="all"> <title>Selected dark colour scheme</title> <svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"> <path stroke="none" d="M0 0h24v24H0z" fill="none"></path> <path d="M12 3c.132 0 .263 0 .393 0a7.5 7.5 0 0 0 7.92 12.446a9 9 0 1 1 -8.313 -12.454z"></path> </svg> </symbol> <symbol id="svg-sun" viewBox="0 0 24 24" pointer-events="all"> <title>Selected light colour scheme</title> <svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"> <circle cx="12" cy="12" r="5"></circle> <line x1="12" y1="1" x2="12" y2="3"></line> <line x1="12" y1="21" x2="12" y2="23"></line> <line x1="4.22" y1="4.22" x2="5.64" y2="5.64"></line> <line x1="18.36" y1="18.36" x2="19.78" y2="19.78"></line> <line x1="1" y1="12" x2="3" y2="12"></line> <line x1="21" y1="12" x2="23" y2="12"></line> <line x1="4.22" y1="19.78" x2="5.64" y2="18.36"></line> <line x1="18.36" y1="5.64" x2="19.78" y2="4.22"></line> </svg> </symbol> </svg> <script> document.documentElement.dataset.colour_scheme = localStorage.getItem("colour_scheme") || "auto" </script> <section id="pep-page-section"> <header> <h1>Python Enhancement Proposals</h1> <ul class="breadcrumbs"> <li><a href="https://www.python.org/" title="The Python Programming Language">Python</a> » </li> <li><a href="../pep-0000/">PEP Index</a> » </li> <li>PEP 460</li> </ul> <button id="colour-scheme-cycler" onClick="setColourScheme(nextColourScheme())"> <svg aria-hidden="true" class="colour-scheme-icon-when-auto"><use href="#svg-sun-half"></use></svg> <svg aria-hidden="true" class="colour-scheme-icon-when-dark"><use href="#svg-moon"></use></svg> <svg aria-hidden="true" class="colour-scheme-icon-when-light"><use href="#svg-sun"></use></svg> <span class="visually-hidden">Toggle light / dark / auto colour theme</span> </button> </header> <article> <section id="pep-content"> <h1 class="page-title">PEP 460 – Add binary interpolation and formatting</h1> <dl class="rfc2822 field-list simple"> <dt class="field-odd">Author<span class="colon">:</span></dt> <dd class="field-odd">Antoine Pitrou <solipsis at pitrou.net></dd> <dt class="field-even">Status<span class="colon">:</span></dt> <dd class="field-even"><abbr title="Removed from consideration by sponsor or authors">Withdrawn</abbr></dd> <dt class="field-odd">Type<span class="colon">:</span></dt> <dd class="field-odd"><abbr title="Normative PEP with a new feature for Python, implementation change for CPython or interoperability standard for the ecosystem">Standards Track</abbr></dd> <dt class="field-even">Created<span class="colon">:</span></dt> <dd class="field-even">06-Jan-2014</dd> <dt class="field-odd">Python-Version<span class="colon">:</span></dt> <dd class="field-odd">3.5</dd> </dl> <hr class="docutils" /> <section id="contents"> <details><summary>Table of Contents</summary><ul class="simple"> <li><a class="reference internal" href="#abstract">Abstract</a></li> <li><a class="reference internal" href="#rationale">Rationale</a></li> <li><a class="reference internal" href="#binary-formatting-features">Binary formatting features</a><ul> <li><a class="reference internal" href="#supported-features">Supported features</a></li> <li><a class="reference internal" href="#unsupported-features">Unsupported features</a></li> </ul> </li> <li><a class="reference internal" href="#criticisms">Criticisms</a></li> <li><a class="reference internal" href="#other-proposals">Other proposals</a><ul> <li><a class="reference internal" href="#a-new-type-datatype">A new type datatype</a></li> </ul> </li> <li><a class="reference internal" href="#resolution">Resolution</a></li> <li><a class="reference internal" href="#references">References</a></li> <li><a class="reference internal" href="#copyright">Copyright</a></li> </ul> </details></section> <section id="abstract"> <h2><a class="toc-backref" href="#abstract" role="doc-backlink">Abstract</a></h2> <p>This PEP proposes to add minimal formatting operations to bytes and bytearray objects. The proposed additions are:</p> <ul class="simple"> <li><code class="docutils literal notranslate"><span class="pre">bytes</span> <span class="pre">%</span> <span class="pre">...</span></code> and <code class="docutils literal notranslate"><span class="pre">bytearray</span> <span class="pre">%</span> <span class="pre">...</span></code> for percent-formatting, similar in syntax to percent-formatting on <code class="docutils literal notranslate"><span class="pre">str</span></code> objects (accepting a single object, a tuple or a dict).</li> <li><code class="docutils literal notranslate"><span class="pre">bytes.format(...)</span></code> and <code class="docutils literal notranslate"><span class="pre">bytearray.format(...)</span></code> for a formatting similar in syntax to <code class="docutils literal notranslate"><span class="pre">str.format()</span></code> (accepting positional as well as keyword arguments).</li> <li><code class="docutils literal notranslate"><span class="pre">bytes.format_map(...)</span></code> and <code class="docutils literal notranslate"><span class="pre">bytearray.format_map(...)</span></code> for an API similar to <code class="docutils literal notranslate"><span class="pre">str.format_map(...)</span></code>, with the same formatting syntax and semantics as <code class="docutils literal notranslate"><span class="pre">bytes.format()</span></code> and <code class="docutils literal notranslate"><span class="pre">bytearray.format()</span></code>.</li> </ul> </section> <section id="rationale"> <h2><a class="toc-backref" href="#rationale" role="doc-backlink">Rationale</a></h2> <p>In Python 2, <code class="docutils literal notranslate"><span class="pre">str</span> <span class="pre">%</span> <span class="pre">args</span></code> and <code class="docutils literal notranslate"><span class="pre">str.format(args)</span></code> allow the formatting and interpolation of bytestrings. This feature has commonly been used for the assembling of protocol messages when protocols are known to use a fixed encoding.</p> <p>Python 3 generally mandates that text be stored and manipulated as unicode (i.e. <code class="docutils literal notranslate"><span class="pre">str</span></code> objects, not <code class="docutils literal notranslate"><span class="pre">bytes</span></code>). In some cases, though, it makes sense to manipulate <code class="docutils literal notranslate"><span class="pre">bytes</span></code> objects directly. Typical usage is binary network protocols, where you can want to interpolate and assemble several bytes object (some of them literals, some of them compute) to produce complete protocol messages. For example, protocols such as HTTP or SIP have headers with ASCII names and opaque “textual” values using a varying and/or sometimes ill-defined encoding. Moreover, those headers can be followed by a binary body… which can be chunked and decorated with ASCII headers and trailers!</p> <p>While there are reasonably efficient ways to accumulate binary data (such as using a <code class="docutils literal notranslate"><span class="pre">bytearray</span></code> object, the <code class="docutils literal notranslate"><span class="pre">bytes.join</span></code> method or even <code class="docutils literal notranslate"><span class="pre">io.BytesIO</span></code>), none of them leads to the kind of readable and intuitive code that is produced by a %-formatted or {}-formatted template and a formatting operation.</p> </section> <section id="binary-formatting-features"> <h2><a class="toc-backref" href="#binary-formatting-features" role="doc-backlink">Binary formatting features</a></h2> <section id="supported-features"> <h3><a class="toc-backref" href="#supported-features" role="doc-backlink">Supported features</a></h3> <p>In this proposal, percent-formatting for <code class="docutils literal notranslate"><span class="pre">bytes</span></code> and <code class="docutils literal notranslate"><span class="pre">bytearray</span></code> supports the following features:</p> <ul class="simple"> <li>Looking up formatting arguments by position as well as by name (i.e., <code class="docutils literal notranslate"><span class="pre">%s</span></code> as well as <code class="docutils literal notranslate"><span class="pre">%(name)s</span></code>).</li> <li><code class="docutils literal notranslate"><span class="pre">%s</span></code> will try to get a <code class="docutils literal notranslate"><span class="pre">Py_buffer</span></code> on the given value, and fallback on calling <code class="docutils literal notranslate"><span class="pre">__bytes__</span></code>. The resulting binary data is inserted at the given point in the string. This is expected to work with bytes, bytearray and memoryview objects (as well as a couple others such as pathlib’s path objects).</li> <li><code class="docutils literal notranslate"><span class="pre">%c</span></code> will accept an integer between 0 and 255, and insert a byte of the given value.</li> </ul> <p>Braces-formatting for <code class="docutils literal notranslate"><span class="pre">bytes</span></code> and <code class="docutils literal notranslate"><span class="pre">bytearray</span></code> supports the following features:</p> <ul class="simple"> <li>All the kinds of argument lookup supported by <code class="docutils literal notranslate"><span class="pre">str.format()</span></code> (explicit positional lookup, auto-incremented positional lookup, keyword lookup, attribute lookup, etc.)</li> <li>Insertion of binary data when no modifier or layout is specified (e.g. <code class="docutils literal notranslate"><span class="pre">{}</span></code>, <code class="docutils literal notranslate"><span class="pre">{0}</span></code>, <code class="docutils literal notranslate"><span class="pre">{name}</span></code>). This has the same semantics as <code class="docutils literal notranslate"><span class="pre">%s</span></code> for percent-formatting (see above).</li> <li>The <code class="docutils literal notranslate"><span class="pre">c</span></code> modifier will accept an integer between 0 and 255, and insert a byte of the given value (same as <code class="docutils literal notranslate"><span class="pre">%c</span></code> above).</li> </ul> </section> <section id="unsupported-features"> <h3><a class="toc-backref" href="#unsupported-features" role="doc-backlink">Unsupported features</a></h3> <p>All other features present in formatting of <code class="docutils literal notranslate"><span class="pre">str</span></code> objects (either through the percent operator or the <code class="docutils literal notranslate"><span class="pre">str.format()</span></code> method) are unsupported. Those features imply treating the recipient of the operator or method as text, which goes counter to the text / bytes separation (for example, accepting <code class="docutils literal notranslate"><span class="pre">%d</span></code> as a format code would imply that the bytes object really is an ASCII-compatible text string).</p> <p>Amongst those unsupported features are not only most type-specific format codes, but also the various layout specifiers such as padding or alignment. Besides, <code class="docutils literal notranslate"><span class="pre">str</span></code> objects are not acceptable as arguments to the formatting operations, even when using e.g. the <code class="docutils literal notranslate"><span class="pre">%s</span></code> format code.</p> <p><code class="docutils literal notranslate"><span class="pre">__format__</span></code> isn’t called.</p> </section> </section> <section id="criticisms"> <h2><a class="toc-backref" href="#criticisms" role="doc-backlink">Criticisms</a></h2> <ul class="simple"> <li>The development cost and maintenance cost.</li> <li>In 3.3 encoding to ASCII or latin-1 is as fast as memcpy (but it still creates a separate object).</li> <li>Developers will have to work around the lack of binary formatting anyway, if they want to support Python 3.4 and earlier.</li> <li>bytes.join() is consistently faster than format to join bytes strings (XXX <em>is it?</em>).</li> <li>Formatting functions could be implemented in a third party module, rather than added to builtin types.</li> </ul> </section> <section id="other-proposals"> <h2><a class="toc-backref" href="#other-proposals" role="doc-backlink">Other proposals</a></h2> <section id="a-new-type-datatype"> <h3><a class="toc-backref" href="#a-new-type-datatype" role="doc-backlink">A new type datatype</a></h3> <p>It was proposed to create a new datatype specialized for “network programming”. The authors of this PEP believe this is counter-productive. Python 3 already has several major types dedicated to manipulation of binary data: <code class="docutils literal notranslate"><span class="pre">bytes</span></code>, <code class="docutils literal notranslate"><span class="pre">bytearray</span></code>, <code class="docutils literal notranslate"><span class="pre">memoryview</span></code>, <code class="docutils literal notranslate"><span class="pre">io.BytesIO</span></code>.</p> <p>Adding yet another type would make things more confusing for users, and interoperability between libraries more painful (also potentially sub-optimal, due to the necessary conversions).</p> <p>Moreover, not one type would be needed, but two: one immutable type (to allow for hashing), and one mutable type (as efficient accumulation is often necessary when working with network messages).</p> </section> </section> <section id="resolution"> <h2><a class="toc-backref" href="#resolution" role="doc-backlink">Resolution</a></h2> <p>This PEP is made obsolete by the <a class="reference external" href="https://mail.python.org/pipermail/python-dev/2014-March/133621.html">acceptance</a> of <a class="pep reference internal" href="../pep-0461/" title="PEP 461 – Adding % formatting to bytes and bytearray">PEP 461</a>, which introduces a more extended formatting language for bytes objects in conjunction with the modulo operator.</p> </section> <section id="references"> <h2><a class="toc-backref" href="#references" role="doc-backlink">References</a></h2> <ul class="simple"> <li><a class="reference external" href="http://bugs.python.org/issue3982">Issue #3982: support .format for bytes</a></li> <li><a class="reference external" href="http://mercurial.selenic.com/">Mercurial project</a></li> <li><a class="reference external" href="http://twistedmatrix.com/trac/">Twisted project</a></li> <li><a class="reference external" href="http://docs.python.org/2/library/stdtypes.html#string-formatting">Documentation of Python 2 formatting (str % args)</a></li> <li><a class="reference external" href="http://docs.python.org/2/library/string.html#formatstrings">Documentation of Python 2 formatting (str.format)</a></li> </ul> </section> <section id="copyright"> <h2><a class="toc-backref" href="#copyright" role="doc-backlink">Copyright</a></h2> <p>This document has been placed in the public domain.</p> </section> </section> <hr class="docutils" /> <p>Source: <a class="reference external" href="https://github.com/python/peps/blob/main/peps/pep-0460.rst">https://github.com/python/peps/blob/main/peps/pep-0460.rst</a></p> <p>Last modified: <a class="reference external" href="https://github.com/python/peps/commits/main/peps/pep-0460.rst">2023-09-09 17:39:29 GMT</a></p> </article> <nav id="pep-sidebar"> <h2>Contents</h2> <ul> <li><a class="reference internal" href="#abstract">Abstract</a></li> <li><a class="reference internal" href="#rationale">Rationale</a></li> <li><a class="reference internal" href="#binary-formatting-features">Binary formatting features</a><ul> <li><a class="reference internal" href="#supported-features">Supported features</a></li> <li><a class="reference internal" href="#unsupported-features">Unsupported features</a></li> </ul> </li> <li><a class="reference internal" href="#criticisms">Criticisms</a></li> <li><a class="reference internal" href="#other-proposals">Other proposals</a><ul> <li><a class="reference internal" href="#a-new-type-datatype">A new type datatype</a></li> </ul> </li> <li><a class="reference internal" href="#resolution">Resolution</a></li> <li><a class="reference internal" href="#references">References</a></li> <li><a class="reference internal" href="#copyright">Copyright</a></li> </ul> <br> <a id="source" href="https://github.com/python/peps/blob/main/peps/pep-0460.rst">Page Source (GitHub)</a> </nav> </section> <script src="../_static/colour_scheme.js"></script> <script src="../_static/wrap_tables.js"></script> <script src="../_static/sticky_banner.js"></script> </body> </html>