CINXE.COM

<!DOCTYPE html> <html lang="en"> <head> <meta charset="utf-8"> <meta name="viewport" content="width=device-width, initial-scale=1.0"> <meta name="color-scheme" content="light dark"> <title>PEP 3112 – Bytes literals in Python 3000 | peps.python.org</title> <link rel="shortcut icon" href="../_static/py.png"> <link rel="canonical" href="https://peps.python.org/pep-3112/"> <link rel="stylesheet" href="../_static/style.css" type="text/css"> <link rel="stylesheet" href="../_static/mq.css" type="text/css"> <link rel="stylesheet" href="../_static/pygments.css" type="text/css" media="(prefers-color-scheme: light)" id="pyg-light"> <link rel="stylesheet" href="../_static/pygments_dark.css" type="text/css" media="(prefers-color-scheme: dark)" id="pyg-dark"> <link rel="alternate" type="application/rss+xml" title="Latest PEPs" href="https://peps.python.org/peps.rss"> <meta property="og:title" content='PEP 3112 – Bytes literals in Python 3000 | peps.python.org'> <meta property="og:description" content="This PEP proposes a literal syntax for the bytes objects introduced in PEP 358. The purpose is to provide a convenient way to spell ASCII strings and arbitrary binary data."> <meta property="og:type" content="website"> <meta property="og:url" content="https://peps.python.org/pep-3112/"> <meta property="og:site_name" content="Python Enhancement Proposals (PEPs)"> <meta property="og:image" content="https://peps.python.org/_static/og-image.png"> <meta property="og:image:alt" content="Python PEPs"> <meta property="og:image:width" content="200"> <meta property="og:image:height" content="200"> <meta name="description" content="This PEP proposes a literal syntax for the bytes objects introduced in PEP 358. The purpose is to provide a convenient way to spell ASCII strings and arbitrary binary data."> <meta name="theme-color" content="#3776ab"> </head> <body> <svg xmlns="http://www.w3.org/2000/svg" style="display: none;"> <symbol id="svg-sun-half" viewBox="0 0 24 24" pointer-events="all"> <title>Following system colour scheme</title> <svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"> <circle cx="12" cy="12" r="9"></circle> <path d="M12 3v18m0-12l4.65-4.65M12 14.3l7.37-7.37M12 19.6l8.85-8.85"></path> </svg> </symbol> <symbol id="svg-moon" viewBox="0 0 24 24" pointer-events="all"> <title>Selected dark colour scheme</title> <svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"> <path stroke="none" d="M0 0h24v24H0z" fill="none"></path> <path d="M12 3c.132 0 .263 0 .393 0a7.5 7.5 0 0 0 7.92 12.446a9 9 0 1 1 -8.313 -12.454z"></path> </svg> </symbol> <symbol id="svg-sun" viewBox="0 0 24 24" pointer-events="all"> <title>Selected light colour scheme</title> <svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"> <circle cx="12" cy="12" r="5"></circle> <line x1="12" y1="1" x2="12" y2="3"></line> <line x1="12" y1="21" x2="12" y2="23"></line> <line x1="4.22" y1="4.22" x2="5.64" y2="5.64"></line> <line x1="18.36" y1="18.36" x2="19.78" y2="19.78"></line> <line x1="1" y1="12" x2="3" y2="12"></line> <line x1="21" y1="12" x2="23" y2="12"></line> <line x1="4.22" y1="19.78" x2="5.64" y2="18.36"></line> <line x1="18.36" y1="5.64" x2="19.78" y2="4.22"></line> </svg> </symbol> </svg> <script> document.documentElement.dataset.colour_scheme = localStorage.getItem("colour_scheme") || "auto" </script> <section id="pep-page-section"> <header> <h1>Python Enhancement Proposals</h1> <ul class="breadcrumbs"> <li><a href="https://www.python.org/" title="The Python Programming Language">Python</a> » </li> <li><a href="../pep-0000/">PEP Index</a> » </li> <li>PEP 3112</li> </ul> <button id="colour-scheme-cycler" onClick="setColourScheme(nextColourScheme())"> <svg aria-hidden="true" class="colour-scheme-icon-when-auto"><use href="#svg-sun-half"></use></svg> <svg aria-hidden="true" class="colour-scheme-icon-when-dark"><use href="#svg-moon"></use></svg> <svg aria-hidden="true" class="colour-scheme-icon-when-light"><use href="#svg-sun"></use></svg> Toggle light / dark / auto colour theme </button> </header> <article> <section id="pep-content"> <h1 class="page-title">PEP 3112 – Bytes literals in Python 3000</h1> <dl class="rfc2822 field-list simple"> <dt class="field-odd">Author:</dt> <dd class="field-odd">Jason Orendorff <jason.orendorff at gmail.com></dd> <dt class="field-even">Status:</dt> <dd class="field-even"><abbr title="Accepted and implementation complete, or no longer active">Final</abbr></dd> <dt class="field-odd">Type:</dt> <dd class="field-odd"><abbr title="Normative PEP with a new feature for Python, implementation change for CPython or interoperability standard for the ecosystem">Standards Track</abbr></dd> <dt class="field-even">Requires:</dt> <dd class="field-even"><a class="reference external" href="../pep-0358/">358</a></dd> <dt class="field-odd">Created:</dt> <dd class="field-odd">23-Feb-2007</dd> <dt class="field-even">Python-Version:</dt> <dd class="field-even">3.0</dd> <dt class="field-odd">Post-History:</dt> <dd class="field-odd">23-Feb-2007</dd> </dl> <hr class="docutils" /> <section id="contents"> <details><summary>Table of Contents</summary><ul class="simple"> <li><a class="reference internal" href="#abstract">Abstract</a></li> <li><a class="reference internal" href="#motivation">Motivation</a></li> <li><a class="reference internal" href="#grammar-changes">Grammar Changes</a></li> <li><a class="reference internal" href="#semantics">Semantics</a></li> <li><a class="reference internal" href="#rationale">Rationale</a></li> <li><a class="reference internal" href="#reference-implementation">Reference Implementation</a></li> <li><a class="reference internal" href="#references">References</a></li> <li><a class="reference internal" href="#copyright">Copyright</a></li> </ul> </details></section> <section id="abstract"> <h2><a class="toc-backref" href="#abstract" role="doc-backlink">Abstract</a></h2> This PEP proposes a literal syntax for the <code class="docutils literal notranslate">bytes</code> objects introduced in <a class="pep reference internal" href="../pep-0358/" title="PEP 358 – The “bytes” Object">PEP 358</a>. The purpose is to provide a convenient way to spell ASCII strings and arbitrary binary data. </section> <section id="motivation"> <h2><a class="toc-backref" href="#motivation" role="doc-backlink">Motivation</a></h2> Existing spellings of an ASCII string in Python 3000 include: <div class="highlight-default notranslate"><div class="highlight"><pre>bytes('Hello world', 'ascii') 'Hello world'.encode('ascii') </pre></div> </div> The proposed syntax is: <div class="highlight-default notranslate"><div class="highlight"><pre>b'Hello world' </pre></div> </div> Existing spellings of an 8-bit binary sequence in Python 3000 include: <div class="highlight-default notranslate"><div class="highlight"><pre>bytes([0x7f, 0x45, 0x4c, 0x46, 0x01, 0x01, 0x01, 0x00]) bytes('\x7fELF\x01\x01\x01\0', 'latin-1') '7f454c4601010100'.decode('hex') </pre></div> </div> The proposed syntax is: <div class="highlight-default notranslate"><div class="highlight"><pre>b'\x7f\x45\x4c\x46\x01\x01\x01\x00' b'\x7fELF\x01\x01\x01\0' </pre></div> </div> In both cases, the advantages of the new syntax are brevity, some small efficiency gain, and the detection of encoding errors at compile time rather than at runtime. The brevity benefit is especially felt when using the string-like methods of bytes objects: <div class="highlight-default notranslate"><div class="highlight"><pre>lines = bdata.split(bytes('\n', 'ascii')) # existing syntax lines = bdata.split(b'\n') # proposed syntax </pre></div> </div> And when converting code from Python 2.x to Python 3000: <div class="highlight-default notranslate"><div class="highlight"><pre>sok.send('EXIT\r\n') # Python 2.x sok.send('EXIT\r\n'.encode('ascii')) # Python 3000 existing sok.send(b'EXIT\r\n') # proposed </pre></div> </div> </section> <section id="grammar-changes"> <h2><a class="toc-backref" href="#grammar-changes" role="doc-backlink">Grammar Changes</a></h2> The proposed syntax is an extension of the existing string syntax <a class="footnote-reference brackets" href="#stringliterals" id="id1">[1]</a>. The new syntax for strings, including the new bytes literal, is: <div class="highlight-default notranslate"><div class="highlight"><pre>stringliteral: [stringprefix] (shortstring | longstring) stringprefix: "b" | "r" | "br" | "B" | "R" | "BR" | "Br" | "bR" shortstring: "'" shortstringitem* "'" | '"' shortstringitem* '"' longstring: "'''" longstringitem* "'''" | '"""' longstringitem* '"""' shortstringitem: shortstringchar | escapeseq longstringitem: longstringchar | escapeseq shortstringchar: <any source character except "\" or newline or the quote> longstringchar: <any source character except "\"> escapeseq: "\" NL | "\\" | "\'" | '\"' | "\a" | "\b" | "\f" | "\n" | "\r" | "\t" | "\v" | "\ooo" | "\xhh" | "\uxxxx" | "\Uxxxxxxxx" | "\N{name}" </pre></div> </div> The following additional restrictions apply only to bytes literals (<code class="docutils literal notranslate">stringliteral</code> tokens with <code class="docutils literal notranslate">b</code> or <code class="docutils literal notranslate">B</code> in the <code class="docutils literal notranslate">stringprefix</code>): <ul class="simple"> <li>Each <code class="docutils literal notranslate">shortstringchar</code> or <code class="docutils literal notranslate">longstringchar</code> must be a character between 1 and 127 inclusive, regardless of any encoding declaration <a class="footnote-reference brackets" href="#encodings" id="id2">[2]</a> in the source file.</li> <li>The Unicode-specific escape sequences <code class="docutils literal notranslate">\u</code>xxxx, <code class="docutils literal notranslate">\U</code>xxxxxxxx, and <code class="docutils literal notranslate">\N{</code>name<code class="docutils literal notranslate">}</code> are unrecognized in Python 2.x and forbidden in Python 3000.</li> </ul> Adjacent bytes literals are subject to the same concatenation rules as adjacent string literals <a class="footnote-reference brackets" href="#concat" id="id3">[3]</a>. A bytes literal adjacent to a string literal is an error. </section> <section id="semantics"> <h2><a class="toc-backref" href="#semantics" role="doc-backlink">Semantics</a></h2> Each evaluation of a bytes literal produces a new <code class="docutils literal notranslate">bytes</code> object. The bytes in the new object are the bytes represented by the <code class="docutils literal notranslate">shortstringitem</code> or <code class="docutils literal notranslate">longstringitem</code> parts of the literal, in the same order. </section> <section id="rationale"> <h2><a class="toc-backref" href="#rationale" role="doc-backlink">Rationale</a></h2> The proposed syntax provides a cleaner migration path from Python 2.x to Python 3000 for most code involving 8-bit strings. Preserving the old 8-bit meaning of a string literal is usually as simple as adding a <code class="docutils literal notranslate">b</code> prefix. The one exception is Python 2.x strings containing bytes >127, which must be rewritten using escape sequences. Transcoding a source file from one encoding to another, and fixing up the encoding declaration, should preserve the meaning of the program. Python 2.x non-Unicode strings violate this principle; Python 3000 bytes literals shouldn’t. A string literal with a <code class="docutils literal notranslate">b</code> in the prefix is always a syntax error in Python 2.5, so this syntax can be introduced in Python 2.6, along with the <code class="docutils literal notranslate">bytes</code> type. A bytes literal produces a new object each time it is evaluated, like list displays and unlike string literals. This is necessary because bytes literals, like lists and unlike strings, are mutable <a class="footnote-reference brackets" href="#eachnew" id="id4">[4]</a>. </section> <section id="reference-implementation"> <h2><a class="toc-backref" href="#reference-implementation" role="doc-backlink">Reference Implementation</a></h2> Thomas Wouters has checked an implementation into the Py3K branch, r53872. </section> <section id="references"> <h2><a class="toc-backref" href="#references" role="doc-backlink">References</a></h2> <aside class="footnote-list brackets"> <aside class="footnote brackets" id="stringliterals" role="doc-footnote"> <dt class="label" id="stringliterals">[<a href="#id1">1</a>]</dt> <dd><a class="reference external" href="http://docs.python.org/reference/lexical_analysis.html#string-literals">http://docs.python.org/reference/lexical_analysis.html#string-literals</a></aside> <aside class="footnote brackets" id="encodings" role="doc-footnote"> <dt class="label" id="encodings">[<a href="#id2">2</a>]</dt> <dd><a class="reference external" href="http://docs.python.org/reference/lexical_analysis.html#encoding-declarations">http://docs.python.org/reference/lexical_analysis.html#encoding-declarations</a></aside> <aside class="footnote brackets" id="concat" role="doc-footnote"> <dt class="label" id="concat">[<a href="#id3">3</a>]</dt> <dd><a class="reference external" href="http://docs.python.org/reference/lexical_analysis.html#string-literal-concatenation">http://docs.python.org/reference/lexical_analysis.html#string-literal-concatenation</a></aside> <aside class="footnote brackets" id="eachnew" role="doc-footnote"> <dt class="label" id="eachnew">[<a href="#id4">4</a>]</dt> <dd><a class="reference external" href="https://mail.python.org/pipermail/python-3000/2007-February/005779.html">https://mail.python.org/pipermail/python-3000/2007-February/005779.html</a></aside> </aside> </section> <section id="copyright"> <h2><a class="toc-backref" href="#copyright" role="doc-backlink">Copyright</a></h2> This document has been placed in the public domain. </section> </section> <hr class="docutils" /> Source: <a class="reference external" href="https://github.com/python/peps/blob/main/peps/pep-3112.rst">https://github.com/python/peps/blob/main/peps/pep-3112.rst</a> Last modified: <a class="reference external" href="https://github.com/python/peps/commits/main/peps/pep-3112.rst">2023-09-09 17:39:29 GMT</a> </article> <nav id="pep-sidebar"> <h2>Contents</h2> <ul> <li><a class="reference internal" href="#abstract">Abstract</a></li> <li><a class="reference internal" href="#motivation">Motivation</a></li> <li><a class="reference internal" href="#grammar-changes">Grammar Changes</a></li> <li><a class="reference internal" href="#semantics">Semantics</a></li> <li><a class="reference internal" href="#rationale">Rationale</a></li> <li><a class="reference internal" href="#reference-implementation">Reference Implementation</a></li> <li><a class="reference internal" href="#references">References</a></li> <li><a class="reference internal" href="#copyright">Copyright</a></li> </ul> <a id="source" href="https://github.com/python/peps/blob/main/peps/pep-3112.rst">Page Source (GitHub)</a> </nav> </section> <script src="../_static/colour_scheme.js"></script> <script src="../_static/wrap_tables.js"></script> <script src="../_static/sticky_banner.js"></script> </body> </html>