CINXE.COM

<!DOCTYPE html> <html lang="en"> <head> <meta charset="utf-8"> <meta name="viewport" content="width=device-width, initial-scale=1.0"> <meta name="color-scheme" content="light dark"> <title>PEP 358 – The “bytes” Object | peps.python.org</title> <link rel="shortcut icon" href="../_static/py.png"> <link rel="canonical" href="https://peps.python.org/pep-0358/"> <link rel="stylesheet" href="../_static/style.css" type="text/css"> <link rel="stylesheet" href="../_static/mq.css" type="text/css"> <link rel="stylesheet" href="../_static/pygments.css" type="text/css" media="(prefers-color-scheme: light)" id="pyg-light"> <link rel="stylesheet" href="../_static/pygments_dark.css" type="text/css" media="(prefers-color-scheme: dark)" id="pyg-dark"> <link rel="alternate" type="application/rss+xml" title="Latest PEPs" href="https://peps.python.org/peps.rss"> <meta property="og:title" content='PEP 358 – The “bytes” Object | peps.python.org'> <meta property="og:description" content="This PEP outlines the introduction of a raw bytes sequence type. Adding the bytes type is one step in the transition to Unicode-based str objects which will be introduced in Python 3.0."> <meta property="og:type" content="website"> <meta property="og:url" content="https://peps.python.org/pep-0358/"> <meta property="og:site_name" content="Python Enhancement Proposals (PEPs)"> <meta property="og:image" content="https://peps.python.org/_static/og-image.png"> <meta property="og:image:alt" content="Python PEPs"> <meta property="og:image:width" content="200"> <meta property="og:image:height" content="200"> <meta name="description" content="This PEP outlines the introduction of a raw bytes sequence type. Adding the bytes type is one step in the transition to Unicode-based str objects which will be introduced in Python 3.0."> <meta name="theme-color" content="#3776ab"> </head> <body> <svg xmlns="http://www.w3.org/2000/svg" style="display: none;"> <symbol id="svg-sun-half" viewBox="0 0 24 24" pointer-events="all"> <title>Following system colour scheme</title> <svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"> <circle cx="12" cy="12" r="9"></circle> <path d="M12 3v18m0-12l4.65-4.65M12 14.3l7.37-7.37M12 19.6l8.85-8.85"></path> </svg> </symbol> <symbol id="svg-moon" viewBox="0 0 24 24" pointer-events="all"> <title>Selected dark colour scheme</title> <svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"> <path stroke="none" d="M0 0h24v24H0z" fill="none"></path> <path d="M12 3c.132 0 .263 0 .393 0a7.5 7.5 0 0 0 7.92 12.446a9 9 0 1 1 -8.313 -12.454z"></path> </svg> </symbol> <symbol id="svg-sun" viewBox="0 0 24 24" pointer-events="all"> <title>Selected light colour scheme</title> <svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"> <circle cx="12" cy="12" r="5"></circle> <line x1="12" y1="1" x2="12" y2="3"></line> <line x1="12" y1="21" x2="12" y2="23"></line> <line x1="4.22" y1="4.22" x2="5.64" y2="5.64"></line> <line x1="18.36" y1="18.36" x2="19.78" y2="19.78"></line> <line x1="1" y1="12" x2="3" y2="12"></line> <line x1="21" y1="12" x2="23" y2="12"></line> <line x1="4.22" y1="19.78" x2="5.64" y2="18.36"></line> <line x1="18.36" y1="5.64" x2="19.78" y2="4.22"></line> </svg> </symbol> </svg> <script> document.documentElement.dataset.colour_scheme = localStorage.getItem("colour_scheme") || "auto" </script> <section id="pep-page-section"> <header> <h1>Python Enhancement Proposals</h1> <ul class="breadcrumbs"> <li><a href="https://www.python.org/" title="The Python Programming Language">Python</a> » </li> <li><a href="../pep-0000/">PEP Index</a> » </li> <li>PEP 358</li> </ul> <button id="colour-scheme-cycler" onClick="setColourScheme(nextColourScheme())"> <svg aria-hidden="true" class="colour-scheme-icon-when-auto"><use href="#svg-sun-half"></use></svg> <svg aria-hidden="true" class="colour-scheme-icon-when-dark"><use href="#svg-moon"></use></svg> <svg aria-hidden="true" class="colour-scheme-icon-when-light"><use href="#svg-sun"></use></svg> Toggle light / dark / auto colour theme </button> </header> <article> <section id="pep-content"> <h1 class="page-title">PEP 358 – The “bytes” Object</h1> <dl class="rfc2822 field-list simple"> <dt class="field-odd">Author:</dt> <dd class="field-odd">Neil Schemenauer <nas at arctrix.com>, Guido van Rossum <guido at python.org></dd> <dt class="field-even">Status:</dt> <dd class="field-even"><abbr title="Accepted and implementation complete, or no longer active">Final</abbr></dd> <dt class="field-odd">Type:</dt> <dd class="field-odd"><abbr title="Normative PEP with a new feature for Python, implementation change for CPython or interoperability standard for the ecosystem">Standards Track</abbr></dd> <dt class="field-even">Created:</dt> <dd class="field-even">15-Feb-2006</dd> <dt class="field-odd">Python-Version:</dt> <dd class="field-odd">2.6, 3.0</dd> <dt class="field-even">Post-History:</dt> <dd class="field-even"></dd> </dl> <hr class="docutils" /> <section id="contents"> <details><summary>Table of Contents</summary><ul class="simple"> <li><a class="reference internal" href="#update">Update</a></li> <li><a class="reference internal" href="#abstract">Abstract</a></li> <li><a class="reference internal" href="#motivation">Motivation</a></li> <li><a class="reference internal" href="#specification">Specification</a></li> <li><a class="reference internal" href="#out-of-scope-issues">Out of Scope Issues</a></li> <li><a class="reference internal" href="#open-issues">Open Issues</a></li> <li><a class="reference internal" href="#frequently-asked-questions">Frequently Asked Questions</a></li> <li><a class="reference internal" href="#copyright">Copyright</a></li> </ul> </details></section> <section id="update"> <h2><a class="toc-backref" href="#update" role="doc-backlink">Update</a></h2> This PEP has partially been superseded by <a class="pep reference internal" href="../pep-3137/" title="PEP 3137 – Immutable Bytes and Mutable Buffer">PEP 3137</a>. </section> <section id="abstract"> <h2><a class="toc-backref" href="#abstract" role="doc-backlink">Abstract</a></h2> This PEP outlines the introduction of a raw bytes sequence type. Adding the bytes type is one step in the transition to Unicode-based str objects which will be introduced in Python 3.0. The PEP describes how the bytes type should work in Python 2.6, as well as how it should work in Python 3.0. (Occasionally there are differences because in Python 2.6, we have two string types, str and unicode, while in Python 3.0 we will only have one string type, whose name will be str but whose semantics will be like the 2.6 unicode type.) </section> <section id="motivation"> <h2><a class="toc-backref" href="#motivation" role="doc-backlink">Motivation</a></h2> Python’s current string objects are overloaded. They serve to hold both sequences of characters and sequences of bytes. This overloading of purpose leads to confusion and bugs. In future versions of Python, string objects will be used for holding character data. The bytes object will fulfil the role of a byte container. Eventually the unicode type will be renamed to str and the old str type will be removed. </section> <section id="specification"> <h2><a class="toc-backref" href="#specification" role="doc-backlink">Specification</a></h2> A bytes object stores a mutable sequence of integers that are in the range 0 to 255. Unlike string objects, indexing a bytes object returns an integer. Assigning or comparing an object that is not an integer to an element causes a <code class="docutils literal notranslate">TypeError</code> exception. Assigning an element to a value outside the range 0 to 255 causes a <code class="docutils literal notranslate">ValueError</code> exception. The <code class="docutils literal notranslate">.__len__()</code> method of bytes returns the number of integers stored in the sequence (i.e. the number of bytes). The constructor of the bytes object has the following signature: <div class="highlight-default notranslate"><div class="highlight"><pre>bytes([initializer[, encoding]]) </pre></div> </div> If no arguments are provided then a bytes object containing zero elements is created and returned. The initializer argument can be a string (in 2.6, either str or unicode), an iterable of integers, or a single integer. The pseudo-code for the constructor (optimized for clear semantics, not for speed) is: <div class="highlight-default notranslate"><div class="highlight"><pre>def bytes(initializer=0, encoding=None): if isinstance(initializer, int): # In 2.6, int -> (int, long) initializer = [0]*initializer elif isinstance(initializer, basestring): if isinstance(initializer, unicode): # In 3.0, "if True" if encoding is None: # In 3.0, raise TypeError("explicit encoding required") encoding = sys.getdefaultencoding() initializer = initializer.encode(encoding) initializer = [ord(c) for c in initializer] else: if encoding is not None: raise TypeError("no encoding allowed for this initializer") tmp = [] for c in initializer: if not isinstance(c, int): raise TypeError("initializer must be iterable of ints") if not 0 <= c < 256: raise ValueError("initializer element out of range") tmp.append(c) initializer = tmp new = <new bytes object of length len(initializer)> for i, c in enumerate(initializer): new[i] = c return new </pre></div> </div> The <code class="docutils literal notranslate">.__repr__()</code> method returns a string that can be evaluated to generate a new bytes object containing a bytes literal: <div class="highlight-default notranslate"><div class="highlight"><pre>>>> bytes([10, 20, 30]) b'\n\x14\x1e' </pre></div> </div> The object has a <code class="docutils literal notranslate">.decode()</code> method equivalent to the <code class="docutils literal notranslate">.decode()</code> method of the str object. The object has a classmethod <code class="docutils literal notranslate">.fromhex()</code> that takes a string of characters from the set <code class="docutils literal notranslate">[0-9a-fA-F ]</code> and returns a bytes object (similar to <code class="docutils literal notranslate">binascii.unhexlify</code>). For example: <div class="highlight-default notranslate"><div class="highlight"><pre>>>> bytes.fromhex('5c5350ff') b'\\SP\xff' >>> bytes.fromhex('5c 53 50 ff') b'\\SP\xff' </pre></div> </div> The object has a <code class="docutils literal notranslate">.hex()</code> method that does the reverse conversion (similar to <code class="docutils literal notranslate">binascii.hexlify</code>): <div class="highlight-default notranslate"><div class="highlight"><pre>>> bytes([92, 83, 80, 255]).hex() '5c5350ff' </pre></div> </div> The bytes object has some methods similar to list methods, and others similar to str methods. Here is a complete list of methods, with their approximate signatures: <div class="highlight-default notranslate"><div class="highlight"><pre>.__add__(bytes) -> bytes .__contains__(int | bytes) -> bool .__delitem__(int | slice) -> None .__delslice__(int, int) -> None .__eq__(bytes) -> bool .__ge__(bytes) -> bool .__getitem__(int | slice) -> int | bytes .__getslice__(int, int) -> bytes .__gt__(bytes) -> bool .__iadd__(bytes) -> bytes .__imul__(int) -> bytes .__iter__() -> iterator .__le__(bytes) -> bool .__len__() -> int .__lt__(bytes) -> bool .__mul__(int) -> bytes .__ne__(bytes) -> bool .__reduce__(...) -> ... .__reduce_ex__(...) -> ... .__repr__() -> str .__reversed__() -> bytes .__rmul__(int) -> bytes .__setitem__(int | slice, int | iterable[int]) -> None .__setslice__(int, int, iterable[int]) -> Bote .append(int) -> None .count(int) -> int .decode(str) -> str | unicode # in 3.0, only str .endswith(bytes) -> bool .extend(iterable[int]) -> None .find(bytes) -> int .index(bytes | int) -> int .insert(int, int) -> None .join(iterable[bytes]) -> bytes .partition(bytes) -> (bytes, bytes, bytes) .pop([int]) -> int .remove(int) -> None .replace(bytes, bytes) -> bytes .rindex(bytes | int) -> int .rpartition(bytes) -> (bytes, bytes, bytes) .split(bytes) -> list[bytes] .startswith(bytes) -> bool .reverse() -> None .rfind(bytes) -> int .rindex(bytes | int) -> int .rsplit(bytes) -> list[bytes] .translate(bytes, [bytes]) -> bytes </pre></div> </div> Note the conspicuous absence of <code class="docutils literal notranslate">.isupper()</code>, <code class="docutils literal notranslate">.upper()</code>, and friends. (But see “Open Issues” below.) There is no <code class="docutils literal notranslate">.__hash__()</code> because the object is mutable. There is no use case for a <code class="docutils literal notranslate">.sort()</code> method. The bytes type also supports the buffer interface, supporting reading and writing binary (but not character) data. </section> <section id="out-of-scope-issues"> <h2><a class="toc-backref" href="#out-of-scope-issues" role="doc-backlink">Out of Scope Issues</a></h2> <ul class="simple"> <li>Python 3k will have a much different I/O subsystem. Deciding how that I/O subsystem will work and interact with the bytes object is out of the scope of this PEP. The expectation however is that binary I/O will read and write bytes, while text I/O will read strings. Since the bytes type supports the buffer interface, the existing binary I/O operations in Python 2.6 will support bytes objects.</li> <li>It has been suggested that a special method named <code class="docutils literal notranslate">.__bytes__()</code> be added to the language to allow objects to be converted into byte arrays. This decision is out of scope.</li> <li>A bytes literal of the form <code class="docutils literal notranslate">b"..."</code> is also proposed. This is the subject of <a class="pep reference internal" href="../pep-3112/" title="PEP 3112 – Bytes literals in Python 3000">PEP 3112</a>.</li> </ul> </section> <section id="open-issues"> <h2><a class="toc-backref" href="#open-issues" role="doc-backlink">Open Issues</a></h2> <ul class="simple"> <li>The <code class="docutils literal notranslate">.decode()</code> method is redundant since a bytes object <code class="docutils literal notranslate">b</code> can also be decoded by calling <code class="docutils literal notranslate">unicode(b, <encoding>)</code> (in 2.6) or <code class="docutils literal notranslate">str(b, <encoding>)</code> (in 3.0). Do we need encode/decode methods at all? In a sense the spelling using a constructor is cleaner.</li> <li>Need to specify the methods still more carefully.</li> <li>Pickling and marshalling support need to be specified.</li> <li>Should all those list methods really be implemented?</li> <li>A case could be made for supporting <code class="docutils literal notranslate">.ljust()</code>, <code class="docutils literal notranslate">.rjust()</code>, <code class="docutils literal notranslate">.center()</code> with a mandatory second argument.</li> <li>A case could be made for supporting <code class="docutils literal notranslate">.split()</code> with a mandatory argument.</li> <li>A case could even be made for supporting <code class="docutils literal notranslate">.islower()</code>, <code class="docutils literal notranslate">.isupper()</code>, <code class="docutils literal notranslate">.isspace()</code>, <code class="docutils literal notranslate">.isalpha()</code>, <code class="docutils literal notranslate">.isalnum()</code>, <code class="docutils literal notranslate">.isdigit()</code> and the corresponding conversions (<code class="docutils literal notranslate">.lower()</code> etc.), using the ASCII definitions for letters, digits and whitespace. If this is accepted, the cases for <code class="docutils literal notranslate">.ljust()</code>, <code class="docutils literal notranslate">.rjust()</code>, <code class="docutils literal notranslate">.center()</code> and <code class="docutils literal notranslate">.split()</code> become much stronger, and they should have default arguments as well, using an ASCII space or all ASCII whitespace (for <code class="docutils literal notranslate">.split()</code>).</li> </ul> </section> <section id="frequently-asked-questions"> <h2><a class="toc-backref" href="#frequently-asked-questions" role="doc-backlink">Frequently Asked Questions</a></h2> Q: Why have the optional encoding argument when the encode method of Unicode objects does the same thing? A: In the current version of Python, the encode method returns a str object and we cannot change that without breaking code. The construct <code class="docutils literal notranslate">bytes(s.encode(...))</code> is expensive because it has to copy the byte sequence multiple times. Also, Python generally provides two ways of converting an object of type A into an object of type B: ask an A instance to convert itself to a B, or ask the type B to create a new instance from an A. Depending on what A and B are, both APIs make sense; sometimes reasons of decoupling require that A can’t know about B, in which case you have to use the latter approach; sometimes B can’t know about A, in which case you have to use the former. Q: Why does bytes ignore the encoding argument if the initializer is a str? (This only applies to 2.6.) A: There is no sane meaning that the encoding can have in that case. str objects are byte arrays and they know nothing about the encoding of character data they contain. We need to assume that the programmer has provided a str object that already uses the desired encoding. If you need something other than a pure copy of the bytes then you need to first decode the string. For example: <div class="highlight-default notranslate"><div class="highlight"><pre>bytes(s.decode(encoding1), encoding2) </pre></div> </div> Q: Why not have the encoding argument default to Latin-1 (or some other encoding that covers the entire byte range) rather than ASCII? A: The system default encoding for Python is ASCII. It seems least confusing to use that default. Also, in Py3k, using Latin-1 as the default might not be what users expect. For example, they might prefer a Unicode encoding. Any default will not always work as expected. At least ASCII will complain loudly if you try to encode non-ASCII data. </section> <section id="copyright"> <h2><a class="toc-backref" href="#copyright" role="doc-backlink">Copyright</a></h2> This document has been placed in the public domain. </section> </section> <hr class="docutils" /> Source: <a class="reference external" href="https://github.com/python/peps/blob/main/peps/pep-0358.rst">https://github.com/python/peps/blob/main/peps/pep-0358.rst</a> Last modified: <a class="reference external" href="https://github.com/python/peps/commits/main/peps/pep-0358.rst">2025-02-01 08:59:27 GMT</a> </article> <nav id="pep-sidebar"> <h2>Contents</h2> <ul> <li><a class="reference internal" href="#update">Update</a></li> <li><a class="reference internal" href="#abstract">Abstract</a></li> <li><a class="reference internal" href="#motivation">Motivation</a></li> <li><a class="reference internal" href="#specification">Specification</a></li> <li><a class="reference internal" href="#out-of-scope-issues">Out of Scope Issues</a></li> <li><a class="reference internal" href="#open-issues">Open Issues</a></li> <li><a class="reference internal" href="#frequently-asked-questions">Frequently Asked Questions</a></li> <li><a class="reference internal" href="#copyright">Copyright</a></li> </ul> <a id="source" href="https://github.com/python/peps/blob/main/peps/pep-0358.rst">Page Source (GitHub)</a> </nav> </section> <script src="../_static/colour_scheme.js"></script> <script src="../_static/wrap_tables.js"></script> <script src="../_static/sticky_banner.js"></script> </body> </html>