CINXE.COM
PEP 688: Making the buffer protocol accessible in Python - PEPs - Discussions on Python.org
<!DOCTYPE html> <html lang="en"> <head> <meta charset="utf-8"> <title>PEP 688: Making the buffer protocol accessible in Python - PEPs - Discussions on Python.org</title> <meta name="description" content="I just posted PEP 688 to support static typing of buffer objects: Summary: Add types.Buffer for use in isinstance/issubclass checks for the buffer protocol. (Implemented through an __instancecheck__ written in C tha&hellip;"> <meta name="generator" content="Discourse 3.4.0.beta3-dev - https://github.com/discourse/discourse version 5bf5d1335680f28a8eb65c488353be9585eed08e"> <link rel="icon" type="image/png" href="https://global.discourse-cdn.com/flex016/uploads/python1/optimized/1X/9997f0605d56c4bfecd63594f52f42cdafd6b06a_2_32x32.png"> <link rel="apple-touch-icon" type="image/png" href="https://global.discourse-cdn.com/flex016/uploads/python1/optimized/1X/4c06143de7870c35963b818b15b395092a434991_2_180x180.png"> <meta name="theme-color" media="(prefers-color-scheme: light)" content="#ffffff"> <meta name="theme-color" media="(prefers-color-scheme: dark)" content="#111111"> <meta name="viewport" content="width=device-width, initial-scale=1.0, minimum-scale=1.0, user-scalable=yes, viewport-fit=cover"> <link rel="canonical" href="https://discuss.python.org/t/pep-688-making-the-buffer-protocol-accessible-in-python/15265" /> <link rel="search" type="application/opensearchdescription+xml" href="https://discuss.python.org/opensearch.xml" title="Discussions on Python.org Search"> <link href="https://sea2.discourse-cdn.com/flex016/stylesheets/color_definitions_base__2_89bb3b818d8860c6da830055d8964bc4f806db7f.css?__ws=discuss.python.org" media="all" rel="stylesheet" class="light-scheme"/><link href="https://sea2.discourse-cdn.com/flex016/stylesheets/color_definitions_dark_1_2_0ef303cf72b1435fdf680c3db5c67cce60860418.css?__ws=discuss.python.org" media="(prefers-color-scheme: dark)" rel="stylesheet" class="dark-scheme"/> <link href="https://sea2.discourse-cdn.com/flex016/stylesheets/desktop_ae2c05eb022973cfd87e169e0799a6bd34290cdc.css?__ws=discuss.python.org" media="all" rel="stylesheet" data-target="desktop" /> <link href="https://sea2.discourse-cdn.com/flex016/stylesheets/checklist_ae2c05eb022973cfd87e169e0799a6bd34290cdc.css?__ws=discuss.python.org" media="all" rel="stylesheet" data-target="checklist" /> <link href="https://sea2.discourse-cdn.com/flex016/stylesheets/discourse-adplugin_ae2c05eb022973cfd87e169e0799a6bd34290cdc.css?__ws=discuss.python.org" media="all" rel="stylesheet" data-target="discourse-adplugin" /> <link href="https://sea2.discourse-cdn.com/flex016/stylesheets/discourse-akismet_ae2c05eb022973cfd87e169e0799a6bd34290cdc.css?__ws=discuss.python.org" media="all" rel="stylesheet" data-target="discourse-akismet" /> <link href="https://sea2.discourse-cdn.com/flex016/stylesheets/discourse-cakeday_ae2c05eb022973cfd87e169e0799a6bd34290cdc.css?__ws=discuss.python.org" media="all" rel="stylesheet" data-target="discourse-cakeday" /> <link href="https://sea2.discourse-cdn.com/flex016/stylesheets/discourse-chat-integration_ae2c05eb022973cfd87e169e0799a6bd34290cdc.css?__ws=discuss.python.org" media="all" rel="stylesheet" data-target="discourse-chat-integration" /> <link href="https://sea2.discourse-cdn.com/flex016/stylesheets/discourse-details_ae2c05eb022973cfd87e169e0799a6bd34290cdc.css?__ws=discuss.python.org" media="all" rel="stylesheet" data-target="discourse-details" /> <link href="https://sea2.discourse-cdn.com/flex016/stylesheets/discourse-lazy-videos_ae2c05eb022973cfd87e169e0799a6bd34290cdc.css?__ws=discuss.python.org" media="all" rel="stylesheet" data-target="discourse-lazy-videos" /> <link href="https://sea2.discourse-cdn.com/flex016/stylesheets/discourse-local-dates_ae2c05eb022973cfd87e169e0799a6bd34290cdc.css?__ws=discuss.python.org" media="all" rel="stylesheet" data-target="discourse-local-dates" /> <link href="https://sea2.discourse-cdn.com/flex016/stylesheets/discourse-math_ae2c05eb022973cfd87e169e0799a6bd34290cdc.css?__ws=discuss.python.org" media="all" rel="stylesheet" data-target="discourse-math" /> <link href="https://sea2.discourse-cdn.com/flex016/stylesheets/discourse-narrative-bot_ae2c05eb022973cfd87e169e0799a6bd34290cdc.css?__ws=discuss.python.org" media="all" rel="stylesheet" data-target="discourse-narrative-bot" /> <link href="https://sea2.discourse-cdn.com/flex016/stylesheets/discourse-policy_ae2c05eb022973cfd87e169e0799a6bd34290cdc.css?__ws=discuss.python.org" media="all" rel="stylesheet" data-target="discourse-policy" /> <link href="https://sea2.discourse-cdn.com/flex016/stylesheets/discourse-presence_ae2c05eb022973cfd87e169e0799a6bd34290cdc.css?__ws=discuss.python.org" media="all" rel="stylesheet" data-target="discourse-presence" /> <link href="https://sea2.discourse-cdn.com/flex016/stylesheets/discourse-solved_ae2c05eb022973cfd87e169e0799a6bd34290cdc.css?__ws=discuss.python.org" media="all" rel="stylesheet" data-target="discourse-solved" /> <link href="https://sea2.discourse-cdn.com/flex016/stylesheets/discourse-templates_ae2c05eb022973cfd87e169e0799a6bd34290cdc.css?__ws=discuss.python.org" media="all" rel="stylesheet" data-target="discourse-templates" /> <link href="https://sea2.discourse-cdn.com/flex016/stylesheets/discourse-topic-voting_ae2c05eb022973cfd87e169e0799a6bd34290cdc.css?__ws=discuss.python.org" media="all" rel="stylesheet" data-target="discourse-topic-voting" /> <link href="https://sea2.discourse-cdn.com/flex016/stylesheets/discourse-user-notes_ae2c05eb022973cfd87e169e0799a6bd34290cdc.css?__ws=discuss.python.org" media="all" rel="stylesheet" data-target="discourse-user-notes" /> <link href="https://sea2.discourse-cdn.com/flex016/stylesheets/discourse-yearly-review_ae2c05eb022973cfd87e169e0799a6bd34290cdc.css?__ws=discuss.python.org" media="all" rel="stylesheet" data-target="discourse-yearly-review" /> <link href="https://sea2.discourse-cdn.com/flex016/stylesheets/footnote_ae2c05eb022973cfd87e169e0799a6bd34290cdc.css?__ws=discuss.python.org" media="all" rel="stylesheet" data-target="footnote" /> <link href="https://sea2.discourse-cdn.com/flex016/stylesheets/hosted-site_ae2c05eb022973cfd87e169e0799a6bd34290cdc.css?__ws=discuss.python.org" media="all" rel="stylesheet" data-target="hosted-site" /> <link href="https://sea2.discourse-cdn.com/flex016/stylesheets/poll_ae2c05eb022973cfd87e169e0799a6bd34290cdc.css?__ws=discuss.python.org" media="all" rel="stylesheet" data-target="poll" /> <link href="https://sea2.discourse-cdn.com/flex016/stylesheets/spoiler-alert_ae2c05eb022973cfd87e169e0799a6bd34290cdc.css?__ws=discuss.python.org" media="all" rel="stylesheet" data-target="spoiler-alert" /> <link href="https://sea2.discourse-cdn.com/flex016/stylesheets/discourse-topic-voting_desktop_ae2c05eb022973cfd87e169e0799a6bd34290cdc.css?__ws=discuss.python.org" media="all" rel="stylesheet" data-target="discourse-topic-voting_desktop" /> <link href="https://sea2.discourse-cdn.com/flex016/stylesheets/poll_desktop_ae2c05eb022973cfd87e169e0799a6bd34290cdc.css?__ws=discuss.python.org" media="all" rel="stylesheet" data-target="poll_desktop" /> <link href="https://sea2.discourse-cdn.com/flex016/stylesheets/desktop_theme_4_93427783b0a70199f2e06d65a8eb2ca5c365dbcd.css?__ws=discuss.python.org" media="all" rel="stylesheet" data-target="desktop_theme" data-theme-id="4" data-theme-name="unformatted code detector"/> <link href="https://sea2.discourse-cdn.com/flex016/stylesheets/desktop_theme_2_5372499f761ae877fe1ed2c103226d632862786d.css?__ws=discuss.python.org" media="all" rel="stylesheet" data-target="desktop_theme" data-theme-id="2" data-theme-name="light"/> <link rel="alternate nofollow" type="application/rss+xml" title="RSS feed of 'PEP 688: Making the buffer protocol accessible in Python'" href="https://discuss.python.org/t/pep-688-making-the-buffer-protocol-accessible-in-python/15265.rss" /> <meta property="og:site_name" content="Discussions on Python.org" /> <meta property="og:type" content="website" /> <meta name="twitter:card" content="summary" /> <meta name="twitter:image" content="https://global.discourse-cdn.com/flex016/uploads/python1/original/1X/f93ff97c4f381b5e8add5a0c163b4ded29f20ed7.png" /> <meta property="og:image" content="https://global.discourse-cdn.com/flex016/uploads/python1/original/1X/f93ff97c4f381b5e8add5a0c163b4ded29f20ed7.png" /> <meta property="og:url" content="https://discuss.python.org/t/pep-688-making-the-buffer-protocol-accessible-in-python/15265" /> <meta name="twitter:url" content="https://discuss.python.org/t/pep-688-making-the-buffer-protocol-accessible-in-python/15265" /> <meta property="og:title" content="PEP 688: Making the buffer protocol accessible in Python" /> <meta name="twitter:title" content="PEP 688: Making the buffer protocol accessible in Python" /> <meta property="og:description" content="I just posted PEP 688 to support static typing of buffer objects: Summary: Add types.Buffer for use in isinstance/issubclass checks for the buffer protocol. (Implemented through an __instancecheck__ written in C that checks for the bf_getbuffer slot.) Remove the static typing special case that makes bytes also mean bytearray and memoryview to static type checkers. The PEP is intended for Python 3.12; the 3.11 feature freeze is too close. I’m happy to hear any feedback, but some specific qu..." /> <meta name="twitter:description" content="I just posted PEP 688 to support static typing of buffer objects: Summary: Add types.Buffer for use in isinstance/issubclass checks for the buffer protocol. (Implemented through an __instancecheck__ written in C that checks for the bf_getbuffer slot.) Remove the static typing special case that makes bytes also mean bytearray and memoryview to static type checkers. The PEP is intended for Python 3.12; the 3.11 feature freeze is too close. I’m happy to hear any feedback, but some specific qu..." /> <meta property="og:article:section" content="PEPs" /> <meta property="og:article:section:color" content="0088CC" /> <meta name="twitter:label1" value="Reading time" /> <meta name="twitter:data1" value="8 mins 🕑" /> <meta name="twitter:label2" value="Likes" /> <meta name="twitter:data2" value="35 ❤" /> <meta property="article:published_time" content="2022-04-25T14:51:42+00:00" /> <meta property="og:ignore_canonical" content="true" /> <link rel="next" href="/t/pep-688-making-the-buffer-protocol-accessible-in-python/15265?page=2"> </head> <body class="crawler browser-update"> <header> <a href="/"> Discussions on Python.org </a> </header> <div id="main-outlet" class="wrap" role="main"> <div id="topic-title"> <h1> <a href="/t/pep-688-making-the-buffer-protocol-accessible-in-python/15265">PEP 688: Making the buffer protocol accessible in Python</a> </h1> <div class="topic-category" itemscope itemtype="http://schema.org/BreadcrumbList"> <span itemprop="itemListElement" itemscope itemtype="http://schema.org/ListItem"> <a href="/c/peps/19" class="badge-wrapper bullet" itemprop="item"> <span class='badge-category-bg' style='background-color: #0088CC'></span> <span class='badge-category clear-badge'> <span class='category-name' itemprop='name'>PEPs</span> </span> </a> <meta itemprop="position" content="1" /> </span> </div> </div> <div itemscope itemtype='http://schema.org/DiscussionForumPosting'> <meta itemprop='headline' content='PEP 688: Making the buffer protocol accessible in Python'> <link itemprop='url' href='https://discuss.python.org/t/pep-688-making-the-buffer-protocol-accessible-in-python/15265'> <meta itemprop='datePublished' content='2022-04-25T14:51:42Z'> <meta itemprop='articleSection' content='PEPs'> <meta itemprop='keywords' content=''> <div itemprop='publisher' itemscope itemtype="http://schema.org/Organization"> <meta itemprop='name' content='Python Software Foundation'> <div itemprop='logo' itemscope itemtype="http://schema.org/ImageObject"> <meta itemprop='url' content='https://global.discourse-cdn.com/flex016/uploads/python1/original/1X/c7591c98caf3b31d4d9c6f322f41ed9d80a50800.png'> </div> </div> <div id='post_1' class='topic-body crawler-post'> <div class='crawler-post-meta'> <span class="creator" itemprop="author" itemscope itemtype="http://schema.org/Person"> <a itemprop="url" href='https://discuss.python.org/u/Jelle'><span itemprop='name'>Jelle</span></a> (Jelle Zijlstra) </span> <link itemprop="mainEntityOfPage" href="https://discuss.python.org/t/pep-688-making-the-buffer-protocol-accessible-in-python/15265"> <span class="crawler-post-infos"> <time datetime='2022-04-25T14:51:42Z' class='post-time'> April 25, 2022, 2:51pm </time> <meta itemprop='dateModified' content='2022-04-25T14:51:42Z'> <span itemprop='position'>1</span> </span> </div> <div class='post' itemprop='text'> <p>I just posted PEP 688 to support static typing of buffer objects:</p> <aside class="onebox allowlistedgeneric" data-onebox-src="https://peps.python.org/pep-0688/"> <header class="source"> <img src="https://peps.python.org/pep-0688/../_static/py.png" class="site-icon" width="16" height="16"> <a href="https://peps.python.org/pep-0688/" target="_blank" rel="noopener">peps.python.org</a> </header> <article class="onebox-body"> <h3><a href="https://peps.python.org/pep-0688/" target="_blank" rel="noopener">PEP 688 – Making the buffer protocol accessible in Python | peps.python.org</a></h3> <p>Python Enhancement Proposals (PEPs)</p> </article> <div class="onebox-metadata"> </div> <div style="clear: both"></div> </aside> <p>Summary:</p> <ul> <li>Add <code>types.Buffer</code> for use in isinstance/issubclass checks for the buffer protocol. (Implemented through an <code>__instancecheck__</code> written in C that checks for the <code>bf_getbuffer</code> slot.)</li> <li>Remove the static typing special case that makes <code>bytes</code> also mean <code>bytearray</code> and <code>memoryview</code> to static type checkers.</li> <li>The PEP is intended for Python 3.12; the 3.11 feature freeze is too close.</li> </ul> <p>I’m happy to hear any feedback, but some specific questions:</p> <ul> <li>Are there use cases other than static typing for runtime checking of buffer types? Does the PEP’s design support those use cases?</li> <li>Should we attempt to support checking for writable buffers as opposed to readonly buffers? If so, how should we do that?</li> </ul> </div> <div itemprop="interactionStatistic" itemscope itemtype="http://schema.org/InteractionCounter"> <meta itemprop="interactionType" content="http://schema.org/LikeAction"/> <meta itemprop="userInteractionCount" content="1" /> <span class='post-likes'>1 Like</span> </div> <div class='crawler-linkback-list' itemscope itemtype='http://schema.org/ItemList'> <div itemprop='itemListElement' itemscope itemtype='http://schema.org/ListItem'> <a itemprop='url' href="https://discuss.python.org/t/pep-688-take-2-making-the-buffer-protocol-accessible-in-python/19756">PEP 688, take 2: Making the buffer protocol accessible in Python</a> <meta itemprop='position' content='2'> </div> </div> </div> <div id='post_2' itemprop='comment' itemscope itemtype='http://schema.org/Comment' class='topic-body crawler-post'> <div class='crawler-post-meta'> <span class="creator" itemprop="author" itemscope itemtype="http://schema.org/Person"> <a itemprop="url" href='https://discuss.python.org/u/domdfcoding'><span itemprop='name'>domdfcoding</span></a> (Dominic Davis-Foster) </span> <span class="crawler-post-infos"> <time itemprop='datePublished' datetime='2022-04-25T15:40:54Z' class='post-time'> April 25, 2022, 3:40pm </time> <meta itemprop='dateModified' content='2022-04-25T15:40:54Z'> <span itemprop='position'>2</span> </span> </div> <div class='post' itemprop='text'> <p>Presumably C code can already check for the presence of bf_getbuffer to determine whether a type is a buffer? Could typing-extensions have a small C module that provides types.Bytes exactly as in CPython?</p> </div> <div itemprop="interactionStatistic" itemscope itemtype="http://schema.org/InteractionCounter"> <meta itemprop="interactionType" content="http://schema.org/LikeAction"/> <meta itemprop="userInteractionCount" content="0" /> <span class='post-likes'></span> </div> </div> <div id='post_3' itemprop='comment' itemscope itemtype='http://schema.org/Comment' class='topic-body crawler-post'> <div class='crawler-post-meta'> <span class="creator" itemprop="author" itemscope itemtype="http://schema.org/Person"> <a itemprop="url" href='https://discuss.python.org/u/davidism'><span itemprop='name'>davidism</span></a> (David Lord) </span> <span class="crawler-post-infos"> <time itemprop='datePublished' datetime='2022-04-25T15:44:39Z' class='post-time'> April 25, 2022, 3:44pm </time> <meta itemprop='dateModified' content='2022-04-25T15:44:39Z'> <span itemprop='position'>3</span> </span> </div> <div class='post' itemprop='text'> <p>Could you explain more about removing the “bytes also means bytearray” typing? As the referenced test shows, Werkzeug makes some assumptions with it. I’m not sure what I think yet. It’s not that we <em>explicitly</em> do anything to support bytearray along with bytes, it just happens to work. It seems like that should still type check: if something is marked as accepting bytes, then passing a bytearray should still pass as long as only valid things are done to it.</p> </div> <div itemprop="interactionStatistic" itemscope itemtype="http://schema.org/InteractionCounter"> <meta itemprop="interactionType" content="http://schema.org/LikeAction"/> <meta itemprop="userInteractionCount" content="0" /> <span class='post-likes'></span> </div> </div> <div id='post_4' itemprop='comment' itemscope itemtype='http://schema.org/Comment' class='topic-body crawler-post'> <div class='crawler-post-meta'> <span class="creator" itemprop="author" itemscope itemtype="http://schema.org/Person"> <a itemprop="url" href='https://discuss.python.org/u/Jelle'><span itemprop='name'>Jelle</span></a> (Jelle Zijlstra) </span> <span class="crawler-post-infos"> <time itemprop='datePublished' datetime='2022-04-25T15:47:35Z' class='post-time'> April 25, 2022, 3:47pm </time> <meta itemprop='dateModified' content='2022-04-25T15:47:35Z'> <span itemprop='position'>4</span> </span> </div> <div class='post' itemprop='text'> <p>That’s possible. Currently typing-extensions is a single Python file. I worry that making it into a package including C code will cause problems: more risk of breakage on unusual platforms; more complicated wheel builds; harder to get it approved for users who need security approval to install third-party packages; breakage for people who vendor it.</p> </div> <div itemprop="interactionStatistic" itemscope itemtype="http://schema.org/InteractionCounter"> <meta itemprop="interactionType" content="http://schema.org/LikeAction"/> <meta itemprop="userInteractionCount" content="0" /> <span class='post-likes'></span> </div> </div> <div id='post_5' itemprop='comment' itemscope itemtype='http://schema.org/Comment' class='topic-body crawler-post'> <div class='crawler-post-meta'> <span class="creator" itemprop="author" itemscope itemtype="http://schema.org/Person"> <a itemprop="url" href='https://discuss.python.org/u/Jelle'><span itemprop='name'>Jelle</span></a> (Jelle Zijlstra) </span> <span class="crawler-post-infos"> <time itemprop='datePublished' datetime='2022-04-25T15:52:20Z' class='post-time'> April 25, 2022, 3:52pm </time> <meta itemprop='dateModified' content='2022-04-25T15:52:20Z'> <span itemprop='position'>5</span> </span> </div> <div class='post' itemprop='text'> <p>This is discussed in <a href="https://peps.python.org/pep-0688/#keep-bytearray-compatible-with-bytes" class="inline-onebox">PEP 688 – Making the buffer protocol accessible in Python | peps.python.org</a>.</p> <p>This is definitely a point I’m open to changing if there’s consensus in favor of keeping the special case, but I’d rather not:</p> <ul> <li>In general, when you put a nominal type in an annotation, that means you accept only that type. Exceptions to this rule make the type system harder to understand for users.</li> <li>bytearray is mostly compatible with bytes, but not entirely. Notably, it’s not hashable. If you have an API that accepts bytes and you decide to add caching (using the bytes object as a dict key), suddenly your API won’t work for bytearray any more.</li> </ul> </div> <div itemprop="interactionStatistic" itemscope itemtype="http://schema.org/InteractionCounter"> <meta itemprop="interactionType" content="http://schema.org/LikeAction"/> <meta itemprop="userInteractionCount" content="3" /> <span class='post-likes'>3 Likes</span> </div> </div> <div id='post_6' itemprop='comment' itemscope itemtype='http://schema.org/Comment' class='topic-body crawler-post'> <div class='crawler-post-meta'> <span class="creator" itemprop="author" itemscope itemtype="http://schema.org/Person"> <a itemprop="url" href='https://discuss.python.org/u/guido'><span itemprop='name'>guido</span></a> (Guido van Rossum) </span> <span class="crawler-post-infos"> <time itemprop='datePublished' datetime='2022-04-25T15:53:14Z' class='post-time'> April 25, 2022, 3:53pm </time> <meta itemprop='dateModified' content='2022-04-25T15:53:14Z'> <span itemprop='position'>6</span> </span> </div> <div class='post' itemprop='text'> <aside class="quote group-committers" data-username="Jelle" data-post="1" data-topic="15265"> <div class="title"> <div class="quote-controls"></div> <img loading="lazy" alt="" width="24" height="24" src="https://sea2.discourse-cdn.com/flex016/user_avatar/discuss.python.org/jelle/48/1049_2.png" class="avatar"> Jelle Zijlstra:</div> <blockquote> <p>Remove the static typing special case that makes <code>bytes</code> also mean <code>bytearray</code> and <code>memoryview</code> to static type checkers.</p> </blockquote> </aside> <p>I have to admit I haven’t had time to read the PEP yet, but I worry that this is going to cause a lot of churn and permanently add a lot of ugliness to all signatures that currently support bytes (since most of them also support bytearray and many also support memoryview).</p> </div> <div itemprop="interactionStatistic" itemscope itemtype="http://schema.org/InteractionCounter"> <meta itemprop="interactionType" content="http://schema.org/LikeAction"/> <meta itemprop="userInteractionCount" content="1" /> <span class='post-likes'>1 Like</span> </div> </div> <div id='post_7' itemprop='comment' itemscope itemtype='http://schema.org/Comment' class='topic-body crawler-post'> <div class='crawler-post-meta'> <span class="creator" itemprop="author" itemscope itemtype="http://schema.org/Person"> <a itemprop="url" href='https://discuss.python.org/u/davidism'><span itemprop='name'>davidism</span></a> (David Lord) </span> <span class="crawler-post-infos"> <time itemprop='datePublished' datetime='2022-04-25T16:00:04Z' class='post-time'> April 25, 2022, 4:00pm </time> <meta itemprop='dateModified' content='2022-04-25T16:00:04Z'> <span itemprop='position'>7</span> </span> </div> <div class='post' itemprop='text'> <p>I did read that section, but my concern is the same as Guido’s. This makes the signature more complex, which makes it harder for general users to understand.</p> <p>From my perspective it doesn’t make the types more accurate. Bytearray is duck-type compatible with bytes in the vast majority of cases where it is implicitly allowed right now.</p> </div> <div itemprop="interactionStatistic" itemscope itemtype="http://schema.org/InteractionCounter"> <meta itemprop="interactionType" content="http://schema.org/LikeAction"/> <meta itemprop="userInteractionCount" content="0" /> <span class='post-likes'></span> </div> </div> <div id='post_8' itemprop='comment' itemscope itemtype='http://schema.org/Comment' class='topic-body crawler-post'> <div class='crawler-post-meta'> <span class="creator" itemprop="author" itemscope itemtype="http://schema.org/Person"> <a itemprop="url" href='https://discuss.python.org/u/AlexWaygood'><span itemprop='name'>AlexWaygood</span></a> (Alex Waygood) </span> <span class="crawler-post-infos"> <time itemprop='datePublished' datetime='2022-04-25T16:05:42Z' class='post-time'> April 25, 2022, 4:05pm </time> <meta itemprop='dateModified' content='2022-04-25T16:05:42Z'> <span itemprop='position'>8</span> </span> </div> <div class='post' itemprop='text'> <p>This mypy issue is a great example of why this PEP is needed, in my opinion: <a href="https://github.com/python/mypy/issues/12643" class="inline-onebox" rel="noopener nofollow ugc">Surprising error involving bytes duck-type compatibility · Issue #12643 · python/mypy · GitHub</a></p> <p>It’s true that the status quo “works” in a lot of situations. But when it <em>doesn’t</em> work, it can have utterly bewildering effects for end users. In this case, mypy’s special-casing for bytes duck-typing has led it to assess that a line of code is simultaneously reachable and yet also unreachable.</p> <p>Perhaps you might argue that this specific case is an issue with mypy’s implementation, rather than an issue with the general rule of “type checkers should pretend bytearray and memoryview are compatible with bytes”. But it’s hard for me to see what the “correct” solution for mypy is here, if we want to maintain the fiction that memoryview and bytearray are subtypes of bytes.</p> </div> <div itemprop="interactionStatistic" itemscope itemtype="http://schema.org/InteractionCounter"> <meta itemprop="interactionType" content="http://schema.org/LikeAction"/> <meta itemprop="userInteractionCount" content="0" /> <span class='post-likes'></span> </div> </div> <div id='post_9' itemprop='comment' itemscope itemtype='http://schema.org/Comment' class='topic-body crawler-post'> <div class='crawler-post-meta'> <span class="creator" itemprop="author" itemscope itemtype="http://schema.org/Person"> <a itemprop="url" href='https://discuss.python.org/u/AlexWaygood'><span itemprop='name'>AlexWaygood</span></a> (Alex Waygood) </span> <span class="crawler-post-infos"> <time itemprop='datePublished' datetime='2022-04-25T16:10:04Z' class='post-time'> April 25, 2022, 4:10pm </time> <meta itemprop='dateModified' content='2022-04-25T16:10:04Z'> <span itemprop='position'>9</span> </span> </div> <div class='post' itemprop='text'> <p><a class="mention" href="/u/jelle">@Jelle</a>, in the PEP, perhaps it might be good to discuss in a bit more depth the problems that the status quo has caused over the years? There’s a few links taking you to external discussions, but not much discussion in the text itself.</p> </div> <div itemprop="interactionStatistic" itemscope itemtype="http://schema.org/InteractionCounter"> <meta itemprop="interactionType" content="http://schema.org/LikeAction"/> <meta itemprop="userInteractionCount" content="0" /> <span class='post-likes'></span> </div> </div> <div id='post_10' itemprop='comment' itemscope itemtype='http://schema.org/Comment' class='topic-body crawler-post'> <div class='crawler-post-meta'> <span class="creator" itemprop="author" itemscope itemtype="http://schema.org/Person"> <a itemprop="url" href='https://discuss.python.org/u/guido'><span itemprop='name'>guido</span></a> (Guido van Rossum) </span> <span class="crawler-post-infos"> <time itemprop='datePublished' datetime='2022-04-25T16:40:54Z' class='post-time'> April 25, 2022, 4:40pm </time> <meta itemprop='dateModified' content='2022-04-25T16:40:54Z'> <span itemprop='position'>10</span> </span> </div> <div class='post' itemprop='text'> <p>The advantage of the status quo is that it almost always works (hashing bytes isn’t that common) and makes the annotations a lot more readable. With this proposal, every</p> <pre data-code-wrap="py"><code class="lang-nohighlight">def foo(arg: bytes) -> bytes: ... </code></pre> <p>may have to become</p> <pre data-code-wrap="py"><code class="lang-nohighlight">def foo(arg: bytes|bytearray|memoryview) -> bytes|bytearray|memoryview: ... </code></pre> <p>and that’s just a lot harder to grok for a human. Plus it may be incorrect – depending on what the code does it may never return a memoryview even if the input is one.</p> <p>I worry that this is just going to be one of the many papercuts that will turn off the “no typing in my backyard” crowd while not solving an important pain point for typing users.</p> <p>I would also like to remind readers that there are plenty of other situations where types don’t capture the whole picture. The intention of typing for Python was always to be <em>useful</em>, not necessarily to be 100% correct and precise.</p> </div> <div itemprop="interactionStatistic" itemscope itemtype="http://schema.org/InteractionCounter"> <meta itemprop="interactionType" content="http://schema.org/LikeAction"/> <meta itemprop="userInteractionCount" content="2" /> <span class='post-likes'>2 Likes</span> </div> </div> <div id='post_11' itemprop='comment' itemscope itemtype='http://schema.org/Comment' class='topic-body crawler-post'> <div class='crawler-post-meta'> <span class="creator" itemprop="author" itemscope itemtype="http://schema.org/Person"> <a itemprop="url" href='https://discuss.python.org/u/pf_moore'><span itemprop='name'>pf_moore</span></a> (Paul Moore) </span> <span class="crawler-post-infos"> <time itemprop='datePublished' datetime='2022-04-25T17:19:45Z' class='post-time'> April 25, 2022, 5:19pm </time> <meta itemprop='dateModified' content='2022-04-25T17:19:45Z'> <span itemprop='position'>11</span> </span> </div> <div class='post' itemprop='text'> <aside class="quote group-committers" data-username="guido" data-post="10" data-topic="15265"> <div class="title"> <div class="quote-controls"></div> <img loading="lazy" alt="" width="24" height="24" src="https://sea2.discourse-cdn.com/flex016/user_avatar/discuss.python.org/guido/48/21_2.png" class="avatar"> Guido van Rossum:</div> <blockquote> <p>I worry that this is just going to be one of the many papercuts that will turn off the “no typing in my backyard” crowd while not solving an important pain point for typing users.</p> </blockquote> </aside> <p>It certainly feels like that to me (as someone who is at least mildly type-averse).</p> <aside class="quote group-committers" data-username="guido" data-post="10" data-topic="15265"> <div class="title"> <div class="quote-controls"></div> <img loading="lazy" alt="" width="24" height="24" src="https://sea2.discourse-cdn.com/flex016/user_avatar/discuss.python.org/guido/48/21_2.png" class="avatar"> Guido van Rossum:</div> <blockquote> <p>I would also like to remind readers that there are plenty of other situations where types don’t capture the whole picture. The intention of typing for Python was always to be <em>useful</em> , not necessarily to be 100% correct and precise.</p> </blockquote> </aside> <p>Huge +1 on this point. The more “correct and precise” typing gets, the less practical and useful it feels to me. That’s not a criticism of the people working hard on typing, it just seems to me to reflect very different needs and experiences.</p> <p>The current behaviour of <code>bytes</code> as an annotation never even occurred to me - it “just works” the way I expect it to (by which I say bytes, but actually I’m fine with anything that acts sufficiently like a bytes object - duck typing at its best!) Whereas the proposed approaches - whether <code>bytes|bytearray|memoryview</code> or <code>types.Buffer</code> - act as a stumbling block for me, because suddenly I have to think “what do I <em>really</em> mean?” and yet I can’t express the answer I want to give, which is “bytes, or stuff that works like bytes but I don’t want to get too precise or restrictive here…”</p> <p>I’m mostly in favour of having a type that expresses “exposes the buffer protocol”, if people have a need for that. But I’d rather it were thought of as a specialised thing, and “normal” usage remained <code>bytes</code> (with the existing special case that makes it “do what I mean” for casual use retained).</p> </div> <div itemprop="interactionStatistic" itemscope itemtype="http://schema.org/InteractionCounter"> <meta itemprop="interactionType" content="http://schema.org/LikeAction"/> <meta itemprop="userInteractionCount" content="2" /> <span class='post-likes'>2 Likes</span> </div> </div> <div id='post_12' itemprop='comment' itemscope itemtype='http://schema.org/Comment' class='topic-body crawler-post'> <div class='crawler-post-meta'> <span class="creator" itemprop="author" itemscope itemtype="http://schema.org/Person"> <a itemprop="url" href='https://discuss.python.org/u/Jelle'><span itemprop='name'>Jelle</span></a> (Jelle Zijlstra) </span> <span class="crawler-post-infos"> <time itemprop='datePublished' datetime='2022-04-26T01:21:48Z' class='post-time'> April 26, 2022, 1:21am </time> <meta itemprop='dateModified' content='2022-04-26T01:21:48Z'> <span itemprop='position'>12</span> </span> </div> <div class='post' itemprop='text'> <p>Thanks for the feedback, sounds like I should reconsider the bytes/bytearray promotion. I’ll do some more research in this area and report back.</p> <p>One more question though for those who like the bytes shortcut: Should it really also include memoryview? Unlike bytearray, memoryview lacks most of the interface bytes provides; it’s just a buffer that provides sequence access.</p> </div> <div itemprop="interactionStatistic" itemscope itemtype="http://schema.org/InteractionCounter"> <meta itemprop="interactionType" content="http://schema.org/LikeAction"/> <meta itemprop="userInteractionCount" content="0" /> <span class='post-likes'></span> </div> </div> <div id='post_13' itemprop='comment' itemscope itemtype='http://schema.org/Comment' class='topic-body crawler-post'> <div class='crawler-post-meta'> <span class="creator" itemprop="author" itemscope itemtype="http://schema.org/Person"> <a itemprop="url" href='https://discuss.python.org/u/CAM-Gerlach'><span itemprop='name'>CAM-Gerlach</span></a> (C.A.M. Gerlach) </span> <span class="crawler-post-infos"> <time itemprop='datePublished' datetime='2022-04-26T06:52:41Z' class='post-time'> April 26, 2022, 6:52am </time> <meta itemprop='dateModified' content='2022-04-26T06:52:41Z'> <span itemprop='position'>13</span> </span> </div> <div class='post' itemprop='text'> <aside class="quote group-committers" data-username="guido" data-post="10" data-topic="15265"> <div class="title"> <div class="quote-controls"></div> <img loading="lazy" alt="" width="24" height="24" src="https://sea2.discourse-cdn.com/flex016/user_avatar/discuss.python.org/guido/48/21_2.png" class="avatar"> Guido van Rossum:</div> <blockquote> <pre data-code-wrap="python"><code class="lang-python">def foo(arg: bytes|bytearray|memoryview) -> bytes|bytearray|memoryview: </code></pre> </blockquote> </aside> <p>Maybe I’m misunderstanding, but per the primary change in this PEP, couldn’t this instead become just</p> <pre data-code-wrap="python"><code class="lang-python">from types import Buffer def foo(arg: Buffer) -> Buffer: </code></pre> <p>unless you specifically needed those exact types. and only those types? Or is that not adequate here?</p> <p>Also, without this, how would I explicitly specify that I want specifically <code>bytes</code> and not <code>bytearray</code> or <code>memoryview</code>?</p> <p>Just one data point, but as a non-expert user with some experience using static typing for certain projects and not for others, after initially being rather averse to it I’d find this sort of implicit magic rather confusing and complicated to remember and keep track of. And I’m not sure it really changes things for those who aren’t fans of static typing.</p> <p>Or perhaps as a compromise, type checkers could be encouraged to expose an option to determine their behavior in this case, e.g. <code>--strict</code> turns this behavior off, or it requires some flag to turn on?</p> </div> <div itemprop="interactionStatistic" itemscope itemtype="http://schema.org/InteractionCounter"> <meta itemprop="interactionType" content="http://schema.org/LikeAction"/> <meta itemprop="userInteractionCount" content="1" /> <span class='post-likes'>1 Like</span> </div> </div> <div id='post_14' itemprop='comment' itemscope itemtype='http://schema.org/Comment' class='topic-body crawler-post'> <div class='crawler-post-meta'> <span class="creator" itemprop="author" itemscope itemtype="http://schema.org/Person"> <a itemprop="url" href='https://discuss.python.org/u/guido'><span itemprop='name'>guido</span></a> (Guido van Rossum) </span> <span class="crawler-post-infos"> <time itemprop='datePublished' datetime='2022-04-27T00:10:37Z' class='post-time'> April 27, 2022, 12:10am </time> <meta itemprop='dateModified' content='2022-04-27T00:10:37Z'> <span itemprop='position'>14</span> </span> </div> <div class='post' itemprop='text'> <p>I am not sure what the Python-level API is that <code>types.Buffer</code> promises. E.g. does it have a <code>.lower()</code> method? I can’t find the docs for it even. Now, as long as we’re talking arguments in stub files, that doesn’t matter, since all we need is that <code>bytes</code> and <code>bytearray</code> subclass <code>Buffer</code>. But for return types this would limit what the recepient can do, and for code using inline type annotations (i.e. in .py files) it’s also important.</p> </div> <div itemprop="interactionStatistic" itemscope itemtype="http://schema.org/InteractionCounter"> <meta itemprop="interactionType" content="http://schema.org/LikeAction"/> <meta itemprop="userInteractionCount" content="0" /> <span class='post-likes'></span> </div> </div> <div id='post_15' itemprop='comment' itemscope itemtype='http://schema.org/Comment' class='topic-body crawler-post'> <div class='crawler-post-meta'> <span class="creator" itemprop="author" itemscope itemtype="http://schema.org/Person"> <a itemprop="url" href='https://discuss.python.org/u/encukou'><span itemprop='name'>encukou</span></a> (Petr Viktorin) </span> <span class="crawler-post-infos"> <time itemprop='datePublished' datetime='2022-04-27T07:50:35Z' class='post-time'> April 27, 2022, 7:50am </time> <meta itemprop='dateModified' content='2022-04-27T07:50:35Z'> <span itemprop='position'>15</span> </span> </div> <div class='post' itemprop='text'> <aside class="quote group-committers" data-username="guido" data-post="14" data-topic="15265"> <div class="title"> <div class="quote-controls"></div> <img loading="lazy" alt="" width="24" height="24" src="https://sea2.discourse-cdn.com/flex016/user_avatar/discuss.python.org/guido/48/21_2.png" class="avatar"> Guido van Rossum:</div> <blockquote> <p>I am not sure what the Python-level API is that <code>types.Buffer</code> promises</p> </blockquote> </aside> <p>The simplest Python API is <a href="https://docs.python.org/3/library/stdtypes.html#memoryview">memoryview</a>: you can do <code>memoryview(obj)</code> if <code>obj</code> implements the buffer protocol.</p> </div> <div itemprop="interactionStatistic" itemscope itemtype="http://schema.org/InteractionCounter"> <meta itemprop="interactionType" content="http://schema.org/LikeAction"/> <meta itemprop="userInteractionCount" content="0" /> <span class='post-likes'></span> </div> </div> <div id='post_16' itemprop='comment' itemscope itemtype='http://schema.org/Comment' class='topic-body crawler-post'> <div class='crawler-post-meta'> <span class="creator" itemprop="author" itemscope itemtype="http://schema.org/Person"> <a itemprop="url" href='https://discuss.python.org/u/hauntsaninja'><span itemprop='name'>hauntsaninja</span></a> (Shantanu Jain) </span> <span class="crawler-post-infos"> <time itemprop='datePublished' datetime='2022-04-27T07:52:49Z' class='post-time'> April 27, 2022, 7:52am </time> <meta itemprop='dateModified' content='2022-04-27T08:37:27Z'> <span itemprop='position'>16</span> </span> </div> <div class='post' itemprop='text'> <p>As evidenced by my message to typing-sig (apologies, sent before discussions-to was updated), I’m sympathetic to concerns about breaking bytes / bytearray compatibility. However, I find Jelle’s point about hashability fairly compelling — and in general it’s nice to be able to reflect immutability in one’s types.</p> <blockquote> <p>but I don’t want to get too precise or restrictive here</p> </blockquote> <p>Part of the issue is that using <code>bytes</code> as a shorthand prevents people who want to be precise from being precise. If we choose to remove the shorthand, maybe we could suggest use of <code>typing.ByteString</code> as an “imprecise” type.</p> <h4><a name="breaking-down-type-annotation-use-cases-1" class="anchor" href="#breaking-down-type-annotation-use-cases-1"></a>Breaking down type annotation use cases</h4> <p>A) buffers that are passed to C code that use the buffer protocol<br> B) functions that use all the various methods of <code>builtins.bytes</code> (join, find, lower, etc) — these likely also work on <code>bytearray</code> “for free”<br> C) functions that hash bytes<br> D) functions that iterate over bytes or e.g. extend a <code>bytearray</code></p> <h4><a name="in-the-current-world-where-static-duck-typing-lets-us-treat-bytearray-and-memoryview-as-subtypes-of-bytes-2" class="anchor" href="#in-the-current-world-where-static-duck-typing-lets-us-treat-bytearray-and-memoryview-as-subtypes-of-bytes-2"></a>In the current world (where static duck typing lets us treat <code>bytearray</code> and <code>memoryview</code> as subtypes of <code>bytes</code>)</h4> <p>A) is not expressible<br> B) is typed as <code>bytes</code>. hooray static duck typing, type checkers let us pass <code>bytearray</code> and things work. but boo, static duck typing, we’re allowed to pass <code>memoryview</code> and things don’t work.<br> C) is typed as <code>bytes</code>. boo static duck typing, since type checkers let us pass <code>bytearray</code> and writable <code>memoryview</code><br> D) is often typed as <code>bytes</code>, as shorthand for <code>bytes | bytearray | memoryview</code>, but should probably be typed as <code>Sequence[int]</code></p> <h4><a name="in-the-pep-688-world-as-proposed-3" class="anchor" href="#in-the-pep-688-world-as-proposed-3"></a>In the PEP 688 world as proposed</h4> <p>A) is typed as <code>types.Buffer</code><br> B) is typed as <code>bytes | bytearray</code><br> C) is typed as <code>bytes</code>, or maybe <code>bytes | memoryview</code> if we’re willing to hope noone passes us a writable memoryview<br> D) probably should be typed as <code>Sequence[int]</code></p> <h4><a name="in-a-pep-688-world-but-where-we-allow-bytearray-to-duck-type-to-bytes-but-not-memoryview-4" class="anchor" href="#in-a-pep-688-world-but-where-we-allow-bytearray-to-duck-type-to-bytes-but-not-memoryview-4"></a>In a PEP 688 world but where we allow <code>bytearray</code> to duck type to <code>bytes</code> (but not <code>memoryview</code>)</h4> <p>A) is typed as <code>types.Buffer</code><br> B) is typed as <code>bytes</code><br> C) is typed as <code>bytes</code> but we risk someone passing us a <code>bytearray</code> (or <code>bytes | memoryview</code>, with similar caveat to above)<br> D) probably should be typed as <code>Sequence[int]</code></p> <p>Overall I think I could give or take duck type compatibility between <code>bytearray</code> and <code>bytes</code>, but I do strongly feel we should drop <code>memoryview</code> from the bytes shorthand. The only non <code>Sequence[int]</code>-like thing I can think of that works on <code>bytes | bytearray | memoryview</code> is <code>b.hex()</code></p> </div> <div itemprop="interactionStatistic" itemscope itemtype="http://schema.org/InteractionCounter"> <meta itemprop="interactionType" content="http://schema.org/LikeAction"/> <meta itemprop="userInteractionCount" content="3" /> <span class='post-likes'>3 Likes</span> </div> </div> <div id='post_17' itemprop='comment' itemscope itemtype='http://schema.org/Comment' class='topic-body crawler-post'> <div class='crawler-post-meta'> <span class="creator" itemprop="author" itemscope itemtype="http://schema.org/Person"> <a itemprop="url" href='https://discuss.python.org/u/Jelle'><span itemprop='name'>Jelle</span></a> (Jelle Zijlstra) </span> <span class="crawler-post-infos"> <time itemprop='datePublished' datetime='2022-04-27T14:50:56Z' class='post-time'> April 27, 2022, 2:50pm </time> <meta itemprop='dateModified' content='2022-04-27T14:50:56Z'> <span itemprop='position'>17</span> </span> </div> <div class='post' itemprop='text'> <p><code>types.Buffer</code> provides no Python API at all, except that you are allowed to pass the object to C functions that accept a buffer (such as <code>memoryview()</code>).</p> <p>I realized that this is a problem, because it means with PEP 688 as written now there is no way to express a type like "supports the buffer protocol and <code>__getitem__</code> and <code>__len__</code>" (which is approximately the common interface of <code>bytes</code> and <code>memoryview</code>).</p> <p>So to support that, we need a different approach. Perhaps we can make the core create a <code>__buffer__</code> attribute on types that support the buffer protocol. Then in typing, we can simply create Protocols that check that attribute.</p> <p>What should the value of the attribute be? We can’t support the full buffer protocol in a Python call, but perhaps we could make <code>x.__buffer__()</code> return <code>memoryview(x)</code>. Alternatively, we could just set it to <code>True</code>.</p> </div> <div itemprop="interactionStatistic" itemscope itemtype="http://schema.org/InteractionCounter"> <meta itemprop="interactionType" content="http://schema.org/LikeAction"/> <meta itemprop="userInteractionCount" content="0" /> <span class='post-likes'></span> </div> </div> <div id='post_18' itemprop='comment' itemscope itemtype='http://schema.org/Comment' class='topic-body crawler-post'> <div class='crawler-post-meta'> <span class="creator" itemprop="author" itemscope itemtype="http://schema.org/Person"> <a itemprop="url" href='https://discuss.python.org/u/storchaka'><span itemprop='name'>storchaka</span></a> (Serhiy Storchaka) </span> <span class="crawler-post-infos"> <time itemprop='datePublished' datetime='2022-04-27T14:59:52Z' class='post-time'> April 27, 2022, 2:59pm </time> <meta itemprop='dateModified' content='2022-04-27T14:59:52Z'> <span itemprop='position'>18</span> </span> </div> <div class='post' itemprop='text'> <p>Initially I thought that it has some relation to <a href="https://github.com/python/cpython/issues/58006" class="inline-onebox">Allow objects implemented in pure Python to export PEP 3118 buffers · Issue #58006 · python/cpython · GitHub</a>.</p> </div> <div itemprop="interactionStatistic" itemscope itemtype="http://schema.org/InteractionCounter"> <meta itemprop="interactionType" content="http://schema.org/LikeAction"/> <meta itemprop="userInteractionCount" content="0" /> <span class='post-likes'></span> </div> </div> <div id='post_19' itemprop='comment' itemscope itemtype='http://schema.org/Comment' class='topic-body crawler-post'> <div class='crawler-post-meta'> <span class="creator" itemprop="author" itemscope itemtype="http://schema.org/Person"> <a itemprop="url" href='https://discuss.python.org/u/Jelle'><span itemprop='name'>Jelle</span></a> (Jelle Zijlstra) </span> <span class="crawler-post-infos"> <time itemprop='datePublished' datetime='2022-04-27T15:07:34Z' class='post-time'> April 27, 2022, 3:07pm </time> <meta itemprop='dateModified' content='2022-04-27T15:07:34Z'> <span itemprop='position'>19</span> </span> </div> <div class='post' itemprop='text'> <p>Thanks, that’s interesting. Support for buffer classes written in Python sounds like a job for a different PEP. But I learned that <code>__buffer__</code> already has a meaning in PyPy (<a href="https://doc.pypy.org/en/latest/__pypy__-module.html?highlight=__buffer__#generally-available-functionality">docs</a>), so we shouldn’t use that name for flagging buffers unless we actually replicate PyPy’s functionality. I’d suggest <code>__get_buffer__</code> if we go with a method and <code>__is_buffer__</code> if we make it just a boolean.</p> </div> <div itemprop="interactionStatistic" itemscope itemtype="http://schema.org/InteractionCounter"> <meta itemprop="interactionType" content="http://schema.org/LikeAction"/> <meta itemprop="userInteractionCount" content="0" /> <span class='post-likes'></span> </div> </div> <div id='post_20' itemprop='comment' itemscope itemtype='http://schema.org/Comment' class='topic-body crawler-post'> <div class='crawler-post-meta'> <span class="creator" itemprop="author" itemscope itemtype="http://schema.org/Person"> <a itemprop="url" href='https://discuss.python.org/u/storchaka'><span itemprop='name'>storchaka</span></a> (Serhiy Storchaka) </span> <span class="crawler-post-infos"> <time itemprop='datePublished' datetime='2022-04-27T15:18:04Z' class='post-time'> April 27, 2022, 3:18pm </time> <meta itemprop='dateModified' content='2022-04-27T15:18:04Z'> <span itemprop='position'>20</span> </span> </div> <div class='post' itemprop='text'> <p>There is an old problem with the documentation. It uses terms “buffer” (not clear), “object that implement the buffer protocol” (too long), “bytes string”, “bytestring”, “bytes-like object”.</p> <p>The problem is that there are different degrees of “bytelikeness”.</p> <ol> <li>Supports the buffer protocol.</li> <li>Additionally supports <code>len()</code> which returns the size in bytes.</li> <li>Additionally supports indexing and iteration.</li> <li>Has most of <code>str</code> methods (except <code>encode()</code> of course).</li> <li>Supports the continuous buffer protocol.</li> <li> <code>bytes</code> and <code>bytesarray</code> (they not only support the continuous buffer protocol, they are NUL-terminated).</li> <li> <code>bytes</code> only (it is hashable).</li> </ol> <p>Different functions has different requirements for their arguments, and it is difficult to describe it correctly and unambiguously. We need to establish a non-vague terminology and use it consistently.</p> <p>There is <code>collections.abc.ByteString</code> which I expected could be used for type 4, but seems it is going to be deprecated (<a href="https://github.com/python/cpython/issues/91896" class="inline-onebox">Deprecate collections.abc.ByteString · Issue #91896 · python/cpython · GitHub</a>).</p> </div> <div itemprop="interactionStatistic" itemscope itemtype="http://schema.org/InteractionCounter"> <meta itemprop="interactionType" content="http://schema.org/LikeAction"/> <meta itemprop="userInteractionCount" content="3" /> <span class='post-likes'>3 Likes</span> </div> </div> </div> <div role='navigation' itemscope itemtype='http://schema.org/SiteNavigationElement' class="topic-body crawler-post"> <span itemprop='name'><b><a rel="next" itemprop="url" href="/t/pep-688-making-the-buffer-protocol-accessible-in-python/15265?page=2">next page →</a></b></span> </div> </div> <footer class="container wrap"> <nav class='crawler-nav'> <ul> <li itemscope itemtype='http://schema.org/SiteNavigationElement'> <span itemprop='name'> <a href='/' itemprop="url">Home </a> </span> </li> <li itemscope itemtype='http://schema.org/SiteNavigationElement'> <span itemprop='name'> <a href='/categories' itemprop="url">Categories </a> </span> </li> <li itemscope itemtype='http://schema.org/SiteNavigationElement'> <span itemprop='name'> <a href='/guidelines' itemprop="url">Guidelines </a> </span> </li> <li itemscope itemtype='http://schema.org/SiteNavigationElement'> <span itemprop='name'> <a href='/tos' itemprop="url">Terms of Service </a> </span> </li> <li itemscope itemtype='http://schema.org/SiteNavigationElement'> <span itemprop='name'> <a href='/privacy' itemprop="url">Privacy Policy </a> </span> </li> </ul> </nav> <p class='powered-by-link'>Powered by <a href="https://www.discourse.org">Discourse</a>, best viewed with JavaScript enabled</p> </footer> <div class="buorg"><div>Unfortunately, <a href="https://www.discourse.org/faq/#browser">your browser is unsupported</a>. Please <a href="https://browsehappy.com">switch to a supported browser</a> to view rich content, log in and reply.</div></div> </body> </html>