CINXE.COM

<!DOCTYPE html> <html lang="en"> <head> <meta charset="utf-8"> <meta name="viewport" content="width=device-width, initial-scale=1.0"> <meta name="color-scheme" content="light dark"> <title>PEP 522 – Allow BlockingIOError in security sensitive APIs | peps.python.org</title> <link rel="shortcut icon" href="../_static/py.png"> <link rel="canonical" href="https://peps.python.org/pep-0522/"> <link rel="stylesheet" href="../_static/style.css" type="text/css"> <link rel="stylesheet" href="../_static/mq.css" type="text/css"> <link rel="stylesheet" href="../_static/pygments.css" type="text/css" media="(prefers-color-scheme: light)" id="pyg-light"> <link rel="stylesheet" href="../_static/pygments_dark.css" type="text/css" media="(prefers-color-scheme: dark)" id="pyg-dark"> <link rel="alternate" type="application/rss+xml" title="Latest PEPs" href="https://peps.python.org/peps.rss"> <meta property="og:title" content='PEP 522 – Allow BlockingIOError in security sensitive APIs | peps.python.org'> <meta property="og:description" content="A number of APIs in the standard library that return random values nominally suitable for use in security sensitive operations currently have an obscure operating system dependent failure mode that allows them to return values that are not, in fact, sui..."> <meta property="og:type" content="website"> <meta property="og:url" content="https://peps.python.org/pep-0522/"> <meta property="og:site_name" content="Python Enhancement Proposals (PEPs)"> <meta property="og:image" content="https://peps.python.org/_static/og-image.png"> <meta property="og:image:alt" content="Python PEPs"> <meta property="og:image:width" content="200"> <meta property="og:image:height" content="200"> <meta name="description" content="A number of APIs in the standard library that return random values nominally suitable for use in security sensitive operations currently have an obscure operating system dependent failure mode that allows them to return values that are not, in fact, sui..."> <meta name="theme-color" content="#3776ab"> </head> <body> <svg xmlns="http://www.w3.org/2000/svg" style="display: none;"> <symbol id="svg-sun-half" viewBox="0 0 24 24" pointer-events="all"> <title>Following system colour scheme</title> <svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"> <circle cx="12" cy="12" r="9"></circle> <path d="M12 3v18m0-12l4.65-4.65M12 14.3l7.37-7.37M12 19.6l8.85-8.85"></path> </svg> </symbol> <symbol id="svg-moon" viewBox="0 0 24 24" pointer-events="all"> <title>Selected dark colour scheme</title> <svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"> <path stroke="none" d="M0 0h24v24H0z" fill="none"></path> <path d="M12 3c.132 0 .263 0 .393 0a7.5 7.5 0 0 0 7.92 12.446a9 9 0 1 1 -8.313 -12.454z"></path> </svg> </symbol> <symbol id="svg-sun" viewBox="0 0 24 24" pointer-events="all"> <title>Selected light colour scheme</title> <svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"> <circle cx="12" cy="12" r="5"></circle> <line x1="12" y1="1" x2="12" y2="3"></line> <line x1="12" y1="21" x2="12" y2="23"></line> <line x1="4.22" y1="4.22" x2="5.64" y2="5.64"></line> <line x1="18.36" y1="18.36" x2="19.78" y2="19.78"></line> <line x1="1" y1="12" x2="3" y2="12"></line> <line x1="21" y1="12" x2="23" y2="12"></line> <line x1="4.22" y1="19.78" x2="5.64" y2="18.36"></line> <line x1="18.36" y1="5.64" x2="19.78" y2="4.22"></line> </svg> </symbol> </svg> <script> document.documentElement.dataset.colour_scheme = localStorage.getItem("colour_scheme") || "auto" </script> <section id="pep-page-section"> <header> <h1>Python Enhancement Proposals</h1> <ul class="breadcrumbs"> <li><a href="https://www.python.org/" title="The Python Programming Language">Python</a> » </li> <li><a href="../pep-0000/">PEP Index</a> » </li> <li>PEP 522</li> </ul> <button id="colour-scheme-cycler" onClick="setColourScheme(nextColourScheme())"> <svg aria-hidden="true" class="colour-scheme-icon-when-auto"><use href="#svg-sun-half"></use></svg> <svg aria-hidden="true" class="colour-scheme-icon-when-dark"><use href="#svg-moon"></use></svg> <svg aria-hidden="true" class="colour-scheme-icon-when-light"><use href="#svg-sun"></use></svg> Toggle light / dark / auto colour theme </button> </header> <article> <section id="pep-content"> <h1 class="page-title">PEP 522 – Allow BlockingIOError in security sensitive APIs</h1> <dl class="rfc2822 field-list simple"> <dt class="field-odd">Author:</dt> <dd class="field-odd">Alyssa Coghlan <ncoghlan at gmail.com>, Nathaniel J. Smith <njs at pobox.com></dd> <dt class="field-even">Status:</dt> <dd class="field-even"><abbr title="Formally declined and will not be accepted">Rejected</abbr></dd> <dt class="field-odd">Type:</dt> <dd class="field-odd"><abbr title="Normative PEP with a new feature for Python, implementation change for CPython or interoperability standard for the ecosystem">Standards Track</abbr></dd> <dt class="field-even">Requires:</dt> <dd class="field-even"><a class="reference external" href="../pep-0506/">506</a></dd> <dt class="field-odd">Created:</dt> <dd class="field-odd">16-Jun-2016</dd> <dt class="field-even">Python-Version:</dt> <dd class="field-even">3.6</dd> <dt class="field-odd">Resolution:</dt> <dd class="field-odd"><a class="reference external" href="https://mail.python.org/pipermail/security-sig/2016-August/000101.html">Security-SIG message</a></dd> </dl> <hr class="docutils" /> <section id="contents"> <details><summary>Table of Contents</summary><ul class="simple"> <li><a class="reference internal" href="#abstract">Abstract</a></li> <li><a class="reference internal" href="#relationship-with-other-peps">Relationship with other PEPs</a></li> <li><a class="reference internal" href="#pep-rejection">PEP Rejection</a></li> <li><a class="reference internal" href="#changes-independent-of-this-pep">Changes independent of this PEP</a></li> <li><a class="reference internal" href="#proposal">Proposal</a><ul> <li><a class="reference internal" href="#changing-os-urandom-on-platforms-with-the-getrandom-system-call">Changing <code class="docutils literal notranslate">os.urandom()</code> on platforms with the getrandom() system call</a></li> <li><a class="reference internal" href="#adding-secrets-wait-for-system-rng">Adding <code class="docutils literal notranslate">secrets.wait_for_system_rng()</code></a></li> <li><a class="reference internal" href="#limitations-on-scope">Limitations on scope</a></li> </ul> </li> <li><a class="reference internal" href="#rationale">Rationale</a><ul> <li><a class="reference internal" href="#ensuring-the-secrets-module-implicitly-blocks-when-needed">Ensuring the <code class="docutils literal notranslate">secrets</code> module implicitly blocks when needed</a></li> <li><a class="reference internal" href="#raising-blockingioerror-in-os-urandom-on-linux">Raising <code class="docutils literal notranslate">BlockingIOError</code> in <code class="docutils literal notranslate">os.urandom()</code> on Linux</a></li> <li><a class="reference internal" href="#making-secrets-wait-for-system-rng-public">Making <code class="docutils literal notranslate">secrets.wait_for_system_rng()</code> public</a></li> </ul> </li> <li><a class="reference internal" href="#backwards-compatibility-impact-assessment">Backwards Compatibility Impact Assessment</a><ul> <li><a class="reference internal" href="#unaffected-applications">Unaffected Applications</a></li> <li><a class="reference internal" href="#affected-security-sensitive-applications">Affected security sensitive applications</a></li> <li><a class="reference internal" href="#affected-non-security-sensitive-applications">Affected non-security sensitive applications</a></li> </ul> </li> <li><a class="reference internal" href="#additional-background">Additional Background</a><ul> <li><a class="reference internal" href="#why-propose-this-now">Why propose this now?</a></li> <li><a class="reference internal" href="#the-cross-platform-behaviour-of-os-urandom">The cross-platform behaviour of <code class="docutils literal notranslate">os.urandom()</code></a></li> <li><a class="reference internal" href="#problems-with-the-behaviour-of-dev-urandom-on-linux">Problems with the behaviour of <code class="docutils literal notranslate">/dev/urandom</code> on Linux</a></li> <li><a class="reference internal" href="#consequences-of-getrandom-availability-for-python">Consequences of <code class="docutils literal notranslate">getrandom()</code> availability for Python</a></li> </ul> </li> <li><a class="reference internal" href="#references">References</a></li> <li><a class="reference internal" href="#copyright">Copyright</a></li> </ul> </details></section> <section id="abstract"> <h2><a class="toc-backref" href="#abstract" role="doc-backlink">Abstract</a></h2> A number of APIs in the standard library that return random values nominally suitable for use in security sensitive operations currently have an obscure operating system dependent failure mode that allows them to return values that are not, in fact, suitable for such operations. This is due to some operating system kernels (most notably the Linux kernel) permitting reads from <code class="docutils literal notranslate">/dev/urandom</code> before the system random number generator is fully initialized, whereas most other operating systems will implicitly block on such reads until the random number generator is ready. For the lower level <code class="docutils literal notranslate">os.urandom</code> and <code class="docutils literal notranslate">random.SystemRandom</code> APIs, this PEP proposes changing such failures in Python 3.6 from the current silent, hard to detect, and hard to debug, errors to easily detected and debugged errors by raising <code class="docutils literal notranslate">BlockingIOError</code> with a suitable error message, allowing developers the opportunity to unambiguously specify their preferred approach for handling the situation. For the new high level <code class="docutils literal notranslate">secrets</code> API, it proposes to block implicitly if needed whenever random number is generated by that module, as well as to expose a new <code class="docutils literal notranslate">secrets.wait_for_system_rng()</code> function to allow code otherwise using the low level APIs to explicitly wait for the system random number generator to be available. This change will impact any operating system that offers the <code class="docutils literal notranslate">getrandom()</code> system call, regardless of whether the default behaviour of the <code class="docutils literal notranslate">/dev/urandom</code> device is to return potentially predictable results when the system random number generator is not ready (e.g. Linux, NetBSD) or to block (e.g. FreeBSD, Solaris, Illumos). Operating systems that prevent execution of userspace code prior to the initialization of the system random number generator, or do not offer the <code class="docutils literal notranslate">getrandom()</code> syscall, will be entirely unaffected by the proposed change (e.g. Windows, Mac OS X, OpenBSD). The new exception or the blocking behaviour in the <code class="docutils literal notranslate">secrets</code> module would potentially be encountered in the following situations: <ul class="simple"> <li>Python code calling these APIs during Linux system initialization</li> <li>Python code running on improperly initialized Linux systems (e.g. embedded hardware without adequate sources of entropy to seed the system random number generator, or Linux VMs that aren’t configured to accept entropy from the VM host)</li> </ul> </section> <section id="relationship-with-other-peps"> <h2><a class="toc-backref" href="#relationship-with-other-peps" role="doc-backlink">Relationship with other PEPs</a></h2> This PEP depends on the Accepted <a class="pep reference internal" href="../pep-0506/" title="PEP 506 – Adding A Secrets Module To The Standard Library">PEP 506</a>, which adds the <code class="docutils literal notranslate">secrets</code> module. This PEP competes with Victor Stinner’s <a class="pep reference internal" href="../pep-0524/" title="PEP 524 – Make os.urandom() blocking on Linux">PEP 524</a>, which proposes to make <code class="docutils literal notranslate">os.urandom</code> itself implicitly block when the system RNG is not ready. </section> <section id="pep-rejection"> <h2><a class="toc-backref" href="#pep-rejection" role="doc-backlink">PEP Rejection</a></h2> For the reference implementation, Guido rejected this PEP in favour of the unconditional implicit blocking proposal in <a class="pep reference internal" href="../pep-0524/" title="PEP 524 – Make os.urandom() blocking on Linux">PEP 524</a> (which brings CPython’s behaviour on Linux into line with its behaviour on other operating systems). This means any further discussion of appropriate default behaviour for <code class="docutils literal notranslate">os.urandom()</code> in system Python installations in Linux distributions should take place on the respective distro mailing lists, rather than on the upstream CPython mailing lists. </section> <section id="changes-independent-of-this-pep"> <h2><a class="toc-backref" href="#changes-independent-of-this-pep" role="doc-backlink">Changes independent of this PEP</a></h2> CPython interpreter initialization and <code class="docutils literal notranslate">random</code> module initialization have already been updated to gracefully fall back to alternative seeding options if the system random number generator is not ready. This PEP does not compete with the proposal in <a class="pep reference internal" href="../pep-0524/" title="PEP 524 – Make os.urandom() blocking on Linux">PEP 524</a> to add an <code class="docutils literal notranslate">os.getrandom()</code> API to expose the <code class="docutils literal notranslate">getrandom</code> syscall on platforms that offer it. There is sufficient motive for adding that API in the <code class="docutils literal notranslate">os</code> module’s role as a thin wrapper around potentially platform dependent operating system features that it can be added regardless of what happens to the default behaviour of <code class="docutils literal notranslate">os.urandom()</code> on these systems. </section> <section id="proposal"> <h2><a class="toc-backref" href="#proposal" role="doc-backlink">Proposal</a></h2> <section id="changing-os-urandom-on-platforms-with-the-getrandom-system-call"> <h3><a class="toc-backref" href="#changing-os-urandom-on-platforms-with-the-getrandom-system-call" role="doc-backlink">Changing <code class="docutils literal notranslate">os.urandom()</code> on platforms with the getrandom() system call</a></h3> This PEP proposes that in Python 3.6+, <code class="docutils literal notranslate">os.urandom()</code> be updated to call the <code class="docutils literal notranslate">getrandom()</code> syscall in non-blocking mode if available and raise <code class="docutils literal notranslate">BlockingIOError: system random number generator is not ready; see secrets.token_bytes()</code> if the kernel reports that the call would block. This behaviour will then propagate through to the existing <code class="docutils literal notranslate">random.SystemRandom</code>, which provides a relatively thin wrapper around <code class="docutils literal notranslate">os.urandom()</code> that matches the <code class="docutils literal notranslate">random.Random()</code> API. However, the new <code class="docutils literal notranslate">secrets</code> module introduced by <a class="pep reference internal" href="../pep-0506/" title="PEP 506 – Adding A Secrets Module To The Standard Library">PEP 506</a> will be updated to catch the new exception and implicitly wait for the system random number generator if the exception is ever encountered. In all cases, as soon as a call to one of these security sensitive APIs succeeds, all future calls to these APIs in that process will succeed without blocking (once the operating system random number generator is ready after system boot, it remains ready). On Linux and NetBSD, this will replace the previous behaviour of returning potentially predictable results read from <code class="docutils literal notranslate">/dev/urandom</code>. On FreeBSD, Solaris, and Illumos, this will replace the previous behaviour of implicitly blocking until the system random number generator is ready. However, it is not clear if these operating systems actually allow userspace code (and hence Python) to run before the system random number generator is ready. Note that in all cases, if calling the underlying <code class="docutils literal notranslate">getrandom()</code> API reports <code class="docutils literal notranslate">ENOSYS</code> rather than returning a successful response or reporting <code class="docutils literal notranslate">EAGAIN</code>, CPython will continue to fall back to reading from <code class="docutils literal notranslate">/dev/urandom</code> directly. </section> <section id="adding-secrets-wait-for-system-rng"> <h3><a class="toc-backref" href="#adding-secrets-wait-for-system-rng" role="doc-backlink">Adding <code class="docutils literal notranslate">secrets.wait_for_system_rng()</code></a></h3> A new exception shouldn’t be added without a straightforward recommendation for how to resolve that error when encountered (however rare encountering the new error is expected to be in practice). For security sensitive code that actually does need to use the lower level interfaces to the system random number generator (rather than the new <code class="docutils literal notranslate">secrets</code> module), and does receive live bug reports indicating this is a real problem for the userbase of that particular application rather than a theoretical one, this PEP’s recommendation will be to add the following snippet (directly or indirectly) to the <code class="docutils literal notranslate">__main__</code> module: <div class="highlight-default notranslate"><div class="highlight"><pre>import secrets secrets.wait_for_system_rng() </pre></div> </div> Or, if compatibility with versions prior to Python 3.6 is needed: <div class="highlight-default notranslate"><div class="highlight"><pre>try: import secrets except ImportError: pass else: secrets.wait_for_system_rng() </pre></div> </div> Within the <code class="docutils literal notranslate">secrets</code> module itself, this will then be used in <code class="docutils literal notranslate">token_bytes()</code> to block implicitly if the new exception is encountered: <div class="highlight-default notranslate"><div class="highlight"><pre>def token_bytes(nbytes=None): if nbytes is None: nbytes = DEFAULT_ENTROPY try: result = os.urandom(nbytes) except BlockingIOError: wait_for_system_rng() result = os.urandom(nbytes) return result </pre></div> </div> Other parts of the module will then be updated to use <code class="docutils literal notranslate">token_bytes()</code> as their basic random number generation building block, rather than calling <code class="docutils literal notranslate">os.urandom()</code> directly. Application frameworks covering use cases where access to the system random number generator is almost certain to be needed (e.g. web frameworks) may choose to incorporate a call to <code class="docutils literal notranslate">secrets.wait_for_system_rng()</code> implicitly into the commands that start the application such that existing calls to <code class="docutils literal notranslate">os.urandom()</code> will be guaranteed to never raise the new exception when using those frameworks. For cases where the error is encountered for an application which cannot be modified directly, then the following command can be used to wait for the system random number generator to initialize before starting that application: <div class="highlight-default notranslate"><div class="highlight"><pre>python3 -c "import secrets; secrets.wait_for_system_rng()" </pre></div> </div> For example, this snippet could be added to a shell script or a systemd <code class="docutils literal notranslate">ExecStartPre</code> hook (and may prove useful in reliably waiting for the system random number generator to be ready, even if the subsequent command is not itself an application running under Python 3.6) Given the changes proposed to <code class="docutils literal notranslate">os.urandom()</code> above, and the inclusion of an <code class="docutils literal notranslate">os.getrandom()</code> API on systems that support it, the suggested implementation of this function would be: <div class="highlight-default notranslate"><div class="highlight"><pre>if hasattr(os, "getrandom"): # os.getrandom() always blocks waiting for the system RNG by default def wait_for_system_rng(): """Block waiting for system random number generator to be ready""" os.getrandom(1) return else: # As far as we know, other platforms will never get BlockingIOError # below but the implementation makes pessimistic assumptions def wait_for_system_rng(): """Block waiting for system random number generator to be ready""" # If the system RNG is already seeded, don't wait at all try: os.urandom(1) return except BlockingIOError: pass # Avoid the below busy loop if possible try: block_on_system_rng = open("/dev/random", "rb") except FileNotFoundError: pass else: with block_on_system_rng: block_on_system_rng.read(1) # Busy loop until the system RNG is ready while True: try: os.urandom(1) break except BlockingIOError: # Only check once per millisecond time.sleep(0.001) </pre></div> </div> On systems where it is possible to wait for the system RNG to be ready, this function will do so without a busy loop if <code class="docutils literal notranslate">os.getrandom()</code> is defined, <code class="docutils literal notranslate">os.urandom()</code> itself implicitly blocks, or the <code class="docutils literal notranslate">/dev/random</code> device is available. If the system random number generator is ready, this call is guaranteed to never block, even if the system’s <code class="docutils literal notranslate">/dev/random</code> device uses a design that permits it to block intermittently during normal system operation. </section> <section id="limitations-on-scope"> <h3><a class="toc-backref" href="#limitations-on-scope" role="doc-backlink">Limitations on scope</a></h3> No changes are proposed for Windows or Mac OS X systems, as neither of those platforms provides any mechanism to run Python code before the operating system random number generator has been initialized. Mac OS X goes so far as to kernel panic and abort the boot process if it can’t properly initialize the random number generator (although Apple’s restrictions on the supported hardware platforms make that exceedingly unlikely in practice). Similarly, no changes are proposed for other *nix systems that do not offer the <code class="docutils literal notranslate">getrandom()</code> syscall. On these systems, <code class="docutils literal notranslate">os.urandom()</code> will continue to block waiting for the system random number generator to be initialized. While other *nix systems that offer a non-blocking API (other than <code class="docutils literal notranslate">getrandom()</code>) for requesting random numbers suitable for use in security sensitive applications could potentially receive a similar update to the one proposed for <code class="docutils literal notranslate">getrandom()</code> in this PEP, such changes are out of scope for this particular proposal. Python’s behaviour on older versions of affected platforms that do not offer the new <code class="docutils literal notranslate">getrandom()</code> syscall will also remain unchanged. </section> </section> <section id="rationale"> <h2><a class="toc-backref" href="#rationale" role="doc-backlink">Rationale</a></h2> <section id="ensuring-the-secrets-module-implicitly-blocks-when-needed"> <h3><a class="toc-backref" href="#ensuring-the-secrets-module-implicitly-blocks-when-needed" role="doc-backlink">Ensuring the <code class="docutils literal notranslate">secrets</code> module implicitly blocks when needed</a></h3> This is done to help encourage the meme that arises for folks that want the simplest possible answer to the right way to generate security sensitive random numbers to be “Use the secrets module when available or your application might crash unexpectedly”, rather than the more boilerplate heavy “Always call secrets.wait_for_system_rng() when available or your application might crash unexpectedly”. It’s also done due to the BDFL having a higher tolerance for APIs that might block unexpectedly than he does for APIs that might throw an unexpected exception <a class="footnote-reference brackets" href="#id22" id="id1">[11]</a>. </section> <section id="raising-blockingioerror-in-os-urandom-on-linux"> <h3><a class="toc-backref" href="#raising-blockingioerror-in-os-urandom-on-linux" role="doc-backlink">Raising <code class="docutils literal notranslate">BlockingIOError</code> in <code class="docutils literal notranslate">os.urandom()</code> on Linux</a></h3> For several years now, the security community’s guidance has been to use <code class="docutils literal notranslate">os.urandom()</code> (or the <code class="docutils literal notranslate">random.SystemRandom()</code> wrapper) when implementing security sensitive operations in Python. To help improve API discoverability and make it clearer that secrecy and simulation are not the same problem (even though they both involve random numbers), <a class="pep reference internal" href="../pep-0506/" title="PEP 506 – Adding A Secrets Module To The Standard Library">PEP 506</a> collected several of the one line recipes based on the lower level <code class="docutils literal notranslate">os.urandom()</code> API into a new <code class="docutils literal notranslate">secrets</code> module. However, this guidance has also come with a longstanding caveat: developers writing security sensitive software at least for Linux, and potentially for some other *BSD systems, may need to wait until the operating system’s random number generator is ready before relying on it for security sensitive operations. This generally only occurs if <code class="docutils literal notranslate">os.urandom()</code> is read very early in the system initialization process, or on systems with few sources of available entropy (e.g. some kinds of virtualized or embedded systems), but unfortunately the exact conditions that trigger this are difficult to predict, and when it occurs then there is no direct way for userspace to tell it has happened without querying operating system specific interfaces. On *BSD systems (if the particular *BSD variant allows the problem to occur at all) and potentially also Solaris and Illumos, encountering this situation means <code class="docutils literal notranslate">os.urandom()</code> will either block waiting for the system random number generator to be ready (the associated symptom would be for the affected script to pause unexpectedly on the first call to <code class="docutils literal notranslate">os.urandom()</code>) or else will behave the same way as it does on Linux. On Linux, in Python versions up to and including Python 3.4, and in Python 3.5 maintenance versions following Python 3.5.2, there’s no clear indicator to developers that their software may not be working as expected when run early in the Linux boot process, or on hardware without good sources of entropy to seed the operating system’s random number generator: due to the behaviour of the underlying <code class="docutils literal notranslate">/dev/urandom</code> device, <code class="docutils literal notranslate">os.urandom()</code> on Linux returns a result either way, and it takes extensive statistical analysis to show that a security vulnerability exists. By contrast, if <code class="docutils literal notranslate">BlockingIOError</code> is raised in those situations, then developers using Python 3.6+ can easily choose their desired behaviour: <ol class="arabic simple"> <li>Wait for the system RNG at or before application startup (security sensitive)</li> <li>Switch to using the random module (non-security sensitive)</li> </ol> </section> <section id="making-secrets-wait-for-system-rng-public"> <h3><a class="toc-backref" href="#making-secrets-wait-for-system-rng-public" role="doc-backlink">Making <code class="docutils literal notranslate">secrets.wait_for_system_rng()</code> public</a></h3> Earlier versions of this PEP proposed a number of recipes for wrapping <code class="docutils literal notranslate">os.urandom()</code> to make it suitable for use in security sensitive use cases. Discussion of the proposal on the security-sig mailing list prompted the realization <a class="footnote-reference brackets" href="#id20" id="id2">[9]</a> that the core assumption driving the API design in this PEP was that choosing between letting the exception cause the application to fail, blocking waiting for the system RNG to be ready and switching to using the <code class="docutils literal notranslate">random</code> module instead of <code class="docutils literal notranslate">os.urandom</code> is an application and use-case specific decision that should take into account application and use-case specific details. There is no way for the interpreter runtime or support libraries to determine whether a particular use case is security sensitive or not, and while it’s straightforward for application developer to decide how to handle an exception thrown by a particular API, they can’t readily workaround an API blocking when they expected it to be non-blocking. Accordingly, the PEP was updated to add <code class="docutils literal notranslate">secrets.wait_for_system_rng()</code> as an API for applications, scripts and frameworks to use to indicate that they wanted to ensure the system RNG was available before continuing, while library developers could continue to call <code class="docutils literal notranslate">os.urandom()</code> without worrying that it might unexpectedly start blocking waiting for the system RNG to be available. </section> </section> <section id="backwards-compatibility-impact-assessment"> <h2><a class="toc-backref" href="#backwards-compatibility-impact-assessment" role="doc-backlink">Backwards Compatibility Impact Assessment</a></h2> Similar to <a class="pep reference internal" href="../pep-0476/" title="PEP 476 – Enabling certificate verification by default for stdlib http clients">PEP 476</a>, this is a proposal to turn a previously silent security failure into a noisy exception that requires the application developer to make an explicit decision regarding the behaviour they desire. As no changes are proposed for operating systems that don’t provide the <code class="docutils literal notranslate">getrandom()</code> syscall, <code class="docutils literal notranslate">os.urandom()</code> retains its existing behaviour as a nominally blocking API that is non-blocking in practice due to the difficulty of scheduling Python code to run before the operating system random number generator is ready. We believe it may be possible to encounter problems akin to those described in this PEP on at least some *BSD variants, but nobody has explicitly demonstrated that. On Mac OS X and Windows, it appears to be straight up impossible to even try to run a Python interpreter that early in the boot process. On Linux and other platforms with similar <code class="docutils literal notranslate">/dev/urandom</code> behaviour, <code class="docutils literal notranslate">os.urandom()</code> retains its status as a guaranteed non-blocking API. However, the means of achieving that status changes in the specific case of the operating system random number generator not being ready for use in security sensitive operations: historically it would return potentially predictable random data, with this PEP it would change to raise <code class="docutils literal notranslate">BlockingIOError</code>. Developers of affected applications would then be required to make one of the following changes to gain forward compatibility with Python 3.6, based on the kind of application they’re developing. <section id="unaffected-applications"> <h3><a class="toc-backref" href="#unaffected-applications" role="doc-backlink">Unaffected Applications</a></h3> The following kinds of applications would be entirely unaffected by the change, regardless of whether or not they perform security sensitive operations: <ul class="simple"> <li>applications that don’t support Linux</li> <li>applications that are only run on desktops or conventional servers</li> <li>applications that are only run after the system RNG is ready (including those where an application framework calls <code class="docutils literal notranslate">secrets.wait_for_system_rng()</code> on their behalf)</li> </ul> Applications in this category simply won’t encounter the new exception, so it will be reasonable for developers to wait and see if they receive Python 3.6 compatibility bugs related to the new runtime behaviour, rather than attempting to pre-emptively determine whether or not they’re affected. </section> <section id="affected-security-sensitive-applications"> <h3><a class="toc-backref" href="#affected-security-sensitive-applications" role="doc-backlink">Affected security sensitive applications</a></h3> Security sensitive applications would need to either change their system configuration so the application is only started after the operating system random number generator is ready for security sensitive operations, change the application startup code to invoke <code class="docutils literal notranslate">secrets.wait_for_system_rng()</code>, or else switch to using the new <code class="docutils literal notranslate">secrets.token_bytes()</code> API. As an example for components started via a systemd unit file, the following snippet would delay activation until the system RNG was ready: <blockquote> <div>ExecStartPre=python3 -c “import secrets; secrets.wait_for_system_rng()”</div></blockquote> Alternatively, the following snippet will use <code class="docutils literal notranslate">secrets.token_bytes()</code> if available, and fall back to <code class="docutils literal notranslate">os.urandom()</code> otherwise: <blockquote> <div><dl class="simple"> <dt>try:</dt><dd>import secrets.token_bytes as _get_random_bytes</dd> <dt>except ImportError:</dt><dd>import os.urandom as _get_random_bytes</dd> </dl> </div></blockquote> </section> <section id="affected-non-security-sensitive-applications"> <h3><a class="toc-backref" href="#affected-non-security-sensitive-applications" role="doc-backlink">Affected non-security sensitive applications</a></h3> Non-security sensitive applications should be updated to use the <code class="docutils literal notranslate">random</code> module rather than <code class="docutils literal notranslate">os.urandom</code>: <div class="highlight-default notranslate"><div class="highlight"><pre>def pseudorandom_bytes(num_bytes): return random.getrandbits(num_bytes*8).to_bytes(num_bytes, "little") </pre></div> </div> Depending on the details of the application, the random module may offer other APIs that can be used directly, rather than needing to emulate the raw byte sequence produced by the <code class="docutils literal notranslate">os.urandom()</code> API. </section> </section> <section id="additional-background"> <h2><a class="toc-backref" href="#additional-background" role="doc-backlink">Additional Background</a></h2> <section id="why-propose-this-now"> <h3><a class="toc-backref" href="#why-propose-this-now" role="doc-backlink">Why propose this now?</a></h3> The main reason is because the Python 3.5.0 release switched to using the new Linux <code class="docutils literal notranslate">getrandom()</code> syscall when available in order to avoid consuming a file descriptor <a class="footnote-reference brackets" href="#id12" id="id3">[1]</a>, and this had the side effect of making the following operations block waiting for the system random number generator to be ready: <ul class="simple"> <li><code class="docutils literal notranslate">os.urandom</code> (and APIs that depend on it)</li> <li>importing the <code class="docutils literal notranslate">random</code> module</li> <li>initializing the randomized hash algorithm used by some builtin types</li> </ul> While the first of those behaviours is arguably desirable (and consistent with the existing behaviour of <code class="docutils literal notranslate">os.urandom</code> on other operating systems), the latter two behaviours are unnecessary and undesirable, and the last one is now known to cause a system level deadlock when attempting to run Python scripts during the Linux init process with Python 3.5.0 or 3.5.1 <a class="footnote-reference brackets" href="#id13" id="id4">[2]</a>, while the second one can cause problems when using virtual machines without robust entropy sources configured <a class="footnote-reference brackets" href="#id14" id="id5">[3]</a>. Since decoupling these behaviours in CPython will involve a number of implementation changes more appropriate for a feature release than a maintenance release, the relatively simple resolution applied in Python 3.5.2 was to revert all three of them to a behaviour similar to that of previous Python versions: if the new Linux syscall indicates it will block, then Python 3.5.2 will implicitly fall back on reading <code class="docutils literal notranslate">/dev/urandom</code> directly <a class="footnote-reference brackets" href="#id15" id="id6">[4]</a>. However, this bug report also resulted in a range of proposals to add new APIs like <code class="docutils literal notranslate">os.getrandom()</code> <a class="footnote-reference brackets" href="#id16" id="id7">[5]</a>, <code class="docutils literal notranslate">os.urandom_block()</code> <a class="footnote-reference brackets" href="#id17" id="id8">[6]</a>, <code class="docutils literal notranslate">os.pseudorandom()</code> and <code class="docutils literal notranslate">os.cryptorandom()</code> <a class="footnote-reference brackets" href="#id18" id="id9">[7]</a>, or adding new optional parameters to <code class="docutils literal notranslate">os.urandom()</code> itself <a class="footnote-reference brackets" href="#id19" id="id10">[8]</a>, and then attempting to educate users on when they should call those APIs instead of just using a plain <code class="docutils literal notranslate">os.urandom()</code> call. These proposals arguably represent overreactions, as the question of reliably obtaining random numbers suitable for security sensitive work on Linux is a relatively obscure problem of interest mainly to operating system developers and embedded systems programmers, that may not justify expanding the Python standard library’s cross-platform APIs with new Linux-specific concerns. This is especially so with the <code class="docutils literal notranslate">secrets</code> module already being added as the “use this and don’t worry about the low level details” option for developers writing security sensitive software that for some reason can’t rely on even higher level domain specific APIs (like web frameworks) and also don’t need to worry about Python versions prior to Python 3.6. That said, it’s also the case that low cost ARM devices are becoming increasingly prevalent, with a lot of them running Linux, and a lot of folks writing Python applications that run on those devices. That creates an opportunity to take an obscure security problem that currently requires a lot of knowledge about Linux boot processes and provably unpredictable random number generation to diagnose and resolve, and instead turn it into a relatively mundane and easy-to-find-in-an-internet-search runtime exception. </section> <section id="the-cross-platform-behaviour-of-os-urandom"> <h3><a class="toc-backref" href="#the-cross-platform-behaviour-of-os-urandom" role="doc-backlink">The cross-platform behaviour of <code class="docutils literal notranslate">os.urandom()</code></a></h3> On operating systems other than Linux and NetBSD, <code class="docutils literal notranslate">os.urandom()</code> may already block waiting for the operating system’s random number generator to be ready. This will happen at most once in the lifetime of the process, and the call is subsequently guaranteed to be non-blocking. Linux and NetBSD are outliers in that, even when the operating system’s random number generator doesn’t consider itself ready for use in security sensitive operations, reading from the <code class="docutils literal notranslate">/dev/urandom</code> device will return random values based on the entropy it has available. This behaviour is potentially problematic, so Linux 3.17 added a new <code class="docutils literal notranslate">getrandom()</code> syscall that (amongst other benefits) allows callers to either block waiting for the random number generator to be ready, or else request an error return if the random number generator is not ready. Notably, the new API does not support the old behaviour of returning data that is not suitable for security sensitive use cases. Versions of Python prior up to and including Python 3.4 access the Linux <code class="docutils literal notranslate">/dev/urandom</code> device directly. Python 3.5.0 and 3.5.1 (when build on a system that offered the new syscall) called <code class="docutils literal notranslate">getrandom()</code> in blocking mode in order to avoid the use of a file descriptor to access <code class="docutils literal notranslate">/dev/urandom</code>. While there were no specific problems reported due to <code class="docutils literal notranslate">os.urandom()</code> blocking in user code, there were problems due to CPython implicitly invoking the blocking behaviour during interpreter startup and when importing the <code class="docutils literal notranslate">random</code> module. Rather than trying to decouple SipHash initialization from the <code class="docutils literal notranslate">os.urandom()</code> implementation, Python 3.5.2 switched to calling <code class="docutils literal notranslate">getrandom()</code> in non-blocking mode, and falling back to reading from <code class="docutils literal notranslate">/dev/urandom</code> if the syscall indicates it will block. As a result of the above, <code class="docutils literal notranslate">os.urandom()</code> in all Python versions up to and including Python 3.5 propagate the behaviour of the underling <code class="docutils literal notranslate">/dev/urandom</code> device to Python code. </section> <section id="problems-with-the-behaviour-of-dev-urandom-on-linux"> <h3><a class="toc-backref" href="#problems-with-the-behaviour-of-dev-urandom-on-linux" role="doc-backlink">Problems with the behaviour of <code class="docutils literal notranslate">/dev/urandom</code> on Linux</a></h3> The Python <code class="docutils literal notranslate">os</code> module has largely co-evolved with Linux APIs, so having <code class="docutils literal notranslate">os</code> module functions closely follow the behaviour of their Linux operating system level counterparts when running on Linux is typically considered to be a desirable feature. However, <code class="docutils literal notranslate">/dev/urandom</code> represents a case where the current behaviour is acknowledged to be problematic, but fixing it unilaterally at the kernel level has been shown to prevent some Linux distributions from booting (at least in part due to components like Python currently using it for non-security-sensitive purposes early in the system initialization process). As an analogy, consider the following two functions: <div class="highlight-default notranslate"><div class="highlight"><pre>def generate_example_password(): """Generates passwords solely for use in code examples""" return generate_unpredictable_password() def generate_actual_password(): """Generates actual passwords for use in real applications""" return generate_unpredictable_password() </pre></div> </div> If you think of an operating system’s random number generator as a method for generating unpredictable, secret passwords, then you can think of Linux’s <code class="docutils literal notranslate">/dev/urandom</code> as being implemented like: <div class="highlight-default notranslate"><div class="highlight"><pre># Oversimplified artist's conception of the kernel code # implementing /dev/urandom def generate_unpredictable_password(): if system_rng_is_ready: return use_system_rng_to_generate_password() else: # we can't make an unpredictable password; silently return a # potentially predictable one instead: return "p4ssw0rd" </pre></div> </div> In this scenario, the author of <code class="docutils literal notranslate">generate_example_password</code> is fine - even if <code class="docutils literal notranslate">"p4ssw0rd"</code> shows up a bit more often than they expect, it’s only used in examples anyway. However, the author of <code class="docutils literal notranslate">generate_actual_password</code> has a problem - how do they prove that their calls to <code class="docutils literal notranslate">generate_unpredictable_password</code> never follow the path that returns a predictable answer? In real life it’s slightly more complicated than this, because there might be some level of system entropy available – so the fallback might be more like <code class="docutils literal notranslate">return random.choice(["p4ssword", "passw0rd", "p4ssw0rd"])</code> or something even more variable and hence only statistically predictable with better odds than the author of <code class="docutils literal notranslate">generate_actual_password</code> was expecting. This doesn’t really make things more provably secure, though; mostly it just means that if you try to catch the problem in the obvious way – <code class="docutils literal notranslate">if returned_password == "p4ssw0rd": raise UhOh</code> – then it doesn’t work, because <code class="docutils literal notranslate">returned_password</code> might instead be <code class="docutils literal notranslate">p4ssword</code> or even <code class="docutils literal notranslate">pa55word</code>, or just an arbitrary 64 bit sequence selected from fewer than 2**64 possibilities. So this rough sketch does give the right general idea of the consequences of the “more predictable than expected” fallback behaviour, even though it’s thoroughly unfair to the Linux kernel team’s efforts to mitigate the practical consequences of this problem without resorting to breaking backwards compatibility. This design is generally agreed to be a bad idea. As far as we can tell, there are no use cases whatsoever in which this is the behavior you actually want. It has led to the use of insecure <code class="docutils literal notranslate">ssh</code> keys on real systems, and many *nix-like systems (including at least Mac OS X, OpenBSD, and FreeBSD) have modified their <code class="docutils literal notranslate">/dev/urandom</code> implementations so that they never return predictable outputs, either by making reads block in this case, or by simply refusing to run any userspace programs until the system RNG has been initialized. Unfortunately, Linux has so far been unable to follow suit, because it’s been empirically determined that enabling the blocking behavior causes some currently extant distributions to fail to boot. Instead, the new <code class="docutils literal notranslate">getrandom()</code> syscall was introduced, making it possible for userspace applications to access the system random number generator safely, without introducing hard to debug deadlock problems into the system initialization processes of existing Linux distros. </section> <section id="consequences-of-getrandom-availability-for-python"> <h3><a class="toc-backref" href="#consequences-of-getrandom-availability-for-python" role="doc-backlink">Consequences of <code class="docutils literal notranslate">getrandom()</code> availability for Python</a></h3> Prior to the introduction of the <code class="docutils literal notranslate">getrandom()</code> syscall, it simply wasn’t feasible to access the Linux system random number generator in a provably safe way, so we were forced to settle for reading from <code class="docutils literal notranslate">/dev/urandom</code> as the best available option. However, with <code class="docutils literal notranslate">getrandom()</code> insisting on raising an error or blocking rather than returning predictable data, as well as having other advantages, it is now the recommended method for accessing the kernel RNG on Linux, with reading <code class="docutils literal notranslate">/dev/urandom</code> directly relegated to “legacy” status. This moves Linux into the same category as other operating systems like Windows, which doesn’t provide a <code class="docutils literal notranslate">/dev/urandom</code> device at all: the best available option for implementing <code class="docutils literal notranslate">os.urandom()</code> is no longer simply reading bytes from the <code class="docutils literal notranslate">/dev/urandom</code> device. This means that what used to be somebody else’s problem (the Linux kernel development team’s) is now Python’s problem – given a way to detect that the system RNG is not initialized, we have to choose how to handle this situation whenever we try to use the system RNG. It could simply block, as was somewhat inadvertently implemented in 3.5.0, and as is proposed in Victor Stinner’s competing PEP: <div class="highlight-default notranslate"><div class="highlight"><pre># artist's impression of the CPython 3.5.0-3.5.1 behavior def generate_unpredictable_bytes_or_block(num_bytes): while not system_rng_is_ready: wait return unpredictable_bytes(num_bytes) </pre></div> </div> Or it could raise an error, as this PEP proposes (in some cases): <div class="highlight-default notranslate"><div class="highlight"><pre># artist's impression of the behavior proposed in this PEP def generate_unpredictable_bytes_or_raise(num_bytes): if system_rng_is_ready: return unpredictable_bytes(num_bytes) else: raise BlockingIOError </pre></div> </div> Or it could explicitly emulate the <code class="docutils literal notranslate">/dev/urandom</code> fallback behavior, as was implemented in 3.5.2rc1 and is expected to remain for the rest of the 3.5.x cycle: <div class="highlight-default notranslate"><div class="highlight"><pre># artist's impression of the CPython 3.5.2rc1+ behavior def generate_unpredictable_bytes_or_maybe_not(num_bytes): if system_rng_is_ready: return unpredictable_bytes(num_bytes) else: return (b"p4ssw0rd" * (num_bytes // 8 + 1))[:num_bytes] </pre></div> </div> (And the same caveats apply to this sketch as applied to the <code class="docutils literal notranslate">generate_unpredictable_password</code> sketch of <code class="docutils literal notranslate">/dev/urandom</code> above.) There are five places where CPython and the standard library attempt to use the operating system’s random number generator, and thus five places where this decision has to be made: <ul class="simple"> <li>initializing the SipHash used to protect <code class="docutils literal notranslate">str.__hash__</code> and friends against DoS attacks (called unconditionally at startup)</li> <li>initializing the <code class="docutils literal notranslate">random</code> module (called when <code class="docutils literal notranslate">random</code> is imported)</li> <li>servicing user calls to the <code class="docutils literal notranslate">os.urandom</code> public API</li> <li>the higher level <code class="docutils literal notranslate">random.SystemRandom</code> public API</li> <li>the new <code class="docutils literal notranslate">secrets</code> module public API added by <a class="pep reference internal" href="../pep-0506/" title="PEP 506 – Adding A Secrets Module To The Standard Library">PEP 506</a></li> </ul> Previously, these five places all used the same underlying code, and thus made this decision in the same way. This whole problem was first noticed because 3.5.0 switched that underlying code to the <code class="docutils literal notranslate">generate_unpredictable_bytes_or_block</code> behavior, and it turns out that there are some rare cases where Linux boot scripts attempted to run a Python program as part of system initialization, the Python startup sequence blocked while trying to initialize SipHash, and then this triggered a deadlock because the system stopped doing anything – including gathering new entropy – until the Python script was forcibly terminated by an external timer. This is particularly unfortunate since the scripts in question never processed untrusted input, so there was no need for SipHash to be initialized with provably unpredictable random data in the first place. This motivated the change in 3.5.2rc1 to emulate the old <code class="docutils literal notranslate">/dev/urandom</code> behavior in all cases (by calling <code class="docutils literal notranslate">getrandom()</code> in non-blocking mode, and then falling back to reading <code class="docutils literal notranslate">/dev/urandom</code> if the syscall indicates that the <code class="docutils literal notranslate">/dev/urandom</code> pool is not yet fully initialized.) We don’t know whether such problems may also exist in the Fedora/RHEL/CentOS ecosystem, as the build systems for those distributions use chroots on servers running an older operating system kernel that doesn’t offer the <code class="docutils literal notranslate">getrandom()</code> syscall, which means CPython’s current build configuration compiles out the runtime check for that syscall <a class="footnote-reference brackets" href="#id21" id="id11">[10]</a>. A similar problem was found due to the <code class="docutils literal notranslate">random</code> module calling <code class="docutils literal notranslate">os.urandom</code> as a side-effect of import in order to seed the default global <code class="docutils literal notranslate">random.Random()</code> instance. We have not received any specific complaints regarding direct calls to <code class="docutils literal notranslate">os.urandom()</code> or <code class="docutils literal notranslate">random.SystemRandom()</code> blocking with 3.5.0 or 3.5.1 - only problem reports due to the implicit blocking on interpreter startup and as a side-effect of importing the random module. Independently of this PEP, the first two cases have already been updated to never block, regardless of the behaviour of <code class="docutils literal notranslate">os.urandom()</code>. Where <a class="pep reference internal" href="../pep-0524/" title="PEP 524 – Make os.urandom() blocking on Linux">PEP 524</a> proposes to make all 3 of the latter cases block implicitly, this PEP proposes that approach only for the last case (the <code class="docutils literal notranslate">secrets</code>) module, with <code class="docutils literal notranslate">os.urandom()</code> and <code class="docutils literal notranslate">random.SystemRandom()</code> instead raising an exception when they detect that the underlying operating system call would block. </section> </section> <section id="references"> <h2><a class="toc-backref" href="#references" role="doc-backlink">References</a></h2> <aside class="footnote-list brackets"> <aside class="footnote brackets" id="id12" role="doc-footnote"> <dt class="label" id="id12">[<a href="#id3">1</a>]</dt> <dd>os.urandom() should use Linux 3.17 getrandom() syscall (<a class="reference external" href="http://bugs.python.org/issue22181">http://bugs.python.org/issue22181</a>)</aside> <aside class="footnote brackets" id="id13" role="doc-footnote"> <dt class="label" id="id13">[<a href="#id4">2</a>]</dt> <dd>Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom() (<a class="reference external" href="http://bugs.python.org/issue26839">http://bugs.python.org/issue26839</a>)</aside> <aside class="footnote brackets" id="id14" role="doc-footnote"> <dt class="label" id="id14">[<a href="#id5">3</a>]</dt> <dd>“import random” blocks on entropy collection on Linux with low entropy (<a class="reference external" href="http://bugs.python.org/issue25420">http://bugs.python.org/issue25420</a>)</aside> <aside class="footnote brackets" id="id15" role="doc-footnote"> <dt class="label" id="id15">[<a href="#id6">4</a>]</dt> <dd>os.urandom() doesn’t block on Linux anymore (<a class="reference external" href="https://hg.python.org/cpython/rev/9de508dc4837">https://hg.python.org/cpython/rev/9de508dc4837</a>)</aside> <aside class="footnote brackets" id="id16" role="doc-footnote"> <dt class="label" id="id16">[<a href="#id7">5</a>]</dt> <dd>Proposal to add os.getrandom() (<a class="reference external" href="http://bugs.python.org/issue26839#msg267803">http://bugs.python.org/issue26839#msg267803</a>)</aside> <aside class="footnote brackets" id="id17" role="doc-footnote"> <dt class="label" id="id17">[<a href="#id8">6</a>]</dt> <dd>Add os.urandom_block() (<a class="reference external" href="http://bugs.python.org/issue27250">http://bugs.python.org/issue27250</a>)</aside> <aside class="footnote brackets" id="id18" role="doc-footnote"> <dt class="label" id="id18">[<a href="#id9">7</a>]</dt> <dd>Add random.cryptorandom() and random.pseudorandom, deprecate os.urandom() (<a class="reference external" href="http://bugs.python.org/issue27279">http://bugs.python.org/issue27279</a>)</aside> <aside class="footnote brackets" id="id19" role="doc-footnote"> <dt class="label" id="id19">[<a href="#id10">8</a>]</dt> <dd>Always use getrandom() in os.random() on Linux and add block=False parameter to os.urandom() (<a class="reference external" href="http://bugs.python.org/issue27266">http://bugs.python.org/issue27266</a>)</aside> <aside class="footnote brackets" id="id20" role="doc-footnote"> <dt class="label" id="id20">[<a href="#id2">9</a>]</dt> <dd>Application level vs library level design decisions (<a class="reference external" href="https://mail.python.org/pipermail/security-sig/2016-June/000057.html">https://mail.python.org/pipermail/security-sig/2016-June/000057.html</a>)</aside> <aside class="footnote brackets" id="id21" role="doc-footnote"> <dt class="label" id="id21">[<a href="#id11">10</a>]</dt> <dd>Does the HAVE_GETRANDOM_SYSCALL config setting make sense? (<a class="reference external" href="https://mail.python.org/pipermail/security-sig/2016-June/000060.html">https://mail.python.org/pipermail/security-sig/2016-June/000060.html</a>)</aside> <aside class="footnote brackets" id="id22" role="doc-footnote"> <dt class="label" id="id22">[<a href="#id1">11</a>]</dt> <dd>Take a decision for os.urandom() in Python 3.6 (<a class="reference external" href="https://mail.python.org/pipermail/security-sig/2016-August/000084.htm">https://mail.python.org/pipermail/security-sig/2016-August/000084.htm</a>)</aside> </aside> For additional background details beyond those captured in this PEP and Victor’s competing PEP, also see Victor’s prior collection of relevant information and links at <a class="reference external" href="https://haypo-notes.readthedocs.io/summary_python_random_issue.html">https://haypo-notes.readthedocs.io/summary_python_random_issue.html</a> </section> <section id="copyright"> <h2><a class="toc-backref" href="#copyright" role="doc-backlink">Copyright</a></h2> This document has been placed into the public domain. </section> </section> <hr class="docutils" /> Source: <a class="reference external" href="https://github.com/python/peps/blob/main/peps/pep-0522.rst">https://github.com/python/peps/blob/main/peps/pep-0522.rst</a> Last modified: <a class="reference external" href="https://github.com/python/peps/commits/main/peps/pep-0522.rst">2025-02-01 08:59:27 GMT</a> </article> <nav id="pep-sidebar"> <h2>Contents</h2> <ul> <li><a class="reference internal" href="#abstract">Abstract</a></li> <li><a class="reference internal" href="#relationship-with-other-peps">Relationship with other PEPs</a></li> <li><a class="reference internal" href="#pep-rejection">PEP Rejection</a></li> <li><a class="reference internal" href="#changes-independent-of-this-pep">Changes independent of this PEP</a></li> <li><a class="reference internal" href="#proposal">Proposal</a><ul> <li><a class="reference internal" href="#changing-os-urandom-on-platforms-with-the-getrandom-system-call">Changing <code class="docutils literal notranslate">os.urandom()</code> on platforms with the getrandom() system call</a></li> <li><a class="reference internal" href="#adding-secrets-wait-for-system-rng">Adding <code class="docutils literal notranslate">secrets.wait_for_system_rng()</code></a></li> <li><a class="reference internal" href="#limitations-on-scope">Limitations on scope</a></li> </ul> </li> <li><a class="reference internal" href="#rationale">Rationale</a><ul> <li><a class="reference internal" href="#ensuring-the-secrets-module-implicitly-blocks-when-needed">Ensuring the <code class="docutils literal notranslate">secrets</code> module implicitly blocks when needed</a></li> <li><a class="reference internal" href="#raising-blockingioerror-in-os-urandom-on-linux">Raising <code class="docutils literal notranslate">BlockingIOError</code> in <code class="docutils literal notranslate">os.urandom()</code> on Linux</a></li> <li><a class="reference internal" href="#making-secrets-wait-for-system-rng-public">Making <code class="docutils literal notranslate">secrets.wait_for_system_rng()</code> public</a></li> </ul> </li> <li><a class="reference internal" href="#backwards-compatibility-impact-assessment">Backwards Compatibility Impact Assessment</a><ul> <li><a class="reference internal" href="#unaffected-applications">Unaffected Applications</a></li> <li><a class="reference internal" href="#affected-security-sensitive-applications">Affected security sensitive applications</a></li> <li><a class="reference internal" href="#affected-non-security-sensitive-applications">Affected non-security sensitive applications</a></li> </ul> </li> <li><a class="reference internal" href="#additional-background">Additional Background</a><ul> <li><a class="reference internal" href="#why-propose-this-now">Why propose this now?</a></li> <li><a class="reference internal" href="#the-cross-platform-behaviour-of-os-urandom">The cross-platform behaviour of <code class="docutils literal notranslate">os.urandom()</code></a></li> <li><a class="reference internal" href="#problems-with-the-behaviour-of-dev-urandom-on-linux">Problems with the behaviour of <code class="docutils literal notranslate">/dev/urandom</code> on Linux</a></li> <li><a class="reference internal" href="#consequences-of-getrandom-availability-for-python">Consequences of <code class="docutils literal notranslate">getrandom()</code> availability for Python</a></li> </ul> </li> <li><a class="reference internal" href="#references">References</a></li> <li><a class="reference internal" href="#copyright">Copyright</a></li> </ul> <a id="source" href="https://github.com/python/peps/blob/main/peps/pep-0522.rst">Page Source (GitHub)</a> </nav> </section> <script src="../_static/colour_scheme.js"></script> <script src="../_static/wrap_tables.js"></script> <script src="../_static/sticky_banner.js"></script> </body> </html>