CINXE.COM

<!DOCTYPE html> <html lang="en"> <head> <meta charset="utf-8"> <meta name="viewport" content="width=device-width, initial-scale=1.0"> <meta name="color-scheme" content="light dark"> <title>PEP 466 – Network Security Enhancements for Python 2.7.x | peps.python.org</title> <link rel="shortcut icon" href="../_static/py.png"> <link rel="canonical" href="https://peps.python.org/pep-0466/"> <link rel="stylesheet" href="../_static/style.css" type="text/css"> <link rel="stylesheet" href="../_static/mq.css" type="text/css"> <link rel="stylesheet" href="../_static/pygments.css" type="text/css" media="(prefers-color-scheme: light)" id="pyg-light"> <link rel="stylesheet" href="../_static/pygments_dark.css" type="text/css" media="(prefers-color-scheme: dark)" id="pyg-dark"> <link rel="alternate" type="application/rss+xml" title="Latest PEPs" href="https://peps.python.org/peps.rss"> <meta property="og:title" content='PEP 466 – Network Security Enhancements for Python 2.7.x | peps.python.org'> <meta property="og:description" content="Most CPython tracker issues are classified as errors in behaviour or proposed enhancements. Most patches to fix behavioural errors are applied to all active maintenance branches. Enhancement patches are restricted to the default branch that becomes the..."> <meta property="og:type" content="website"> <meta property="og:url" content="https://peps.python.org/pep-0466/"> <meta property="og:site_name" content="Python Enhancement Proposals (PEPs)"> <meta property="og:image" content="https://peps.python.org/_static/og-image.png"> <meta property="og:image:alt" content="Python PEPs"> <meta property="og:image:width" content="200"> <meta property="og:image:height" content="200"> <meta name="description" content="Most CPython tracker issues are classified as errors in behaviour or proposed enhancements. Most patches to fix behavioural errors are applied to all active maintenance branches. Enhancement patches are restricted to the default branch that becomes the..."> <meta name="theme-color" content="#3776ab"> </head> <body> <svg xmlns="http://www.w3.org/2000/svg" style="display: none;"> <symbol id="svg-sun-half" viewBox="0 0 24 24" pointer-events="all"> <title>Following system colour scheme</title> <svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"> <circle cx="12" cy="12" r="9"></circle> <path d="M12 3v18m0-12l4.65-4.65M12 14.3l7.37-7.37M12 19.6l8.85-8.85"></path> </svg> </symbol> <symbol id="svg-moon" viewBox="0 0 24 24" pointer-events="all"> <title>Selected dark colour scheme</title> <svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"> <path stroke="none" d="M0 0h24v24H0z" fill="none"></path> <path d="M12 3c.132 0 .263 0 .393 0a7.5 7.5 0 0 0 7.92 12.446a9 9 0 1 1 -8.313 -12.454z"></path> </svg> </symbol> <symbol id="svg-sun" viewBox="0 0 24 24" pointer-events="all"> <title>Selected light colour scheme</title> <svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"> <circle cx="12" cy="12" r="5"></circle> <line x1="12" y1="1" x2="12" y2="3"></line> <line x1="12" y1="21" x2="12" y2="23"></line> <line x1="4.22" y1="4.22" x2="5.64" y2="5.64"></line> <line x1="18.36" y1="18.36" x2="19.78" y2="19.78"></line> <line x1="1" y1="12" x2="3" y2="12"></line> <line x1="21" y1="12" x2="23" y2="12"></line> <line x1="4.22" y1="19.78" x2="5.64" y2="18.36"></line> <line x1="18.36" y1="5.64" x2="19.78" y2="4.22"></line> </svg> </symbol> </svg> <script> document.documentElement.dataset.colour_scheme = localStorage.getItem("colour_scheme") || "auto" </script> <section id="pep-page-section"> <header> <h1>Python Enhancement Proposals</h1> <ul class="breadcrumbs"> <li><a href="https://www.python.org/" title="The Python Programming Language">Python</a> » </li> <li><a href="../pep-0000/">PEP Index</a> » </li> <li>PEP 466</li> </ul> <button id="colour-scheme-cycler" onClick="setColourScheme(nextColourScheme())"> <svg aria-hidden="true" class="colour-scheme-icon-when-auto"><use href="#svg-sun-half"></use></svg> <svg aria-hidden="true" class="colour-scheme-icon-when-dark"><use href="#svg-moon"></use></svg> <svg aria-hidden="true" class="colour-scheme-icon-when-light"><use href="#svg-sun"></use></svg> <span class="visually-hidden">Toggle light / dark / auto colour theme</span> </button> </header> <article> <section id="pep-content"> <h1 class="page-title">PEP 466 – Network Security Enhancements for Python 2.7.x</h1> <dl class="rfc2822 field-list simple"> <dt class="field-odd">Author<span class="colon">:</span></dt> <dd class="field-odd">Alyssa Coghlan <ncoghlan at gmail.com></dd> <dt class="field-even">Status<span class="colon">:</span></dt> <dd class="field-even"><abbr title="Accepted and implementation complete, or no longer active">Final</abbr></dd> <dt class="field-odd">Type<span class="colon">:</span></dt> <dd class="field-odd"><abbr title="Normative PEP with a new feature for Python, implementation change for CPython or interoperability standard for the ecosystem">Standards Track</abbr></dd> <dt class="field-even">Created<span class="colon">:</span></dt> <dd class="field-even">23-Mar-2014</dd> <dt class="field-odd">Python-Version<span class="colon">:</span></dt> <dd class="field-odd">2.7.9</dd> <dt class="field-even">Post-History<span class="colon">:</span></dt> <dd class="field-even">23-Mar-2014, 24-Mar-2014, 25-Mar-2014, 26-Mar-2014, 16-Apr-2014</dd> <dt class="field-odd">Resolution<span class="colon">:</span></dt> <dd class="field-odd"><a class="reference external" href="https://mail.python.org/pipermail/python-dev/2014-April/134163.html">Python-Dev message</a></dd> </dl> <hr class="docutils" /> <section id="contents"> <details><summary>Table of Contents</summary><ul class="simple"> <li><a class="reference internal" href="#abstract">Abstract</a></li> <li><a class="reference internal" href="#new-security-related-features-in-python-2-7-maintenance-releases">New security related features in Python 2.7 maintenance releases</a></li> <li><a class="reference internal" href="#implementation-status">Implementation status</a></li> <li><a class="reference internal" href="#backwards-compatibility-considerations">Backwards compatibility considerations</a><ul> <li><a class="reference internal" href="#openssl-compatibility">OpenSSL compatibility</a></li> </ul> </li> <li><a class="reference internal" href="#other-considerations">Other Considerations</a><ul> <li><a class="reference internal" href="#maintainability">Maintainability</a></li> <li><a class="reference internal" href="#security-releases">Security releases</a></li> <li><a class="reference internal" href="#integration-testing">Integration testing</a></li> <li><a class="reference internal" href="#handling-lower-security-environments-with-low-risk-tolerance">Handling lower security environments with low risk tolerance</a></li> </ul> </li> <li><a class="reference internal" href="#motivation-and-rationale">Motivation and Rationale</a><ul> <li><a class="reference internal" href="#why-these-particular-changes">Why these particular changes?</a></li> <li><a class="reference internal" href="#rejected-alternative-just-advise-developers-to-migrate-to-python-3">Rejected alternative: just advise developers to migrate to Python 3</a></li> <li><a class="reference internal" href="#rejected-alternative-create-and-release-python-2-8">Rejected alternative: create and release Python 2.8</a></li> <li><a class="reference internal" href="#rejected-alternative-distribute-the-security-enhancements-via-pypi">Rejected alternative: distribute the security enhancements via PyPI</a></li> <li><a class="reference internal" href="#rejected-variant-provide-a-legacy-ssl-infrastructure-branch">Rejected variant: provide a “legacy SSL infrastructure” branch</a></li> <li><a class="reference internal" href="#rejected-variant-synchronise-particular-modules-entirely-with-python-3">Rejected variant: synchronise particular modules entirely with Python 3</a></li> <li><a class="reference internal" href="#rejected-variant-open-ended-backport-policy">Rejected variant: open ended backport policy</a></li> </ul> </li> <li><a class="reference internal" href="#disclosure-of-interest">Disclosure of Interest</a></li> <li><a class="reference internal" href="#acknowledgements">Acknowledgements</a></li> <li><a class="reference internal" href="#references">References</a></li> <li><a class="reference internal" href="#copyright">Copyright</a></li> </ul> </details></section> <section id="abstract"> <h2><a class="toc-backref" href="#abstract" role="doc-backlink">Abstract</a></h2> <p>Most CPython tracker issues are classified as errors in behaviour or proposed enhancements. Most patches to fix behavioural errors are applied to all active maintenance branches. Enhancement patches are restricted to the default branch that becomes the next Python version.</p> <p>This cadence works reasonably well during Python’s normal 18-24 month feature release cycle, which is still applicable to the Python 3 series. However, the age of the standard library in Python 2 has now reached a point where it is sufficiently far behind the state of the art in network security protocols for it to be causing real problems in use cases where upgrading to Python 3 in the near term may not be feasible.</p> <p>In recognition of the additional practical considerations that have arisen during the 4+ year maintenance cycle for Python 2.7, this PEP allows a critical set of network security related features to be backported from Python 3.4 to upcoming Python 2.7.x maintenance releases.</p> <p>While this PEP does not make any changes to the core development team’s handling of security-fix-only branches that are no longer in active maintenance, it <em>does</em> recommend that commercial redistributors providing extended support periods for the Python standard library either backport these features to their supported versions, or else explicitly disclaim support for the use of older versions in roles that involve connecting directly to the public internet.</p> </section> <section id="new-security-related-features-in-python-2-7-maintenance-releases"> <h2><a class="toc-backref" href="#new-security-related-features-in-python-2-7-maintenance-releases" role="doc-backlink">New security related features in Python 2.7 maintenance releases</a></h2> <p>Under this proposal, the following features will be backported from Python 3.4 to upcoming Python 2.7.x maintenance releases:</p> <ul class="simple"> <li>in the <code class="docutils literal notranslate"><span class="pre">os</span></code> module:<ul> <li>persistent file descriptor for <code class="docutils literal notranslate"><span class="pre">os.urandom()</span></code>.</li> </ul> </li> <li>in the <code class="docutils literal notranslate"><span class="pre">hmac</span></code> module:<ul> <li>constant time comparison function (<code class="docutils literal notranslate"><span class="pre">hmac.compare_digest()</span></code>).</li> </ul> </li> <li>in the <code class="docutils literal notranslate"><span class="pre">hashlib</span></code> module:<ul> <li>password hashing function (<code class="docutils literal notranslate"><span class="pre">hashlib.pbkdf2_hmac()</span></code>).</li> <li>details of hash algorithm availability (<code class="docutils literal notranslate"><span class="pre">hashlib.algorithms_guaranteed</span></code> and <code class="docutils literal notranslate"><span class="pre">hashlib.algorithms_available</span></code>).</li> </ul> </li> <li>in the <code class="docutils literal notranslate"><span class="pre">ssl</span></code> module:<ul> <li>this module is almost entirely synchronised with its Python 3 counterpart, bringing TLSv1.x settings, SSLContext manipulation, Server Name Indication, access to platform certificate stores, standard library support for peer hostname validation and more to the Python 2 series.</li> <li>the only <code class="docutils literal notranslate"><span class="pre">ssl</span></code> module features <em>not</em> backported under this policy are the <code class="docutils literal notranslate"><span class="pre">ssl.RAND_*</span></code> functions that provide access to OpenSSL’s random number generation capabilities - use <code class="docutils literal notranslate"><span class="pre">os.urandom()</span></code> instead.</li> </ul> </li> </ul> <p>As a general change in maintenance policy, permission is also granted to upgrade to newer feature releases of OpenSSL when preparing the binary installers for new maintenance releases of Python 2.7.</p> <p>This PEP does NOT propose a general exception for backporting new features to Python 2.7 - every new feature proposed for backporting will still need to be justified independently. In particular, it will need to be explained why relying on an independently updated backport on the Python Package Index instead is not an acceptable solution.</p> </section> <section id="implementation-status"> <h2><a class="toc-backref" href="#implementation-status" role="doc-backlink">Implementation status</a></h2> <p>This PEP originally proposed adding all listed features to the Python 2.7.7 maintenance release. That approach proved to be too ambitious given the limited time frame between the original creation and acceptance of the PEP and the release of Python 2.7.7rc1. Instead, the progress of each individual accepted feature backport is being tracked as an independent enhancement targeting Python 2.7.</p> <p>Implemented for Python 2.7.7:</p> <ul class="simple"> <li><a class="reference external" href="http://bugs.python.org/issue21306">Issue #21306</a>: backport <code class="docutils literal notranslate"><span class="pre">hmac.compare_digest</span></code></li> <li><a class="reference external" href="http://bugs.python.org/issue21462">Issue #21462</a>: upgrade OpenSSL in the Python 2.7 Windows installers</li> </ul> <p>Implemented for Python 2.7.8:</p> <ul class="simple"> <li><a class="reference external" href="http://bugs.python.org/issue21304">Issue #21304</a>: backport <code class="docutils literal notranslate"><span class="pre">hashlib.pbkdf2</span></code></li> </ul> <p>Implemented for Python 2.7.9 (in development):</p> <ul class="simple"> <li><a class="reference external" href="http://bugs.python.org/issue21308">Issue #21308</a>: backport specified <code class="docutils literal notranslate"><span class="pre">ssl</span></code> module features</li> <li><a class="reference external" href="http://bugs.python.org/issue21307">Issue #21307</a>: backport remaining specified <code class="docutils literal notranslate"><span class="pre">hashlib</span></code> module features</li> <li><a class="reference external" href="http://bugs.python.org/issue21305">Issue #21305</a>: backport <code class="docutils literal notranslate"><span class="pre">os.urandom</span></code> shared file descriptor change</li> </ul> </section> <section id="backwards-compatibility-considerations"> <h2><a class="toc-backref" href="#backwards-compatibility-considerations" role="doc-backlink">Backwards compatibility considerations</a></h2> <p>As in the Python 3 series, the backported <code class="docutils literal notranslate"><span class="pre">ssl.create_default_context()</span></code> API is granted a backwards compatibility exemption that permits the protocol, options, cipher and other settings of the created SSL context to be updated in maintenance releases to use higher default security settings. This allows them to appropriately balance compatibility and security at the time of the maintenance release, rather than at the time of the original feature release.</p> <p>This PEP does <em>not</em> grant any other exemptions to the usual backwards compatibility policy for maintenance releases. Instead, by explicitly encouraging the use of feature based checks, it is designed to make it easier to write more secure cross-version compatible Python software, while still limiting the risk of breaking currently working software when upgrading to a new Python 2.7 maintenance release.</p> <p>In all cases where this proposal allows new features to be backported to the Python 2.7 release series, it is possible to write cross-version compatible code that operates by “feature detection” (for example, checking for particular attributes in a module), without needing to explicitly check the Python version.</p> <p>It is then up to library and framework code to provide an appropriate warning and fallback behaviour if a desired feature is found to be missing. While some especially security sensitive software MAY fail outright if a desired security feature is unavailable, most software SHOULD instead emit a warning and continue operating using a slightly degraded security configuration.</p> <p>The backported APIs allow library and application code to perform the following actions after detecting the presence of a relevant network security related feature:</p> <ul class="simple"> <li>explicitly opt in to more secure settings (to allow the use of enhanced security features in older maintenance releases of Python with less secure default behaviour)</li> <li>explicitly opt in to less secure settings (to allow the use of newer Python feature releases in lower security environments)</li> <li>determine the default setting for the feature (this MAY require explicit Python version checks to determine the Python feature release, but DOES NOT require checking for a specific maintenance release)</li> </ul> <p>Security related changes to other modules (such as higher level networking libraries and data format processing libraries) will continue to be made available as backports and new modules on the Python Package Index, as independent distribution remains the preferred approach to handling software that must continue to evolve to handle changing development requirements independently of the Python 2 standard library. Refer to the <a class="reference internal" href="#motivation-and-rationale">Motivation and Rationale</a> section for a review of the characteristics that make the secure networking infrastructure worthy of special consideration.</p> <section id="openssl-compatibility"> <h3><a class="toc-backref" href="#openssl-compatibility" role="doc-backlink">OpenSSL compatibility</a></h3> <p>Under this proposal, OpenSSL may be upgraded to more recent feature releases in Python 2.7 maintenance releases. On Linux and most other POSIX systems, the specific version of OpenSSL used already varies, as CPython dynamically links to the system provided OpenSSL library by default.</p> <p>For the Windows binary installers, the <code class="docutils literal notranslate"><span class="pre">_ssl</span></code> and <code class="docutils literal notranslate"><span class="pre">_hashlib</span></code> modules are statically linked with OpenSSL and the associated symbols are not exported. Marc-Andre Lemburg indicates that updating to newer OpenSSL releases in the <code class="docutils literal notranslate"><span class="pre">egenix-pyopenssl</span></code> binaries has not resulted in any reported compatibility issues <a class="footnote-reference brackets" href="#id11" id="id1">[3]</a></p> <p>The Mac OS X binary installers historically followed the same policy as other POSIX installations and dynamically linked to the Apple provided OpenSSL libraries. However, Apple has now ceased updating these cross-platform libraries, instead requiring that even cross-platform developers adopt Mac OS X specific interfaces to access up to date security infrastructure on their platform. Accordingly, and independently of this PEP, the Mac OS X binary installers were already going to be switched to statically linker newer versions of OpenSSL <a class="footnote-reference brackets" href="#id12" id="id2">[4]</a></p> </section> </section> <section id="other-considerations"> <h2><a class="toc-backref" href="#other-considerations" role="doc-backlink">Other Considerations</a></h2> <section id="maintainability"> <h3><a class="toc-backref" href="#maintainability" role="doc-backlink">Maintainability</a></h3> <p>A number of developers, including Alex Gaynor and Donald Stufft, have expressed interest in carrying out the feature backports covered by this policy, and assisting with any additional maintenance burdens that arise in the Python 2 series as a result.</p> <p>Steve Dower and Brian Curtin have offered to help with the creation of the Windows installers, allowing Martin von Löwis the opportunity to step back from the task of maintaining the 2.7 Windows installer.</p> <p>This PEP is primarily about establishing the consensus needed to allow them to carry out this work. For other core developers, this policy change shouldn’t impose any additional effort beyond potentially reviewing the resulting patches for those developers specifically interested in the affected modules.</p> </section> <section id="security-releases"> <h3><a class="toc-backref" href="#security-releases" role="doc-backlink">Security releases</a></h3> <p>This PEP does not propose any changes to the handling of security releases - those will continue to be source only releases that include only critical security fixes.</p> <p>However, the recommendations for library and application developers are deliberately designed to accommodate commercial redistributors that choose to apply these changes to additional Python release series that are either in security fix only mode, or have been declared “end of life” by the core development team.</p> <p>Whether or not redistributors choose to exercise that option will be up to the individual redistributor.</p> </section> <section id="integration-testing"> <h3><a class="toc-backref" href="#integration-testing" role="doc-backlink">Integration testing</a></h3> <p>Third party integration testing services should offer users the ability to test against multiple Python 2.7 maintenance releases (at least 2.7.6 and 2.7.7+), to ensure that libraries, frameworks and applications can still test their handling of the legacy security infrastructure correctly (either failing or degrading gracefully, depending on the security sensitivity of the software), even after the features covered in this proposal have been backported to the Python 2.7 series.</p> </section> <section id="handling-lower-security-environments-with-low-risk-tolerance"> <h3><a class="toc-backref" href="#handling-lower-security-environments-with-low-risk-tolerance" role="doc-backlink">Handling lower security environments with low risk tolerance</a></h3> <p>For better or for worse (mostly worse), there are some environments where the risk of latent security defects is more tolerated than even a slightly increased risk of regressions in maintenance releases. This proposal largely excludes these environments from consideration where the modules covered by the exemption are concerned - this approach is entirely inappropriate for software connected to the public internet, and defence in depth security principles suggest that it is not appropriate for most private networks either.</p> <p>Downstream redistributors may still choose to cater to such environments, but they will need to handle the process of downgrading the security related modules and doing the associated regression testing themselves. The main CPython continuous integration infrastructure will not cover this scenario.</p> </section> </section> <section id="motivation-and-rationale"> <h2><a class="toc-backref" href="#motivation-and-rationale" role="doc-backlink">Motivation and Rationale</a></h2> <p>The creation of this PEP was prompted primarily by the aging SSL support in the Python 2 series. As of March 2014, the Python 2.7 SSL module is approaching four years of age, and the SSL support in the still popular Python 2.6 release had its feature set locked six years ago.</p> <p>These are simply too old to provide a foundation that can be recommended in good conscience for secure networking software that operates over the public internet, especially in an era where it is becoming quite clearly evident that advanced persistent security threats are even more widespread and more indiscriminate in their targeting than had previously been understood. While they represented reasonable security infrastructure in their time, the state of the art has moved on, and we need to investigate mechanisms for effectively providing more up to date network security infrastructure for users that, for whatever reason, are not currently in a position to migrate to Python 3.</p> <p>While the use of the system OpenSSL installation addresses many of these concerns on Linux platforms, it doesn’t address all of them (in particular, it is still difficult for sotware to explicitly require some higher level security settings). The standard library support can be bypassed by using a third party library like PyOpenSSL or Pycurl, but this still results in a security problem, as these can be difficult dependencies to deploy, and many users will remain unaware that they might want them. Rather than explaining to potentially naive users how to obtain and use these libraries, it seems better to just fix the included batteries.</p> <p>In the case of the binary installers for Windows and Mac OS X that are published on python.org, the version of OpenSSL used is entirely within the control of the Python core development team, but is currently limited to OpenSSL maintenance releases for the version initially shipped with the corresponding Python feature release.</p> <p>With increased popularity comes increased responsibility, and this proposal aims to acknowledge the fact that Python’s popularity and adoption is at a sufficiently high level that some of our design and policy decisions have significant implications beyond the Python development community.</p> <p>As one example, the Python 2 <code class="docutils literal notranslate"><span class="pre">ssl</span></code> module does not support the Server Name Indication standard. While it is possible to obtain SNI support by using the third party <code class="docutils literal notranslate"><span class="pre">requests</span></code> client library, actually doing so currently requires using not only <code class="docutils literal notranslate"><span class="pre">requests</span></code> and its embedded dependencies, but also half a dozen or more additional libraries. The lack of support in the Python 2 series thus serves as an impediment to making effective use of SNI on servers, as Python 2 clients will frequently fail to handle it correctly.</p> <p>Another more critical example is the lack of SSL hostname matching in the Python 2 standard library - it is currently necessary to rely on a third party library, such as <code class="docutils literal notranslate"><span class="pre">requests</span></code> or <code class="docutils literal notranslate"><span class="pre">backports.ssl_match_hostname</span></code> to obtain that functionality in Python 2.</p> <p>The Python 2 series also remains more vulnerable to remote timing attacks on security sensitive comparisons than the Python 3 series, as it lacks a standard library equivalent to the timing attack resistant <code class="docutils literal notranslate"><span class="pre">hmac.compare_digest()</span></code> function. While appropriate secure comparison functions can be implemented in third party extensions, many users don’t even consider the issue and use ordinary equality comparisons instead - while a standard library solution doesn’t automatically fix that problem, it <em>does</em> make the barrier to resolution much lower once the problem is pointed out.</p> <p>Python 2.7 represents the only long term maintenance release the core development team has provided, and it is natural that there will be things that worked over a historically shorter maintenance lifespan that don’t work over this longer support period. In the specific case of the problem described in this PEP, the simplest available solution is to acknowledge that long term maintenance of network security related modules <em>requires</em> the ability to add new features, even while retaining backwards compatibility for existing interfaces.</p> <p>For those familiar with it, it is worth comparing the approach described in this PEP with Red Hat’s handling of its long term open source support commitments: it isn’t the RHEL 6.0 release itself that receives 10 years worth of support, but the overall RHEL 6 <em>series</em>. The individual RHEL 6.x point releases within the series then receive a wide variety of new features, including security enhancements, all while meeting strict backwards compatibility guarantees for existing software. The proposal covered in this PEP brings our approach to long term maintenance more into line with this precedent - we retain our strict backwards compatibility requirements, but make an exception to the restriction against adding new features.</p> <p>To date, downstream redistributors have respected our upstream policy of “no new features in Python maintenance releases”. This PEP explicitly accepts that a more nuanced policy is appropriate in the case of network security related features, and the specific change it describes is deliberately designed such that it is potentially suitable for Red Hat Enterprise Linux and its downstream derivatives.</p> <section id="why-these-particular-changes"> <h3><a class="toc-backref" href="#why-these-particular-changes" role="doc-backlink">Why these particular changes?</a></h3> <p>The key requirement for a feature to be considered for inclusion in this proposal was that it must have security implications <em>beyond</em> the specific application that is written in Python and the system that application is running on. Thus the focus on network security protocols, password storage and related cryptographic infrastructure - Python is a popular choice for the development of web services and clients, and thus the capabilities of widely used Python versions have implications for the security design of other services that may themselves be using newer versions of Python or other development languages, but need to interoperate with clients or servers written using older versions of Python.</p> <p>The intent behind this requirement was to minimise any impact that the introduction of this policy may have on the stability and compatibility of maintenance releases, while still addressing some key security concerns relating to the particular aspects of Python 2.7. It would be thoroughly counterproductive if end users became as cautious about updating to new Python 2.7 maintenance releases as they are about updating to new feature releases within the same release series.</p> <p>The <code class="docutils literal notranslate"><span class="pre">ssl</span></code> module changes are included in this proposal to bring the Python 2 series up to date with the past 4 years of evolution in network security standards, and make it easier for those standards to be broadly adopted in both servers and clients. Similarly the hash algorithm availability indicators in <code class="docutils literal notranslate"><span class="pre">hashlib</span></code> are included to make it easier for applications to detect and employ appropriate hash definitions across both Python 2 and 3.</p> <p>The <code class="docutils literal notranslate"><span class="pre">hmac.compare_digest()</span></code> and <code class="docutils literal notranslate"><span class="pre">hashlib.pbkdf2_hmac()</span></code> are included to help lower the barriers to secure password storage and checking in Python 2 server applications.</p> <p>The <code class="docutils literal notranslate"><span class="pre">os.urandom()</span></code> change has been included in this proposal to further encourage users to leave the task of providing high quality random numbers for cryptographic use cases to operating system vendors. The use of insufficiently random numbers has the potential to compromise <em>any</em> cryptographic system, and operating system developers have more tools available to address that problem adequately than the typical Python application runtime.</p> </section> <section id="rejected-alternative-just-advise-developers-to-migrate-to-python-3"> <h3><a class="toc-backref" href="#rejected-alternative-just-advise-developers-to-migrate-to-python-3" role="doc-backlink">Rejected alternative: just advise developers to migrate to Python 3</a></h3> <p>This alternative represents the status quo. Unfortunately, it has proven to be unworkable in practice, as the backwards compatibility implications mean that this is a non-trivial migration process for large applications and integration projects. While the tools for migration have evolved to a point where it is possible to migrate even large applications opportunistically and incrementally (rather than all at once) by updating code to run in the large common subset of Python 2 and Python 3, using the most recent technology often isn’t a priority in commercial environments.</p> <p>Previously, this was considered an acceptable harm, as while it was an unfortunate problem for the affected developers to have to face, it was seen as an issue between them and their management chain to make the case for infrastructure modernisation, and this case would become naturally more compelling as the Python 3 series evolved.</p> <p>However, now that we’re fully aware of the impact the limitations of the Python 2 standard library may be having on the evolution of internet security standards, I no longer believe that it is reasonable to expect platform and application developers to resolve all of the latent defects in an application’s Unicode correctness solely in order to gain access to the network security enhancements already available in Python 3.</p> <p>While Ubuntu (and to some extent Debian as well) are committed to porting all default system services and scripts to Python 3, and to removing Python 2 from its default distribution images (but not from its archives), this is a mammoth task and won’t be completed for the Ubuntu 14.04 LTS release (at least for the desktop image - it may be achieved for the mobile and server images).</p> <p>Fedora has even more work to do to migrate, and it will take a non-trivial amount of time to migrate the relevant infrastructure components. While Red Hat are also actively working to make it easier for users to use more recent versions of Python on our stable platforms, it’s going to take time for those efforts to start having an impact on end users’ choice of version, and any such changes also don’t benefit the core platform infrastructure that runs in the integrated system Python by necessity.</p> <p>The OpenStack migration to Python 3 is also still in its infancy, and even though that’s a project with an extensive and relatively robust automated test suite, it’s still large enough that it is going to take quite some time to migrate fully to a Python 2/3 compatible code base.</p> <p>And that’s just three of the highest profile open source projects that make heavy use of Python. Given the likely existence of large amounts of legacy code that lacks the kind of automated regression test suite needed to help support a migration from Python 2 to Python 3, there are likely to be many cases where reimplementation (perhaps even in Python 3) proves easier than migration. The key point of this PEP is that those situations affect more people than just the developers and users of the affected application: the existence of clients and servers with outdated network security infrastructure becomes something that developers of secure networked services need to take into account as part of their security design, and that’s a problem that inhibits the adoption of better security standards.</p> <p>As Terry Reedy noted, if we try to persist with the status quo, the likely outcome is that commercial redistributors will attempt to do something like this on behalf of their customers <em>anyway</em>, but in a potentially inconsistent and ad hoc manner. By drawing the scope definition process into the upstream project we are in a better position to influence the approach taken to address the situation and to help ensure some consistency across redistributors.</p> <p>The problem is real, so <em>something</em> needs to change, and this PEP describes my preferred approach to addressing the situation.</p> </section> <section id="rejected-alternative-create-and-release-python-2-8"> <h3><a class="toc-backref" href="#rejected-alternative-create-and-release-python-2-8" role="doc-backlink">Rejected alternative: create and release Python 2.8</a></h3> <p>With sufficient corporate support, it likely <em>would</em> be possible to create and release Python 2.8 (it’s highly unlikely such a project would garner enough interest to be achievable with only volunteers). However, this wouldn’t actually solve the problem, as the aim is to provide a <em>relatively low impact</em> way to incorporate enhanced security features into integrated products and deployments that make use of Python 2.</p> <p>Upgrading to a new Python feature release would mean both more work for the core development team, as well as a more disruptive update that most potential end users would likely just skip entirely.</p> <p>Attempting to create a Python 2.8 release would also bring in suggestions to backport many additional features from Python 3 (such as <code class="docutils literal notranslate"><span class="pre">tracemalloc</span></code> and the improved coroutine support), making the migration from Python 2.7 to this hypothetical 2.8 release even riskier and more disruptive.</p> <p>This is not a recommended approach, as it would involve substantial additional work for a result that is actually less effective in achieving the original aim (which is to eliminate the current widespread use of the aging network security infrastructure in the Python 2 series).</p> <p>Furthermore, while I can’t make any commitments to actually addressing this issue on Red Hat platforms, I <em>can</em> categorically rule out the idea of a Python 2.8 being of any use to me in even attempting to get it addressed.</p> </section> <section id="rejected-alternative-distribute-the-security-enhancements-via-pypi"> <h3><a class="toc-backref" href="#rejected-alternative-distribute-the-security-enhancements-via-pypi" role="doc-backlink">Rejected alternative: distribute the security enhancements via PyPI</a></h3> <p>While this initially appears to be an attractive and easier to manage approach, it actually suffers from several significant problems.</p> <p>Firstly, this is complex, low level, cross-platform code that integrates with the underlying operating system across a variety of POSIX platforms (including Mac OS X) and Windows. The CPython BuildBot fleet is already set up to handle continuous integration in that context, but most of the freely available continuous integration services just offer Linux, and perhaps paid access to Windows. Those services work reasonably well for software that largely runs on the abstraction layers offered by Python and other dynamic languages, as well as the more comprehensive abstraction offered by the JVM, but won’t suffice for the kind of code involved here.</p> <p>The OpenSSL dependency for the network security support also qualifies as the kind of “complex binary dependency” that isn’t yet handled well by the <code class="docutils literal notranslate"><span class="pre">pip</span></code> based software distribution ecosystem. Relying on a third party binary dependency also creates potential compatibility problems for <code class="docutils literal notranslate"><span class="pre">pip</span></code> when running on other interpreters like <code class="docutils literal notranslate"><span class="pre">PyPy</span></code>.</p> <p>Another practical problem with the idea is the fact that <code class="docutils literal notranslate"><span class="pre">pip</span></code> itself relies on the <code class="docutils literal notranslate"><span class="pre">ssl</span></code> support in the standard library (with some additional support from a bundled copy of <code class="docutils literal notranslate"><span class="pre">requests</span></code>, which in turn bundles <code class="docutils literal notranslate"><span class="pre">backport.ssl_match_hostname</span></code>), and hence would require any replacement module to also be bundled within <code class="docutils literal notranslate"><span class="pre">pip</span></code>. This wouldn’t pose any insurmountable difficulties (it’s just another dependency to vendor), but it <em>would</em> mean yet another copy of OpenSSL to keep up to date.</p> <p>This approach also has the same flaw as all other “improve security by renaming things” approaches: they completely miss the users who most need help, and raise significant barriers against being able to encourage users to do the right thing when their infrastructure supports it (since “use this other module” is a much higher impact change than “turn on this higher security setting”). Deprecating the aging SSL infrastructure in the standard library in favour of an external module would be even more user hostile than accepting the slightly increased risk of regressions associated with upgrading it in place.</p> <p>Last, but certainly not least, this approach suffers from the same problem as the idea of doing a Python 2.8 release: likely not solving the actual problem. Commercial redistributors of Python are set up to redistribute <em>Python</em>, and a pre-existing set of additional packages. Getting new packages added to the pre-existing set <em>can</em> be done, but means approaching each and every redistributor and asking them to update their repackaging process accordingly. By contrast, the approach described in this PEP would require redistributors to deliberately <em>opt out</em> of the security enhancements by deliberately downgrading the provided network security infrastructure, which most of them are unlikely to do.</p> </section> <section id="rejected-variant-provide-a-legacy-ssl-infrastructure-branch"> <h3><a class="toc-backref" href="#rejected-variant-provide-a-legacy-ssl-infrastructure-branch" role="doc-backlink">Rejected variant: provide a “legacy SSL infrastructure” branch</a></h3> <p>Earlier versions of this PEP included the concept of a <code class="docutils literal notranslate"><span class="pre">2.7-legacy-ssl</span></code> branch that preserved the exact feature set of the Python 2.7.6 network security infrastructure.</p> <p>In my opinion, anyone that actually wants this is almost certainly making a mistake, and if they insist they really do want it in their specific situation, they’re welcome to either make it themselves or arrange for a downstream redistributor to make it for them.</p> <p>If they are made publicly available, any such rebuilds should be referred to as “Python 2.7 with Legacy SSL” to clearly distinguish them from the official Python 2.7 releases that include more up to date network security infrastructure.</p> <p>After the first Python 2.7 maintenance release that implements this PEP, it would also be appropriate to refer to Python 2.7.6 and earlier releases as “Python 2.7 with Legacy SSL”.</p> </section> <section id="rejected-variant-synchronise-particular-modules-entirely-with-python-3"> <h3><a class="toc-backref" href="#rejected-variant-synchronise-particular-modules-entirely-with-python-3" role="doc-backlink">Rejected variant: synchronise particular modules entirely with Python 3</a></h3> <p>Earlier versions of this PEP suggested synchronising the <code class="docutils literal notranslate"><span class="pre">hmac</span></code>, <code class="docutils literal notranslate"><span class="pre">hashlib</span></code> and <code class="docutils literal notranslate"><span class="pre">ssl</span></code> modules entirely with their Python 3 counterparts.</p> <p>This approach proved too vague to build a compelling case for the exception, and has thus been replaced by the current more explicit proposal.</p> </section> <section id="rejected-variant-open-ended-backport-policy"> <h3><a class="toc-backref" href="#rejected-variant-open-ended-backport-policy" role="doc-backlink">Rejected variant: open ended backport policy</a></h3> <p>Earlier versions of this PEP suggested a general policy change related to future Python 3 enhancements that impact the general security of the internet.</p> <p>That approach created unnecessary uncertainty, so it has been simplified to propose backport a specific concrete set of changes. Future feature backport proposals can refer back to this PEP as precedent, but it will still be necessary to make a specific case for each feature addition to the Python 2.7 long-term support release.</p> </section> </section> <section id="disclosure-of-interest"> <h2><a class="toc-backref" href="#disclosure-of-interest" role="doc-backlink">Disclosure of Interest</a></h2> <p>The author of this PEP currently works for Red Hat on test automation tools. If this proposal is accepted, I will be strongly encouraging Red Hat to take advantage of the resulting opportunity to help improve the overall security of the Python ecosystem. However, I do not speak for Red Hat in this matter, and cannot make any commitments on Red Hat’s behalf.</p> </section> <section id="acknowledgements"> <h2><a class="toc-backref" href="#acknowledgements" role="doc-backlink">Acknowledgements</a></h2> <p>Thanks to Christian Heimes and other for their efforts in greatly improving Python’s SSL support in the Python 3 series, and a variety of members of the Python community for helping me to better understand the implications of the default settings we provide in our SSL modules, and the impact that tolerating the use of SSL infrastructure that was defined in 2010 (Python 2.7) or even 2008 (Python 2.6) potentially has for the security of the web as a whole.</p> <p>Thanks to Donald Stufft and Alex Gaynor for identifying a more limited set of essential security features that allowed the proposal to be made more fine-grained than backporting entire modules from Python 3.4 (<a class="footnote-reference brackets" href="#id15" id="id3">[7]</a>, <a class="footnote-reference brackets" href="#id16" id="id4">[8]</a>).</p> <p>Christian and Donald also provided valuable feedback on a preliminary draft of this proposal.</p> <p>Thanks also to participants in the python-dev mailing list threads (<a class="footnote-reference brackets" href="#id9" id="id5">[1]</a>, <a class="footnote-reference brackets" href="#id10" id="id6">[2]</a>, <a class="footnote-reference brackets" href="#id13" id="id7">[5]</a>, <a class="footnote-reference brackets" href="#id14" id="id8">[6]</a>), as well as the various folks I discussed this issue with at PyCon 2014 in Montreal.</p> </section> <section id="references"> <h2><a class="toc-backref" href="#references" role="doc-backlink">References</a></h2> <aside class="footnote-list brackets"> <aside class="footnote brackets" id="id9" role="doc-footnote"> <dt class="label" id="id9">[<a href="#id5">1</a>]</dt> <dd>PEP 466 discussion (round 1) (<a class="reference external" href="https://mail.python.org/pipermail/python-dev/2014-March/133334.html">https://mail.python.org/pipermail/python-dev/2014-March/133334.html</a>)</aside> <aside class="footnote brackets" id="id10" role="doc-footnote"> <dt class="label" id="id10">[<a href="#id6">2</a>]</dt> <dd>PEP 466 discussion (round 2) (<a class="reference external" href="https://mail.python.org/pipermail/python-dev/2014-March/133389.html">https://mail.python.org/pipermail/python-dev/2014-March/133389.html</a>)</aside> <aside class="footnote brackets" id="id11" role="doc-footnote"> <dt class="label" id="id11">[<a href="#id1">3</a>]</dt> <dd>Marc-Andre Lemburg’s OpenSSL feedback for Windows (<a class="reference external" href="https://mail.python.org/pipermail/python-dev/2014-March/133438.html">https://mail.python.org/pipermail/python-dev/2014-March/133438.html</a>)</aside> <aside class="footnote brackets" id="id12" role="doc-footnote"> <dt class="label" id="id12">[<a href="#id2">4</a>]</dt> <dd>Ned Deily’s OpenSSL feedback for Mac OS X (<a class="reference external" href="https://mail.python.org/pipermail/python-dev/2014-March/133347.html">https://mail.python.org/pipermail/python-dev/2014-March/133347.html</a>)</aside> <aside class="footnote brackets" id="id13" role="doc-footnote"> <dt class="label" id="id13">[<a href="#id7">5</a>]</dt> <dd>PEP 466 discussion (round 3) (<a class="reference external" href="https://mail.python.org/pipermail/python-dev/2014-March/133442.html">https://mail.python.org/pipermail/python-dev/2014-March/133442.html</a>)</aside> <aside class="footnote brackets" id="id14" role="doc-footnote"> <dt class="label" id="id14">[<a href="#id8">6</a>]</dt> <dd>PEP 466 discussion (round 4) (<a class="reference external" href="https://mail.python.org/pipermail/python-dev/2014-March/133472.html">https://mail.python.org/pipermail/python-dev/2014-March/133472.html</a>)</aside> <aside class="footnote brackets" id="id15" role="doc-footnote"> <dt class="label" id="id15">[<a href="#id3">7</a>]</dt> <dd>Donald Stufft’s recommended set of backported features (<a class="reference external" href="https://mail.python.org/pipermail/python-dev/2014-March/133500.html">https://mail.python.org/pipermail/python-dev/2014-March/133500.html</a>)</aside> <aside class="footnote brackets" id="id16" role="doc-footnote"> <dt class="label" id="id16">[<a href="#id4">8</a>]</dt> <dd>Alex Gaynor’s recommended set of backported features (<a class="reference external" href="https://mail.python.org/pipermail/python-dev/2014-March/133503.html">https://mail.python.org/pipermail/python-dev/2014-March/133503.html</a>)</aside> </aside> </section> <section id="copyright"> <h2><a class="toc-backref" href="#copyright" role="doc-backlink">Copyright</a></h2> <p>This document has been placed in the public domain.</p> </section> </section> <hr class="docutils" /> <p>Source: <a class="reference external" href="https://github.com/python/peps/blob/main/peps/pep-0466.rst">https://github.com/python/peps/blob/main/peps/pep-0466.rst</a></p> <p>Last modified: <a class="reference external" href="https://github.com/python/peps/commits/main/peps/pep-0466.rst">2023-10-11 12:05:51 GMT</a></p> </article> <nav id="pep-sidebar"> <h2>Contents</h2> <ul> <li><a class="reference internal" href="#abstract">Abstract</a></li> <li><a class="reference internal" href="#new-security-related-features-in-python-2-7-maintenance-releases">New security related features in Python 2.7 maintenance releases</a></li> <li><a class="reference internal" href="#implementation-status">Implementation status</a></li> <li><a class="reference internal" href="#backwards-compatibility-considerations">Backwards compatibility considerations</a><ul> <li><a class="reference internal" href="#openssl-compatibility">OpenSSL compatibility</a></li> </ul> </li> <li><a class="reference internal" href="#other-considerations">Other Considerations</a><ul> <li><a class="reference internal" href="#maintainability">Maintainability</a></li> <li><a class="reference internal" href="#security-releases">Security releases</a></li> <li><a class="reference internal" href="#integration-testing">Integration testing</a></li> <li><a class="reference internal" href="#handling-lower-security-environments-with-low-risk-tolerance">Handling lower security environments with low risk tolerance</a></li> </ul> </li> <li><a class="reference internal" href="#motivation-and-rationale">Motivation and Rationale</a><ul> <li><a class="reference internal" href="#why-these-particular-changes">Why these particular changes?</a></li> <li><a class="reference internal" href="#rejected-alternative-just-advise-developers-to-migrate-to-python-3">Rejected alternative: just advise developers to migrate to Python 3</a></li> <li><a class="reference internal" href="#rejected-alternative-create-and-release-python-2-8">Rejected alternative: create and release Python 2.8</a></li> <li><a class="reference internal" href="#rejected-alternative-distribute-the-security-enhancements-via-pypi">Rejected alternative: distribute the security enhancements via PyPI</a></li> <li><a class="reference internal" href="#rejected-variant-provide-a-legacy-ssl-infrastructure-branch">Rejected variant: provide a “legacy SSL infrastructure” branch</a></li> <li><a class="reference internal" href="#rejected-variant-synchronise-particular-modules-entirely-with-python-3">Rejected variant: synchronise particular modules entirely with Python 3</a></li> <li><a class="reference internal" href="#rejected-variant-open-ended-backport-policy">Rejected variant: open ended backport policy</a></li> </ul> </li> <li><a class="reference internal" href="#disclosure-of-interest">Disclosure of Interest</a></li> <li><a class="reference internal" href="#acknowledgements">Acknowledgements</a></li> <li><a class="reference internal" href="#references">References</a></li> <li><a class="reference internal" href="#copyright">Copyright</a></li> </ul> <br> <a id="source" href="https://github.com/python/peps/blob/main/peps/pep-0466.rst">Page Source (GitHub)</a> </nav> </section> <script src="../_static/colour_scheme.js"></script> <script src="../_static/wrap_tables.js"></script> <script src="../_static/sticky_banner.js"></script> </body> </html>