CINXE.COM

<!DOCTYPE html> <html lang="en"> <head> <meta charset="utf-8"> <meta name="viewport" content="width=device-width, initial-scale=1.0"> <meta name="color-scheme" content="light dark"> <title>PEP 3121 – Extension Module Initialization and Finalization | peps.python.org</title> <link rel="shortcut icon" href="../_static/py.png"> <link rel="canonical" href="https://peps.python.org/pep-3121/"> <link rel="stylesheet" href="../_static/style.css" type="text/css"> <link rel="stylesheet" href="../_static/mq.css" type="text/css"> <link rel="stylesheet" href="../_static/pygments.css" type="text/css" media="(prefers-color-scheme: light)" id="pyg-light"> <link rel="stylesheet" href="../_static/pygments_dark.css" type="text/css" media="(prefers-color-scheme: dark)" id="pyg-dark"> <link rel="alternate" type="application/rss+xml" title="Latest PEPs" href="https://peps.python.org/peps.rss"> <meta property="og:title" content='PEP 3121 – Extension Module Initialization and Finalization | peps.python.org'> <meta property="og:description" content="Extension module initialization currently has a few deficiencies. There is no cleanup for modules, the entry point name might give naming conflicts, the entry functions don’t follow the usual calling convention, and multiple interpreters are not support..."> <meta property="og:type" content="website"> <meta property="og:url" content="https://peps.python.org/pep-3121/"> <meta property="og:site_name" content="Python Enhancement Proposals (PEPs)"> <meta property="og:image" content="https://peps.python.org/_static/og-image.png"> <meta property="og:image:alt" content="Python PEPs"> <meta property="og:image:width" content="200"> <meta property="og:image:height" content="200"> <meta name="description" content="Extension module initialization currently has a few deficiencies. There is no cleanup for modules, the entry point name might give naming conflicts, the entry functions don’t follow the usual calling convention, and multiple interpreters are not support..."> <meta name="theme-color" content="#3776ab"> </head> <body> <svg xmlns="http://www.w3.org/2000/svg" style="display: none;"> <symbol id="svg-sun-half" viewBox="0 0 24 24" pointer-events="all"> <title>Following system colour scheme</title> <svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"> <circle cx="12" cy="12" r="9"></circle> <path d="M12 3v18m0-12l4.65-4.65M12 14.3l7.37-7.37M12 19.6l8.85-8.85"></path> </svg> </symbol> <symbol id="svg-moon" viewBox="0 0 24 24" pointer-events="all"> <title>Selected dark colour scheme</title> <svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"> <path stroke="none" d="M0 0h24v24H0z" fill="none"></path> <path d="M12 3c.132 0 .263 0 .393 0a7.5 7.5 0 0 0 7.92 12.446a9 9 0 1 1 -8.313 -12.454z"></path> </svg> </symbol> <symbol id="svg-sun" viewBox="0 0 24 24" pointer-events="all"> <title>Selected light colour scheme</title> <svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"> <circle cx="12" cy="12" r="5"></circle> <line x1="12" y1="1" x2="12" y2="3"></line> <line x1="12" y1="21" x2="12" y2="23"></line> <line x1="4.22" y1="4.22" x2="5.64" y2="5.64"></line> <line x1="18.36" y1="18.36" x2="19.78" y2="19.78"></line> <line x1="1" y1="12" x2="3" y2="12"></line> <line x1="21" y1="12" x2="23" y2="12"></line> <line x1="4.22" y1="19.78" x2="5.64" y2="18.36"></line> <line x1="18.36" y1="5.64" x2="19.78" y2="4.22"></line> </svg> </symbol> </svg> <script> document.documentElement.dataset.colour_scheme = localStorage.getItem("colour_scheme") || "auto" </script> <section id="pep-page-section"> <header> <h1>Python Enhancement Proposals</h1> <ul class="breadcrumbs"> <li><a href="https://www.python.org/" title="The Python Programming Language">Python</a> » </li> <li><a href="../pep-0000/">PEP Index</a> » </li> <li>PEP 3121</li> </ul> <button id="colour-scheme-cycler" onClick="setColourScheme(nextColourScheme())"> <svg aria-hidden="true" class="colour-scheme-icon-when-auto"><use href="#svg-sun-half"></use></svg> <svg aria-hidden="true" class="colour-scheme-icon-when-dark"><use href="#svg-moon"></use></svg> <svg aria-hidden="true" class="colour-scheme-icon-when-light"><use href="#svg-sun"></use></svg> <span class="visually-hidden">Toggle light / dark / auto colour theme</span> </button> </header> <article> <section id="pep-content"> <h1 class="page-title">PEP 3121 – Extension Module Initialization and Finalization</h1> <dl class="rfc2822 field-list simple"> <dt class="field-odd">Author<span class="colon">:</span></dt> <dd class="field-odd">Martin von Löwis <martin at v.loewis.de></dd> <dt class="field-even">Status<span class="colon">:</span></dt> <dd class="field-even"><abbr title="Accepted and implementation complete, or no longer active">Final</abbr></dd> <dt class="field-odd">Type<span class="colon">:</span></dt> <dd class="field-odd"><abbr title="Normative PEP with a new feature for Python, implementation change for CPython or interoperability standard for the ecosystem">Standards Track</abbr></dd> <dt class="field-even">Created<span class="colon">:</span></dt> <dd class="field-even">27-Apr-2007</dd> <dt class="field-odd">Python-Version<span class="colon">:</span></dt> <dd class="field-odd">3.0</dd> <dt class="field-even">Post-History<span class="colon">:</span></dt> <dd class="field-even"><p></p></dd> </dl> <hr class="docutils" /> <section id="contents"> <details><summary>Table of Contents</summary><ul class="simple"> <li><a class="reference internal" href="#abstract">Abstract</a></li> <li><a class="reference internal" href="#problems">Problems</a><ul> <li><a class="reference internal" href="#module-finalization">Module Finalization</a></li> <li><a class="reference internal" href="#entry-point-name-conflicts">Entry point name conflicts</a></li> <li><a class="reference internal" href="#entry-point-signature">Entry point signature</a></li> <li><a class="reference internal" href="#multiple-interpreters">Multiple Interpreters</a></li> </ul> </li> <li><a class="reference internal" href="#specification">Specification</a></li> <li><a class="reference internal" href="#example">Example</a></li> <li><a class="reference internal" href="#discussion">Discussion</a></li> <li><a class="reference internal" href="#references">References</a></li> <li><a class="reference internal" href="#copyright">Copyright</a></li> </ul> </details></section> <div class="pep-banner canonical-doc sticky-banner admonition important"> <p class="admonition-title">Important</p> <p>This PEP is a historical document. The up-to-date, canonical documentation can now be found at <a class="reference external" href="https://docs.python.org/3/extending/building.html#c.PyInit_modulename" title="(in Python v3.13)"><code class="xref c c-func docutils literal notranslate"><span class="pre">PyInit_modulename()</span></code></a> and <a class="reference external" href="https://docs.python.org/3/c-api/module.html#c.PyModuleDef" title="(in Python v3.13)"><code class="xref c c-type docutils literal notranslate"><span class="pre">PyModuleDef</span></code></a>.</p> <p class="close-button">×</p> <p>See <a class="pep reference internal" href="../pep-0001/" title="PEP 1 – PEP Purpose and Guidelines">PEP 1</a> for how to propose changes.</p> </div> <section id="abstract"> <h2><a class="toc-backref" href="#abstract" role="doc-backlink">Abstract</a></h2> <p>Extension module initialization currently has a few deficiencies. There is no cleanup for modules, the entry point name might give naming conflicts, the entry functions don’t follow the usual calling convention, and multiple interpreters are not supported well. This PEP addresses these issues.</p> </section> <section id="problems"> <h2><a class="toc-backref" href="#problems" role="doc-backlink">Problems</a></h2> <section id="module-finalization"> <h3><a class="toc-backref" href="#module-finalization" role="doc-backlink">Module Finalization</a></h3> <p>Currently, extension modules are initialized usually once and then “live” forever. The only exception is when Py_Finalize() is called: then the initialization routine is invoked a second time. This is bad from a resource management point of view: memory and other resources might get allocated each time initialization is called, but there is no way to reclaim them. As a result, there is currently no way to completely release all resources Python has allocated.</p> </section> <section id="entry-point-name-conflicts"> <h3><a class="toc-backref" href="#entry-point-name-conflicts" role="doc-backlink">Entry point name conflicts</a></h3> <p>The entry point is currently called init<module>. This might conflict with other symbols also called init<something>. In particular, initsocket is known to have conflicted in the past (this specific problem got resolved as a side effect of renaming the module to _socket).</p> </section> <section id="entry-point-signature"> <h3><a class="toc-backref" href="#entry-point-signature" role="doc-backlink">Entry point signature</a></h3> <p>The entry point is currently a procedure (returning void). This deviates from the usual calling conventions; callers can find out whether there was an error during initialization only by checking PyErr_Occurred. The entry point should return a PyObject*, which will be the module created, or NULL in case of an exception.</p> </section> <section id="multiple-interpreters"> <h3><a class="toc-backref" href="#multiple-interpreters" role="doc-backlink">Multiple Interpreters</a></h3> <p>Currently, extension modules share their state across all interpreters. This allows for undesirable information leakage across interpreters: one script could permanently corrupt objects in an extension module, possibly breaking all scripts in other interpreters.</p> </section> </section> <section id="specification"> <h2><a class="toc-backref" href="#specification" role="doc-backlink">Specification</a></h2> <p>The module initialization routines change their signature to:</p> <div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">PyObject</span> <span class="o">*</span><span class="n">PyInit_</span><span class="o"><</span><span class="n">modulename</span><span class="o">></span><span class="p">()</span> </pre></div> </div> <p>The initialization routine will be invoked once per interpreter, when the module is imported. It should return a new module object each time.</p> <p>In order to store per-module state in C variables, each module object will contain a block of memory that is interpreted only by the module. The amount of memory used for the module is specified at the point of creation of the module.</p> <p>In addition to the initialization function, a module may implement a number of additional callback functions, which are invoked when the module’s tp_traverse, tp_clear, and tp_free functions are invoked, and when the module is reloaded.</p> <p>The entire module definition is combined in a struct PyModuleDef:</p> <div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">struct</span> <span class="n">PyModuleDef</span><span class="p">{</span> <span class="n">PyModuleDef_Base</span> <span class="n">m_base</span><span class="p">;</span> <span class="o">/*</span> <span class="n">To</span> <span class="n">be</span> <span class="n">filled</span> <span class="n">out</span> <span class="n">by</span> <span class="n">the</span> <span class="n">interpreter</span> <span class="o">*/</span> <span class="n">Py_ssize_t</span> <span class="n">m_size</span><span class="p">;</span> <span class="o">/*</span> <span class="n">Size</span> <span class="n">of</span> <span class="n">per</span><span class="o">-</span><span class="n">module</span> <span class="n">data</span> <span class="o">*/</span> <span class="n">PyMethodDef</span> <span class="o">*</span><span class="n">m_methods</span><span class="p">;</span> <span class="n">inquiry</span> <span class="n">m_reload</span><span class="p">;</span> <span class="n">traverseproc</span> <span class="n">m_traverse</span><span class="p">;</span> <span class="n">inquiry</span> <span class="n">m_clear</span><span class="p">;</span> <span class="n">freefunc</span> <span class="n">m_free</span><span class="p">;</span> <span class="p">};</span> </pre></div> </div> <p>Creation of a module is changed to expect an optional PyModuleDef*. The module state will be null-initialized.</p> <p>Each module method will be passed the module object as the first parameter. To access the module data, a function:</p> <div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">void</span><span class="o">*</span> <span class="n">PyModule_GetState</span><span class="p">(</span><span class="n">PyObject</span><span class="o">*</span><span class="p">);</span> </pre></div> </div> <p>will be provided. In addition, to lookup a module more efficiently than going through sys.modules, a function:</p> <div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">PyObject</span><span class="o">*</span> <span class="n">PyState_FindModule</span><span class="p">(</span><span class="n">struct</span> <span class="n">PyModuleDef</span><span class="o">*</span><span class="p">);</span> </pre></div> </div> <p>will be provided. This lookup function will use an index located in the m_base field, to find the module by index, not by name.</p> <p>As all Python objects should be controlled through the Python memory management, usage of “static” type objects is discouraged, unless the type object itself has no memory-managed state. To simplify definition of heap types, a new method:</p> <div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">PyTypeObject</span><span class="o">*</span> <span class="n">PyType_Copy</span><span class="p">(</span><span class="n">PyTypeObject</span><span class="o">*</span><span class="p">);</span> </pre></div> </div> <p>is added.</p> </section> <section id="example"> <h2><a class="toc-backref" href="#example" role="doc-backlink">Example</a></h2> <p>xxmodule.c would be changed to remove the initxx function, and add the following code instead:</p> <div class="highlight-default notranslate"><div class="highlight"><pre><span></span>struct xxstate{ PyObject *ErrorObject; PyObject *Xxo_Type; }; #define xxstate(o) ((struct xxstate*)PyModule_GetState(o)) static int xx_traverse(PyObject *m, visitproc v, void *arg) { Py_VISIT(xxstate(m)->ErrorObject); Py_VISIT(xxstate(m)->Xxo_Type); return 0; } static int xx_clear(PyObject *m) { Py_CLEAR(xxstate(m)->ErrorObject); Py_CLEAR(xxstate(m)->Xxo_Type); return 0; } static struct PyModuleDef xxmodule = { {}, /* m_base */ sizeof(struct xxstate), &xx_methods, 0, /* m_reload */ xx_traverse, xx_clear, 0, /* m_free - not needed, since all is done in m_clear */ } PyObject* PyInit_xx() { PyObject *res = PyModule_New("xx", &xxmodule); if (!res) return NULL; xxstate(res)->ErrorObject = PyErr_NewException("xx.error", NULL, NULL); if (!xxstate(res)->ErrorObject) { Py_DECREF(res); return NULL; } xxstate(res)->XxoType = PyType_Copy(&Xxo_Type); if (!xxstate(res)->Xxo_Type) { Py_DECREF(res); return NULL; } return res; } </pre></div> </div> </section> <section id="discussion"> <h2><a class="toc-backref" href="#discussion" role="doc-backlink">Discussion</a></h2> <p>Tim Peters reports in <a class="footnote-reference brackets" href="#id2" id="id1">[1]</a> that PythonLabs considered such a feature at one point, and lists the following additional hooks which aren’t currently supported in this PEP:</p> <ul class="simple"> <li>when the module object is deleted from sys.modules</li> <li>when Py_Finalize is called</li> <li>when Python exits</li> <li>when the Python DLL is unloaded (Windows only)</li> </ul> </section> <section id="references"> <h2><a class="toc-backref" href="#references" role="doc-backlink">References</a></h2> <aside class="footnote-list brackets"> <aside class="footnote brackets" id="id2" role="doc-footnote"> <dt class="label" id="id2">[<a href="#id1">1</a>]</dt> <dd>Tim Peters, reporting earlier conversation about such a feature <a class="reference external" href="https://mail.python.org/pipermail/python-3000/2006-April/000726.html">https://mail.python.org/pipermail/python-3000/2006-April/000726.html</a></aside> </aside> </section> <section id="copyright"> <h2><a class="toc-backref" href="#copyright" role="doc-backlink">Copyright</a></h2> <p>This document has been placed in the public domain.</p> </section> </section> <hr class="docutils" /> <p>Source: <a class="reference external" href="https://github.com/python/peps/blob/main/peps/pep-3121.rst">https://github.com/python/peps/blob/main/peps/pep-3121.rst</a></p> <p>Last modified: <a class="reference external" href="https://github.com/python/peps/commits/main/peps/pep-3121.rst">2025-02-01 08:59:27 GMT</a></p> </article> <nav id="pep-sidebar"> <h2>Contents</h2> <ul> <li><a class="reference internal" href="#abstract">Abstract</a></li> <li><a class="reference internal" href="#problems">Problems</a><ul> <li><a class="reference internal" href="#module-finalization">Module Finalization</a></li> <li><a class="reference internal" href="#entry-point-name-conflicts">Entry point name conflicts</a></li> <li><a class="reference internal" href="#entry-point-signature">Entry point signature</a></li> <li><a class="reference internal" href="#multiple-interpreters">Multiple Interpreters</a></li> </ul> </li> <li><a class="reference internal" href="#specification">Specification</a></li> <li><a class="reference internal" href="#example">Example</a></li> <li><a class="reference internal" href="#discussion">Discussion</a></li> <li><a class="reference internal" href="#references">References</a></li> <li><a class="reference internal" href="#copyright">Copyright</a></li> </ul> <br> <a id="source" href="https://github.com/python/peps/blob/main/peps/pep-3121.rst">Page Source (GitHub)</a> </nav> </section> <script src="../_static/colour_scheme.js"></script> <script src="../_static/wrap_tables.js"></script> <script src="../_static/sticky_banner.js"></script> </body> </html>