Issue 642578: Expose PyImport_FrozenModules in imp

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"> <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en"> <head> <title> Issue 642578: Expose PyImport_FrozenModules in imp - Python tracker </title> <link rel="shortcut icon" href="@@file/favicon.ico" /> <link rel="stylesheet" type="text/css" href="@@file/main.css" /> <link rel="stylesheet" type="text/css" href="@@file/style.css" /> <link rel="search" type="application/opensearchdescription+xml" href="@@file/osd.xml" title="Python bug tracker search" /> <meta http-equiv="Content-Type" content="text/html; charset=utf-8" /> <script nonce="9a36596116074166d89a1fea0f276c8db6c82a6abf90f51503d1a89ba5c01079" type="text/javascript"> submitted = false; function submit_once() { if (submitted) { alert("Your request is being processed.\nPlease be patient."); return false; } submitted = true; return true; } function help_window(helpurl, width, height) { HelpWin = window.open('https://bugs.python.org/' + helpurl, 'RoundupHelpWindow', 'scrollbars=yes,resizable=yes,toolbar=no,height='+height+',width='+width); HelpWin.focus () } </script> <script type="text/javascript" src="https://ajax.googleapis.com/ajax/libs/jquery/1.6.2/jquery.min.js"></script> <script type="text/javascript" src="https://ajax.googleapis.com/ajax/libs/jqueryui/1.8.15/jquery-ui.js"></script> <script type="text/javascript" src="@@file/issue.item.js"></script> <link rel="stylesheet" type="text/css" href="https://ajax.googleapis.com/ajax/libs/jqueryui/1.8/themes/smoothness/jquery-ui.css" /> </head> <body>  <h1 id="logoheader"> <a accesskey="1" href="." id="logolink"> <img src="@@file/python-logo.gif" alt="homepage" border="0" id="logo" /></a> </h1> <div id="utility-menu">  <div id="searchbox"> <form name="searchform" method="get" action="issue" id="searchform"> <div id="search"> <input type="hidden" name="@columns" value="id,github,activity,title,creator,assignee,status,type" /> <input type="hidden" name="@sort" value="-activity" /> <input type="hidden" name="@filter" value="status" /> <input type="hidden" name="@action" value="searchid" /> <input type="hidden" name="ignore" value="file:content" /> <input class="input-text" id="search-text" name="@search_text" size="10" /> <input type="submit" id="submit" value="search" name="submit" class="input-button" /> <input type="radio" name="status" id="status_notresolved" value="-1,1,3" /> <label for="status_notresolved">open</label> <input type="radio" name="status" checked="checked" id="status_all" value="-1,1,2,3" /> <label for="status_all">all</label> </div> </form> </div> </div> <div id="left-hand-navigation">  <div id="menu"> <ul class="level-one"> <li><a href="https://www.python.org/" title="Go to the Python homepage">Python Home</a></li> <li><a href="https://www.python.org/about/" title="About The Python Language">About</a></li> <li><a href="https://www.python.org/blogs/" title="">News</a></li> <li><a href="https://www.python.org/doc/" title="">Documentation</a></li> <li><a href="https://www.python.org/downloads/" title="">Downloads</a></li> <li><a href="https://www.python.org/community/" title="">Community</a></li> <li><a href="https://www.python.org/psf/" title="Python Software Foundation">Foundation</a></li> <li><a href="https://devguide.python.org/" title="Python Developer's Guide">Developer's Guide</a></li> <li class="selected"><a href="." class="selected" title="Python Issue Tracker">Issue Tracker</a> <ul class="level-two"> <li> <strong>Issues</strong> <ul class="level-three"> <li><a href="issue?@template=search&status=1">Search</a></li> <li><a href="issue?@action=random">Random Issue</a></li> <li> <form method="post" action="issue642578"> <input type="submit" class="form-small" value="Show issue:" /> <input class="form-small" size="4" type="text" name="@number" /> <input type="hidden" name="@type" value="issue" /> <input type="hidden" name="@action" value="show" /> </form> </li> </ul> </li> <li> <strong>Summaries</strong> <ul class="level-three"> <li> <a href="issue?status=1&@sort=-activity&@columns=id%2Cgithub%2Cactivity%2Ctitle%2Ccreator%2Cstatus&@dispname=Issues%20with%20patch&@startwith=0&@group=priority&keywords=2&@action=search&@filter=&@pagesize=50">Issues with patch</a> </li> <li> <a href="issue?status=1&@sort=-activity&@columns=id%2Cgithub%2Cactivity%2Ctitle%2Ccreator%2Cstatus&@dispname=Easy%20issues&@startwith=0&@group=priority&keywords=6&@action=search&@filter=&@pagesize=50">Easy issues</a> </li> <li> <a href="issue?@template=stats">Stats</a> </li> </ul> </li> <li> <strong>User</strong> <form method="post" action="issue642578"> <ul class="level-three"> <li> Login<br /> <input size="10" name="openid_identifier" style="" /><br /> <input size="10" type="password" name="__login_password" /><br /> <input type="hidden" name="@action" value="Login" /> <input type="checkbox" name="remember" id="remember" /> <label for="remember">Remember me?</label><br /> <input class="form-small" type="submit" value="Login" /><br /> <input type="hidden" name="__came_from" value="https://bugs.python.org/issue642578?"> <input type="hidden" name="@sort" value=""/> <input type="hidden" name="@group" value=""/> <input type="hidden" name="@pagesize" value="50"/> <input type="hidden" name="@startwith" value="0"/> </li> <li> </li> <li><a href="user?@template=forgotten">Lost your login?</a></li> </ul> </form> </li> <li> <strong>Administration</strong> <ul class="level-three"> <li> <a href="user?@sort=username">User List</a></li> <li> <a href="user?iscommitter=1&@action=search&@sort=username&@pagesize=300">Committer List</a></li> </ul> </li> <li> <strong>Help</strong> <ul class="level-three"> <li><a href="http://docs.python.org/devguide/triaging.html"> Tracker Documentation</a></li> <li><a href="http://wiki.python.org/moin/TrackerDevelopment"> Tracker Development</a></li> <li><a href="https://github.com/python/psf-infra-meta/issues"> Report Tracker Problem</a></li> </ul> </li> </ul> </li> </ul> </div>  </div>  <div id="content-body"> <div id="body-main"> <div id="content"> <div id="breadcrumb"> Issue642578 </div> <div id="migration-notice"> <div id="migration-images"> <img width="32" src="@@file/python-logo-small.png" /> ➜ <a href="https://github.com/python/cpython/issues"><img width="32" src="@@file/gh-icon.png" /></a> </div> <p>This issue tracker <b>has been migrated to <a href="https://github.com/python/cpython/issues">GitHub</a></b>, and is currently <b>read-only</b>.<br /> For more information, <a title="GitHub FAQs" href="https://devguide.python.org/gh-faq/"> see the GitHub FAQs in the Python's Developer Guide.</a></p> </div> <div> <form method="post" name="itemSynopsis" onsubmit="return submit_once()" enctype="multipart/form-data" action="issue642578"> <div id="gh-issue-link"> <a href="https://github.com/python/cpython/issues/37522"> <img width="32" src="@@file/gh-icon.png" /> <p> <span>This issue has been migrated to GitHub:</span> https://github.com/python/cpython/issues/37522 </p> </a> </div> <fieldset><legend>classification</legend> <table class="form"> <tr> <th class="required"><a href="http://docs.python.org/devguide/triaging.html#title" target="_blank">Title</a>:</th> <td colspan="3"> <span>Expose PyImport_FrozenModules in imp</span> <input type="hidden" name="title" value="Expose PyImport_FrozenModules in imp"> </td> </tr> <tr> <th class="required"><a href="http://docs.python.org/devguide/triaging.html#type" target="_blank">Type</a>:</th> <td></td> <th><a href="http://docs.python.org/devguide/triaging.html#stage" target="_blank">Stage</a>:</th> <td></td> </tr> <tr> <th><a href="http://docs.python.org/devguide/triaging.html#components" target="_blank">Components</a>:</th> <td>Interpreter Core</td> <th><a href="http://docs.python.org/devguide/triaging.html#versions" target="_blank">Versions</a>:</th> <td>Python 2.3</td> </tr> </table> </fieldset> <fieldset><legend>process</legend> <table class="form"> <tr> <th><a href="http://docs.python.org/devguide/triaging.html#status" target="_blank">Status</a>:</th> <td>closed</td> <th><a href="http://docs.python.org/devguide/triaging.html#resolution" target="_blank">Resolution</a>:</th> <td>accepted</td> </tr> <tr> <th> <a href="http://docs.python.org/devguide/triaging.html#dependencies" target="_blank">Dependencies</a>: </th> <td> </td> <th><a href="http://docs.python.org/devguide/triaging.html#superseder" target="_blank">Superseder</a>:</th> <td> </td> </tr> <tr> <th> <a href="http://docs.python.org/devguide/triaging.html#assigned-to" target="_blank">Assigned To</a>: </th> <td> </td> <th> <a href="http://docs.python.org/devguide/triaging.html#nosy-list" target="_blank">Nosy List</a>: </th> <td> jvr, loewis, theller </td> </tr> <tr> <th> <a href="http://docs.python.org/devguide/triaging.html#priority" target="_blank">Priority</a>: </th> <td>normal</td> <th> <a href="http://docs.python.org/devguide/triaging.html#keywords" target="_blank">Keywords</a>: </th> <td>patch</td> </tr> </table> </fieldset> </form> <p>Created on <strong>2002-11-22 22:56</strong> by <strong>jvr</strong>, last changed <strong>2022-04-10 16:05</strong> by <strong>admin</strong>. This issue is now <strong style="color:#00F; background-color:inherit;">closed</strong>.</p> <table class="files"> <tr><th colspan="5" class="header">Files</th></tr> <tr> <th>File name</th> <th>Uploaded</th> <th>Description</th> <th>Edit</th> </tr> <tr> <td> <a href="file4730/import.c.patch">import.c.patch</a> </td> <td> <span>jvr</span>, <span>2002-11-23 23:26</span> </td> <td>expose the builtin frozen modules list</td> <td> </td> </tr> </table> <table class="messages"> <tr><th colspan="4" class="header">Messages (11)</th></tr> <tr> <th> <a href="#msg41739" id="msg41739">msg41739</a> - <a href="msg41739">(view)</a></th> <th>Author: Just van Rossum (jvr) <span title="Contributor form received">*</span> <img src="@@file/triager.png" title="Python triager" alt="(Python triager)" /></th> <th>Date: 2002-11-22 22:56</th> </tr> <tr> <td colspan="4" class="content"> <pre>The attached patch exposes the PyImport_FrozenModules global variable. It can be written through the new imp.set_frozenmodules() function. No read functionality has been provided since that would be harder to do <wink>, and would IMO not be all that useful. Rationale: It allows py2exe/McMillan-Installer-like tools to be written WITHOUT __import__ hooks, while still being able to bootstrap with pure Python code. Use case: I'm in the process of writing a freeze-like tool for MacOSX. Applications on this platform are (usually) simply a directory containing some meta files and arbitrary stuff. I can (and will) make it simply add .pyc files to this directory, but it will be a huge win to be able to put a single file in there containing all needed Python modules. I would prefer to do this without resorting to __import__ hooks, as they add overhead and brittleness. I use a vanilla python executable and bootstrap in Python. Look ma, no C! Implementation: PyImport_FrozenModules is an array of structs containing a pointer to a C string (the name), a pointer to the marshalled code object data and an int specifying the size of the code data (negated if the module is a package). None of these are Python objects. imp.set_frozenmodules() takes a sequence of tuples as its argument: (name, codedata, ispkg). Internally this sequence is converted to a tuple, and a reference is kept in an additional static variable in import.c. The elements of the newly allocated PyImport_FrozenModules array simply point to the actual string data, as referenced by the tuple. The patch doesn't include documentation yet, but I will add that as soon as the patch is otherwise accepted.</pre> </td> </tr> <tr> <th> <a href="#msg41740" id="msg41740">msg41740</a> - <a href="msg41740">(view)</a></th> <th>Author: Just van Rossum (jvr) <span title="Contributor form received">*</span> <img src="@@file/triager.png" title="Python triager" alt="(Python triager)" /></th> <th>Date: 2002-11-22 23:21</th> </tr> <tr> <td colspan="4" class="content"> <pre>Logged In: YES user_id=92689 (doop: the first patch I uploaded contained a syntax error. How embarrassing... The current file is correct.)</pre> </td> </tr> <tr> <th> <a href="#msg41741" id="msg41741">msg41741</a> - <a href="msg41741">(view)</a></th> <th>Author: Just van Rossum (jvr) <span title="Contributor form received">*</span> <img src="@@file/triager.png" title="Python triager" alt="(Python triager)" /></th> <th>Date: 2002-11-23 00:35</th> </tr> <tr> <td colspan="4" class="content"> <pre>Logged In: YES user_id=92689 Ok, I changed my mind and created a matching imp.get_frozenmodules() function that _returns_ a Python representation of PyImport_FrozenModules. Replaced the patch.</pre> </td> </tr> <tr> <th> <a href="#msg41742" id="msg41742">msg41742</a> - <a href="msg41742">(view)</a></th> <th>Author: Martin v. Löwis (loewis) <span title="Contributor form received">*</span> <img src="@@file/committer.png" title="Python committer" alt="(Python committer)" /></th> <th>Date: 2002-11-23 22:14</th> </tr> <tr> <td colspan="4" class="content"> <pre>Logged In: YES user_id=21627 Can you provide a patch for libimp.tex as well?</pre> </td> </tr> <tr> <th> <a href="#msg41743" id="msg41743">msg41743</a> - <a href="msg41743">(view)</a></th> <th>Author: Just van Rossum (jvr) <span title="Contributor form received">*</span> <img src="@@file/triager.png" title="Python triager" alt="(Python triager)" /></th> <th>Date: 2002-11-23 23:26</th> </tr> <tr> <td colspan="4" class="content"> <pre>Logged In: YES user_id=92689 Yes, done in the replaced patch. I've also changed a subtlety in get_frozenmodules(): the previous version returned the original tuple as it was set by set_frozenmodules(), now it always builds the tuple from scratch: it's possible that someone changes PyImport_FrozenModules from C, invalidating the cached tuple.</pre> </td> </tr> <tr> <th> <a href="#msg41744" id="msg41744">msg41744</a> - <a href="msg41744">(view)</a></th> <th>Author: Thomas Heller (theller) <span title="Contributor form received">*</span> <img src="@@file/committer.png" title="Python committer" alt="(Python committer)" /></th> <th>Date: 2002-11-25 08:52</th> </tr> <tr> <td colspan="4" class="content"> <pre>Logged In: YES user_id=11105 I haven't actually tried out the patch, but it looks fine to me.</pre> </td> </tr> <tr> <th> <a href="#msg41745" id="msg41745">msg41745</a> - <a href="msg41745">(view)</a></th> <th>Author: Thomas Heller (theller) <span title="Contributor form received">*</span> <img src="@@file/committer.png" title="Python committer" alt="(Python committer)" /></th> <th>Date: 2002-11-25 19:22</th> </tr> <tr> <td colspan="4" class="content"> <pre>Logged In: YES user_id=11105 Hm, shouldn't overwriting the frozen module table, if it is already set, raise an error? Or should there be more than one frozen tables? And having said that, I could even think of situations where the frozen module tables should be private to *one* interpreter, and not be shared if there have been other ones created by Py_NewInterpreter. But this issue should probably left for a separate patch ;-)</pre> </td> </tr> <tr> <th> <a href="#msg41746" id="msg41746">msg41746</a> - <a href="msg41746">(view)</a></th> <th>Author: Just van Rossum (jvr) <span title="Contributor form received">*</span> <img src="@@file/triager.png" title="Python triager" alt="(Python triager)" /></th> <th>Date: 2002-11-25 19:29</th> </tr> <tr> <td colspan="4" class="content"> <pre>Logged In: YES user_id=92689 [raise error on overwrite] No, for one because it is always set (see <a href="https://github.com/python/cpython/blob/master/Python/frozen.c">Python/frozen.c</a> or do "import __hello__" ;-). Also: it _might_ be useful to extend the list by first retrieving is, appending to it and setting it again. The other "issue": let's make that an issue when rewriting the entire import stuff is an active project... (It would also involve a C API to set/modify the frozen modules list, which, erm, perhaps should be a dict anyway, and and...)</pre> </td> </tr> <tr> <th> <a href="#msg41747" id="msg41747">msg41747</a> - <a href="msg41747">(view)</a></th> <th>Author: Just van Rossum (jvr) <span title="Contributor form received">*</span> <img src="@@file/triager.png" title="Python triager" alt="(Python triager)" /></th> <th>Date: 2002-11-26 00:53</th> </tr> <tr> <td colspan="4" class="content"> <pre>Logged In: YES user_id=92689 Anticipating acceptance of this patch, I've checkin in actual (but optional) employment of imp.set_frozenmodules() in <a href="https://github.com/python/cpython/blob/master/Mac/Lib/bundlebuilder.py">Mac/Lib/bundlebuilder.py</a>. The neat thing about this is that bootstrapping happens in a custom "site" module: I don't even have to wrap the main program...</pre> </td> </tr> <tr> <th> <a href="#msg41748" id="msg41748">msg41748</a> - <a href="msg41748">(view)</a></th> <th>Author: Just van Rossum (jvr) <span title="Contributor form received">*</span> <img src="@@file/triager.png" title="Python triager" alt="(Python triager)" /></th> <th>Date: 2002-11-29 12:33</th> </tr> <tr> <td colspan="4" class="content"> <pre>Logged In: YES user_id=92689 I'm ready to commit this. Any objections?</pre> </td> </tr> <tr> <th> <a href="#msg41749" id="msg41749">msg41749</a> - <a href="msg41749">(view)</a></th> <th>Author: Just van Rossum (jvr) <span title="Contributor form received">*</span> <img src="@@file/triager.png" title="Python triager" alt="(Python triager)" /></th> <th>Date: 2002-11-29 20:52</th> </tr> <tr> <td colspan="4" class="content"> <pre>Logged In: YES user_id=92689 Checked in with minor improvements as: <a href="https://github.com/python/cpython/blob/master/Misc/NEWS">Misc/NEWS</a> rev. 1.550 <a href="https://github.com/python/cpython/blob/master/Doc/lib/libimp.tex">Doc/lib/libimp.tex</a> rev. 1.33 <a href="https://github.com/python/cpython/blob/master/Python/import.c">Python/import.c</a> rev. 2.211</pre> </td> </tr> </table> <table class="history table table-condensed table-striped"><tr><th colspan="4" class="header"> History </th></tr><tr> <th>Date</th> <th>User</th> <th>Action</th> <th>Args</th> </tr> <tr><td>2022-04-10 16:05:55</td><td>admin</td><td>set</td><td>github: 37522</td></tr> <tr><td>2002-11-22 22:56:04</td><td>jvr</td><td>create</td><td></td></tr> </table> </div> </div>  <div id="footer"> <div id="credits"> Supported by <a href="https://python.org/psf-landing/" title="The Python Software Foundation">The Python Software Foundation</a>, <br> Powered by <a href="http://roundup.sourceforge.net" title="Powered by the Roundup Issue Tracker">Roundup</a> </div>  Copyright © 1990-2022, <a href="http://python.org/psf">Python Software Foundation</a><br /> <a href="http://python.org/about/legal">Legal Statements</a> </div>  </div>  </div>  </body> </html>

CINXE.COM

Issue 642578: Expose PyImport_FrozenModules in imp - Python tracker