Issue 210821: On Windows, APIs passing FILE* break with Borland C (PR#121)

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"> <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en"> <head> <title> Issue 210821: On Windows, APIs passing FILE* break with Borland C (PR#121) - Python tracker </title> <link rel="shortcut icon" href="@@file/favicon.ico" /> <link rel="stylesheet" type="text/css" href="@@file/main.css" /> <link rel="stylesheet" type="text/css" href="@@file/style.css" /> <link rel="search" type="application/opensearchdescription+xml" href="@@file/osd.xml" title="Python bug tracker search" /> <meta http-equiv="Content-Type" content="text/html; charset=utf-8" /> <script nonce="5eabb67fc333efc32023db5de46b98e9efee99a324fb102120d93339843d0cef" type="text/javascript"> submitted = false; function submit_once() { if (submitted) { alert("Your request is being processed.\nPlease be patient."); return false; } submitted = true; return true; } function help_window(helpurl, width, height) { HelpWin = window.open('https://bugs.python.org/' + helpurl, 'RoundupHelpWindow', 'scrollbars=yes,resizable=yes,toolbar=no,height='+height+',width='+width); HelpWin.focus () } </script> <script type="text/javascript" src="https://ajax.googleapis.com/ajax/libs/jquery/1.6.2/jquery.min.js"></script> <script type="text/javascript" src="https://ajax.googleapis.com/ajax/libs/jqueryui/1.8.15/jquery-ui.js"></script> <script type="text/javascript" src="@@file/issue.item.js"></script> <link rel="stylesheet" type="text/css" href="https://ajax.googleapis.com/ajax/libs/jqueryui/1.8/themes/smoothness/jquery-ui.css" /> </head> <body>  <h1 id="logoheader"> <a accesskey="1" href="." id="logolink"> <img src="@@file/python-logo.gif" alt="homepage" border="0" id="logo" /></a> </h1> <div id="utility-menu">  <div id="searchbox"> <form name="searchform" method="get" action="issue" id="searchform"> <div id="search"> <input type="hidden" name="@columns" value="id,github,activity,title,creator,assignee,status,type" /> <input type="hidden" name="@sort" value="-activity" /> <input type="hidden" name="@filter" value="status" /> <input type="hidden" name="@action" value="searchid" /> <input type="hidden" name="ignore" value="file:content" /> <input class="input-text" id="search-text" name="@search_text" size="10" /> <input type="submit" id="submit" value="search" name="submit" class="input-button" /> <input type="radio" name="status" id="status_notresolved" value="-1,1,3" /> <label for="status_notresolved">open</label> <input type="radio" name="status" checked="checked" id="status_all" value="-1,1,2,3" /> <label for="status_all">all</label> </div> </form> </div> </div> <div id="left-hand-navigation">  <div id="menu"> <ul class="level-one"> <li><a href="https://www.python.org/" title="Go to the Python homepage">Python Home</a></li> <li><a href="https://www.python.org/about/" title="About The Python Language">About</a></li> <li><a href="https://www.python.org/blogs/" title="">News</a></li> <li><a href="https://www.python.org/doc/" title="">Documentation</a></li> <li><a href="https://www.python.org/downloads/" title="">Downloads</a></li> <li><a href="https://www.python.org/community/" title="">Community</a></li> <li><a href="https://www.python.org/psf/" title="Python Software Foundation">Foundation</a></li> <li><a href="https://devguide.python.org/" title="Python Developer's Guide">Developer's Guide</a></li> <li class="selected"><a href="." class="selected" title="Python Issue Tracker">Issue Tracker</a> <ul class="level-two"> <li> <strong>Issues</strong> <ul class="level-three"> <li><a href="issue?@template=search&status=1">Search</a></li> <li><a href="issue?@action=random">Random Issue</a></li> <li> <form method="post" action="issue210821"> <input type="submit" class="form-small" value="Show issue:" /> <input class="form-small" size="4" type="text" name="@number" /> <input type="hidden" name="@type" value="issue" /> <input type="hidden" name="@action" value="show" /> </form> </li> </ul> </li> <li> <strong>Summaries</strong> <ul class="level-three"> <li> <a href="issue?status=1&@sort=-activity&@columns=id%2Cgithub%2Cactivity%2Ctitle%2Ccreator%2Cstatus&@dispname=Issues%20with%20patch&@startwith=0&@group=priority&keywords=2&@action=search&@filter=&@pagesize=50">Issues with patch</a> </li> <li> <a href="issue?status=1&@sort=-activity&@columns=id%2Cgithub%2Cactivity%2Ctitle%2Ccreator%2Cstatus&@dispname=Easy%20issues&@startwith=0&@group=priority&keywords=6&@action=search&@filter=&@pagesize=50">Easy issues</a> </li> <li> <a href="issue?@template=stats">Stats</a> </li> </ul> </li> <li> <strong>User</strong> <form method="post" action="issue210821"> <ul class="level-three"> <li> Login<br /> <input size="10" name="openid_identifier" style="" /><br /> <input size="10" type="password" name="__login_password" /><br /> <input type="hidden" name="@action" value="Login" /> <input type="checkbox" name="remember" id="remember" /> <label for="remember">Remember me?</label><br /> <input class="form-small" type="submit" value="Login" /><br /> <input type="hidden" name="__came_from" value="https://bugs.python.org/issue210821?"> <input type="hidden" name="@sort" value=""/> <input type="hidden" name="@group" value=""/> <input type="hidden" name="@pagesize" value="50"/> <input type="hidden" name="@startwith" value="0"/> </li> <li> </li> <li><a href="user?@template=forgotten">Lost your login?</a></li> </ul> </form> </li> <li> <strong>Administration</strong> <ul class="level-three"> <li> <a href="user?@sort=username">User List</a></li> <li> <a href="user?iscommitter=1&@action=search&@sort=username&@pagesize=300">Committer List</a></li> </ul> </li> <li> <strong>Help</strong> <ul class="level-three"> <li><a href="http://docs.python.org/devguide/triaging.html"> Tracker Documentation</a></li> <li><a href="http://wiki.python.org/moin/TrackerDevelopment"> Tracker Development</a></li> <li><a href="https://github.com/python/psf-infra-meta/issues"> Report Tracker Problem</a></li> </ul> </li> </ul> </li> </ul> </div>  </div>  <div id="content-body"> <div id="body-main"> <div id="content"> <div id="breadcrumb"> Issue210821 </div> <div id="migration-notice"> <div id="migration-images"> <img width="32" src="@@file/python-logo-small.png" /> ➜ <a href="https://github.com/python/cpython/issues"><img width="32" src="@@file/gh-icon.png" /></a> </div> <p>This issue tracker <b>has been migrated to <a href="https://github.com/python/cpython/issues">GitHub</a></b>, and is currently <b>read-only</b>.<br /> For more information, <a title="GitHub FAQs" href="https://devguide.python.org/gh-faq/"> see the GitHub FAQs in the Python's Developer Guide.</a></p> </div> <div> <form method="post" name="itemSynopsis" onsubmit="return submit_once()" enctype="multipart/form-data" action="issue210821"> <div id="gh-issue-link"> <a href="https://github.com/python/cpython/issues/32824"> <img width="32" src="@@file/gh-icon.png" /> <p> <span>This issue has been migrated to GitHub:</span> https://github.com/python/cpython/issues/32824 </p> </a> </div> <fieldset><legend>classification</legend> <table class="form"> <tr> <th class="required"><a href="http://docs.python.org/devguide/triaging.html#title" target="_blank">Title</a>:</th> <td colspan="3"> <span>On Windows, APIs passing FILE* break with Borland C (PR#121)</span> <input type="hidden" name="title" value="On Windows, APIs passing FILE* break with Borland C (PR#121)"> </td> </tr> <tr> <th class="required"><a href="http://docs.python.org/devguide/triaging.html#type" target="_blank">Type</a>:</th> <td>enhancement</td> <th><a href="http://docs.python.org/devguide/triaging.html#stage" target="_blank">Stage</a>:</th> <td></td> </tr> <tr> <th><a href="http://docs.python.org/devguide/triaging.html#components" target="_blank">Components</a>:</th> <td>Interpreter Core</td> <th><a href="http://docs.python.org/devguide/triaging.html#versions" target="_blank">Versions</a>:</th> <td></td> </tr> </table> </fieldset> <fieldset><legend>process</legend> <table class="form"> <tr> <th><a href="http://docs.python.org/devguide/triaging.html#status" target="_blank">Status</a>:</th> <td>closed</td> <th><a href="http://docs.python.org/devguide/triaging.html#resolution" target="_blank">Resolution</a>:</th> <td>later</td> </tr> <tr> <th> <a href="http://docs.python.org/devguide/triaging.html#dependencies" target="_blank">Dependencies</a>: </th> <td> </td> <th><a href="http://docs.python.org/devguide/triaging.html#superseder" target="_blank">Superseder</a>:</th> <td> </td> </tr> <tr> <th> <a href="http://docs.python.org/devguide/triaging.html#assigned-to" target="_blank">Assigned To</a>: </th> <td> jhylton </td> <th> <a href="http://docs.python.org/devguide/triaging.html#nosy-list" target="_blank">Nosy List</a>: </th> <td> gvanrossum, jhylton, nobody, strife35 </td> </tr> <tr> <th> <a href="http://docs.python.org/devguide/triaging.html#priority" target="_blank">Priority</a>: </th> <td>normal</td> <th> <a href="http://docs.python.org/devguide/triaging.html#keywords" target="_blank">Keywords</a>: </th> <td></td> </tr> </table> </fieldset> </form> <p>Created on <strong>2000-08-01 21:10</strong> by <strong>anonymous</strong>, last changed <strong>2022-04-10 16:02</strong> by <strong>admin</strong>. This issue is now <strong style="color:#00F; background-color:inherit;">closed</strong>.</p> <table class="messages"> <tr><th colspan="4" class="header">Messages (11)</th></tr> <tr> <th> <a href="#msg670" id="msg670">msg670</a> - <a href="msg670">(view)</a></th> <th>Author: Nobody/Anonymous (nobody)</th> <th>Date: 2000-08-01 21:10</th> </tr> <tr> <td colspan="4" class="content"> <pre>Jitterbug-Id: 121 Submitted-By: "Brad Clements" <<a href="mailto:bkc@murkworks.com">bkc@murkworks.com</a>> Date: Wed, 3 Nov 1999 15:53:01 -0500 Version: None OS: None The system encountered a fatal error After command: Received: The last error code was: Connection refused Web interface using {HYPERLINK "<a href="http://samba.anu.edu.au/jitterbug/">http://samba.anu.edu.au/jitterbug/</a>"}Jitterbug {HYPERLINK "../python-bugs/"}Public interface {HYPERLINK "../python-bugs.private"}Private interface (requires password) {HYPERLINK "<a href="http://www.python.org/">http://www.python.org/</a>"}Python home page ----- Here's what I said for Python 1.5.2 on Win32 #0 PyRun_SimpleFile, PyRun_File and other functions that take a FILE * are not usable on WIN32 from non-VC applications because python15.dll is statically linked to the MS runtime DLL. Embedding applications that try to use these functions are passing in FILE * structures that do not match MS's runtime format. For example, I'm using Python in a Borland C++ Builder application. Although I can open a FILE *, when passed to python15.dll the FILE * is not usable. The addition of two helper functions would solve this problem: FILE * PyRun_OpenFile(char *file, char *mode) { return fopen(file,mode) } int PyRun_CloseFile(FILE *ptr) { return fclose(ptr) } This way embedding apps could get python15.dll to open the file and it would work. A temporary workaround is to always load the .pyc file in PyRun_SimpleFile.. Brad Clements, <a href="mailto:bkc@murkworks.com">bkc@murkworks.com</a> (315)268-1000 <a href="http://www.murkworks.com">http://www.murkworks.com</a> (315)268-9812 Fax netmeeting: ils://ils.murkworks.com ICQ: 14856937 ==================================================================== Audit trail: Fri Nov 05 10:31:10 1999 guido moved from incoming to request</pre> </td> </tr> <tr> <th> <a href="#msg671" id="msg671">msg671</a> - <a href="msg671">(view)</a></th> <th>Author: Nobody/Anonymous (nobody)</th> <th>Date: 2000-08-01 21:10</th> </tr> <tr> <td colspan="4" class="content"> <pre>From: Guido van Rossum <<a href="mailto:guido@CNRI.Reston.VA.US">guido@CNRI.Reston.VA.US</a>> Subject: Re: [Python-bugs-list] Python 1.5.2 bug, tried to post but got error (PR#121) Date: Wed, 03 Nov 1999 20:08:11 -0500 > I tried to post a bug, but got this error: > > The system encountered a fatal error We were being slammed by a defective spider (or, if you're more paranoid, by a hacker) so we temporarily turned off the webserver. It should be back on now. > Here's what I said for Python 1.5.2 on Win32 #0 > > PyRun_SimpleFile, PyRun_File and other functions that take a FILE * are not > usable on WIN32 from non-VC applications because python15.dll is statically > linked to the MS runtime DLL. Embedding applications that try to use these > functions are passing in FILE * structures that do not match MS's runtime > format. > > For example, I'm using Python in a Borland C++ Builder application. Although I > can open a FILE *, when passed to python15.dll the FILE * is not usable. > > The addition of two helper functions would solve this problem: > > FILE * PyRun_OpenFile(char *file, char *mode) > { > return fopen(file,mode) > } > > int PyRun_CloseFile(FILE *ptr) > { > return fclose(ptr) > } > > This way embedding apps could get python15.dll to open the file and it would > work. > > A temporary workaround is to always load the .pyc file in PyRun_SimpleFile.. This is an elegant solution. I think I'll add it. Could you mail me your suggestion again with the legal boilerplate included? See <a href="http://www.python.org/1.5/bugrelease.html">http://www.python.org/1.5/bugrelease.html</a> for the text and explanation. Our lawyers require that I request this silliness... --Guido van Rossum (home page: <a href="http://www.python.org/~guido/">http://www.python.org/~guido/</a>) </pre> </td> </tr> <tr> <th> <a href="#msg672" id="msg672">msg672</a> - <a href="msg672">(view)</a></th> <th>Author: Nobody/Anonymous (nobody)</th> <th>Date: 2000-08-01 21:10</th> </tr> <tr> <td colspan="4" class="content"> <pre>From: "Brad Clements" <<a href="mailto:bkc@murkworks.com">bkc@murkworks.com</a>> Subject: Re: [Python-bugs-list] Python 1.5.2 bug, tried to post but got error (PR#121) Date: Wed, 3 Nov 1999 20:22:21 -0500 Here it is again. Also, I believe this will resolve FAQ entry 8.10 > 8.10. Can't get Py_RunSimpleFile() to work. > This is very sensitive to the compiler vendor, version and (perhaps) even > options. If the FILE* structure in your embedding program isn't the same as > is assumed by the Python interpreter it won't work. The Python 1.5.* DLLs > (python15.dll) are all compiled with MS VC++ 5.0 and with > multithreading-DLL options (/MD, I think). > > If you can't change compilers or flags, try using Py_RunSimpleString(). A > trick to get it to run an arbitrary file is to construct a call to > execfile() with the name of your file as argument. > > > > --------------------------------------------------------------------------- > ----- > Here's what I said for Python 1.5.2 on Win32 #0 > > PyRun_SimpleFile, PyRun_File and other functions that take a FILE * are not > usable on WIN32 from non-VC applications because python15.dll is statically > linked to the MS runtime DLL. Embedding applications that try to use these > functions are passing in FILE * structures that do not match MS's runtime > format. > > For example, I'm using Python in a Borland C++ Builder application. Although I > can open a FILE *, when passed to python15.dll the FILE * is not usable. > > The addition of two helper functions would solve this problem: > > FILE * PyRun_OpenFile(char *file, char *mode) > { > return fopen(file,mode) > } > > int PyRun_CloseFile(FILE *ptr) > { > return fclose(ptr) > } > > This way embedding apps could get python15.dll to open the file and it would > work. > > A temporary workaround is to always load the .pyc file in PyRun_SimpleFile.. I confirm that, to the best of my knowledge and belief, this contribution is free of any claims of third parties under copyright, patent or other rights or interests ("claims"). To the extent that I have any such claims, I hereby grant to CNRI a nonexclusive, irrevocable, royalty-free, worldwide license to reproduce, distribute, perform and/or display publicly, prepare derivative versions, and otherwise use this contribution as part of the Python software and its related documentation, or any derivative versions thereof, at no cost to CNRI or its licensed users, and to authorize others to do so. I acknowledge that CNRI may, at its sole discretion, decide whether or not to incorporate this contribution in the Python software and its related documentation. I further grant CNRI permission to use my name and other identifying information provided to CNRI by me for use in connection with the Python software and its related documentation. Brad Clements, <a href="mailto:bkc@murkworks.com">bkc@murkworks.com</a> (315)268-1000 <a href="http://www.murkworks.com">http://www.murkworks.com</a> (315)268-9812 Fax netmeeting: ils://ils.murkworks.com ICQ: 14856937 </pre> </td> </tr> <tr> <th> <a href="#msg673" id="msg673">msg673</a> - <a href="msg673">(view)</a></th> <th>Author: Jeremy Hylton (jhylton) <img src="@@file/triager.png" title="Python triager" alt="(Python triager)" /></th> <th>Date: 2000-09-07 22:01</th> </tr> <tr> <td colspan="4" class="content"> <pre>Please do triage on this bug.</pre> </td> </tr> <tr> <th> <a href="#msg674" id="msg674">msg674</a> - <a href="msg674">(view)</a></th> <th>Author: Jeremy Hylton (jhylton) <img src="@@file/triager.png" title="Python triager" alt="(Python triager)" /></th> <th>Date: 2000-09-12 16:01</th> </tr> <tr> <td colspan="4" class="content"> <pre>Guido -- In 1999, you said you would add this feature. Did you? </pre> </td> </tr> <tr> <th> <a href="#msg675" id="msg675">msg675</a> - <a href="msg675">(view)</a></th> <th>Author: Guido van Rossum (gvanrossum) <span title="Contributor form received">*</span> <img src="@@file/committer.png" title="Python committer" alt="(Python committer)" /></th> <th>Date: 2000-09-13 10:53</th> </tr> <tr> <td colspan="4" class="content"> <pre>This is a feature request. Jeremy, how's that feature request PEP coming?</pre> </td> </tr> <tr> <th> <a href="#msg676" id="msg676">msg676</a> - <a href="msg676">(view)</a></th> <th>Author: Jeremy Hylton (jhylton) <img src="@@file/triager.png" title="Python triager" alt="(Python triager)" /></th> <th>Date: 2000-09-15 19:21</th> </tr> <tr> <td colspan="4" class="content"> <pre>added to <a href="https://www.python.org/dev/peps/pep-0042/">pep 42</a> </pre> </td> </tr> <tr> <th> <a href="#msg677" id="msg677">msg677</a> - <a href="msg677">(view)</a></th> <th>Author: adam edelstein (strife35)</th> <th>Date: 2001-08-23 14:33</th> </tr> <tr> <td colspan="4" class="content"> <pre>Logged In: YES user_id=305840 I have a related question, I have a project that I am working on, and I ran into an enormous amount of problems with file i/o as a result of pythons standard file stuf not using win32api, what I'm doing to prevent the problem is re- building python, and my project (embedded into python) but in rebuilding i have edited fileobject.c to force all file operation calls to use their win32api counterparts by defining macros to replace the stdio functions. I.E. fopen () -> createfile(), ...The macro replacement accepts the original arguements, converts them as neccessary to the ones needed by the win32API func, calls the win32api function and then returns a value which models the behavior of the original function. As you may have guessed already this presented some interesting problems as most of these functions are all passing around (FILE *) structures. To work around this I am returning the HANDLE's that these functions return letting the program pretend that they are FILE *s. I would have macrod FILE * to be a HANDLE except that one of the first functions PyFile_AsFile(f) is of type FILE *, and didnt like that very much.... Finally I had replaced almost everything.. Currently I am down to 2 last problems, dealing with fputs(), and that in now in my tracebacks sys.stdout instead of being an open path to stdout so as to enable sys.stdout.write('whatever') to write to stdout and thus to screen, it becomes an error as 'my' file opener i believe attempts to open a file named stdout.. I havent had much luck trying to figure what happens in sysmodule, and was wondering if any one had any ideas comments suggestions or help to offer. </pre> </td> </tr> <tr> <th> <a href="#msg678" id="msg678">msg678</a> - <a href="msg678">(view)</a></th> <th>Author: Guido van Rossum (gvanrossum) <span title="Contributor form received">*</span> <img src="@@file/committer.png" title="Python committer" alt="(Python committer)" /></th> <th>Date: 2001-08-23 14:38</th> </tr> <tr> <td colspan="4" class="content"> <pre>Logged In: YES user_id=6380 To strife35 (Adam Edelstein?): It might be better to have a separate object type that resembles a file object in its Python interface but uses the win32api calls rather than stdio. Can you explain why you need to use win32api I/O calls directly rather than via the MS-provided standard C library (which makes Win32 API calls underneath)?</pre> </td> </tr> <tr> <th> <a href="#msg679" id="msg679">msg679</a> - <a href="msg679">(view)</a></th> <th>Author: adam edelstein (strife35)</th> <th>Date: 2001-08-23 15:00</th> </tr> <tr> <td colspan="4" class="content"> <pre>Logged In: YES user_id=305840 The problem arrises in that this project is running over a network, and later I get problems with lockfiles and deleting files because although files may appear to be "closed" at the python level from the old stdio functions like fclose, they are indeed still open at the NT level and as a result cant be deleted. This becomes a real problem with OCI files, and other things on NT. It is not a problem on other os's like Solaris, Unix, linux, hpux, etc.., it is a WINnt problem Adam</pre> </td> </tr> <tr> <th> <a href="#msg680" id="msg680">msg680</a> - <a href="msg680">(view)</a></th> <th>Author: Guido van Rossum (gvanrossum) <span title="Contributor form received">*</span> <img src="@@file/committer.png" title="Python committer" alt="(Python committer)" /></th> <th>Date: 2001-08-23 16:36</th> </tr> <tr> <td colspan="4" class="content"> <pre>Logged In: YES user_id=6380 Hm. Are you *sure* that you are calling the close() method? I find it hard to believe that the stdio fclose() wouldn't make the corresponding Win32 call to really close the object. A fairly common failure mode in Python is to have a local variable referencing an open file and assuming that when the local variable goes out of scope, the file is closed. This will *usually* happen, but sometimes the stack frame is kept alive (e.g. by exception handling machinery) and that prevents the file from being closed. I am very familiar with the Windows problem that you can't remove/rename a file that's still open -- but I've never heard of a case where making the explicit close() call doesn't fix it. Just trying to be helpful -- what you're embarking on seems a big project...</pre> </td> </tr> </table> <table class="history table table-condensed table-striped"><tr><th colspan="4" class="header"> History </th></tr><tr> <th>Date</th> <th>User</th> <th>Action</th> <th>Args</th> </tr> <tr><td>2022-04-10 16:02:13</td><td>admin</td><td>set</td><td>github: 32824</td></tr> <tr><td>2000-08-01 21:10:13</td><td>anonymous</td><td>create</td><td></td></tr> </table> </div> </div>  <div id="footer"> <div id="credits"> Supported by <a href="https://python.org/psf-landing/" title="The Python Software Foundation">The Python Software Foundation</a>, <br> Powered by <a href="http://roundup.sourceforge.net" title="Powered by the Roundup Issue Tracker">Roundup</a> </div>  Copyright © 1990-2022, <a href="http://python.org/psf">Python Software Foundation</a><br /> <a href="http://python.org/about/legal">Legal Statements</a> </div>  </div>  </div>  </body> </html>

CINXE.COM

Issue 210821: On Windows, APIs passing FILE* break with Borland C (PR#121) - Python tracker