PyPy

<?xml version="1.0" encoding="utf-8"?> <?xml-stylesheet type="text/xsl" href="assets/xml/rss.xsl" media="all"?><rss version="2.0" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:atom="http://www.w3.org/2005/Atom"><channel><title>PyPy</title><link>https://www.pypy.org/</link><description>A Faster Python</description><atom:link href="https://www.pypy.org/rss.xml" rel="self" type="application/rss+xml"></atom:link><language>en</language><copyright>Contents © 2025 <a href="mailto:pypy-dev@pypy.org">The PyPy Team</a> </copyright><lastBuildDate>Wed, 26 Feb 2025 12:28:22 GMT</lastBuildDate><generator>Nikola (getnikola.com)</generator><docs>http://blogs.law.harvard.edu/tech/rss</docs><item><title>PyPy v7.3.19 release</title><link>https://www.pypy.org/posts/2025/02/pypy-v7319-release.html</link><dc:creator>mattip</dc:creator><description><section id="pypy-v7-3-19-release-of-python-2-7-3-10-and-3-11-beta"> <h2>PyPy v7.3.19: release of python 2.7, 3.10 and 3.11 beta</h2> The PyPy team is proud to release version 7.3.19 of PyPy. This is primarily a bug-fix release fixing JIT-related problems and follows quickly on the heels of the previous release on Feb 6, 2025. This release includes a python 3.11 interpreter. There were bugs in the first beta that could prevent its wider use, so we are continuing to call this release "beta". In the next release we will drop 3.10 and remove the "beta" label. The release includes three different interpreters: <ul class="simple"> <li>PyPy2.7, which is an interpreter supporting the syntax and the features of Python 2.7 including the stdlib for CPython 2.7.18+ (the <code class="docutils literal">+</code> is for backported security updates)</li> <li>PyPy3.10, which is an interpreter supporting the syntax and the features of Python 3.10, including the stdlib for CPython 3.10.16.</li> <li>PyPy3.11, which is an interpreter supporting the syntax and the features of Python 3.11, including the stdlib for CPython 3.11.11.</li> </ul> The interpreters are based on much the same codebase, thus the triple release. This is a micro release, all APIs are compatible with the other 7.3 releases. It follows after 7.3.17 release on August 28, 2024. We recommend updating. You can find links to download the releases here: <blockquote> <a class="reference external" href="https://pypy.org/download.html">https://pypy.org/download.html</a> </blockquote> We would like to thank our donors for the continued support of the PyPy project. If PyPy is not quite good enough for your needs, we are available for <a class="reference external" href="https://www.pypy.org/pypy-sponsors.html">direct consulting</a> work. If PyPy is helping you out, we would love to hear about it and encourage submissions to our <a class="reference external" href="https://pypy.org/blog">blog</a> via a pull request to <a class="reference external" href="https://github.com/pypy/pypy.org">https://github.com/pypy/pypy.org</a> We would also like to thank our contributors and encourage new people to join the project. PyPy has many layers and we need help with all of them: bug fixes, <a class="reference external" href="https://doc.pypy.org/">PyPy</a> and <a class="reference external" href="https://rpython.readthedocs.org">RPython</a> documentation improvements, or general <a class="reference external" href="https://doc.pypy.org/en/latest/project-ideas.html">help</a> with making RPython's JIT even better. If you are a python library maintainer and use C-extensions, please consider making a <a class="reference external" href="https://hpyproject.org/">HPy</a> / <a class="reference external" href="https://cffi.readthedocs.io">CFFI</a> / <a class="reference external" href="https://cppyy.readthedocs.io">cppyy</a> version of your library that would be performant on PyPy. In any case, both <a class="reference external" href="https://github.com/joerick/cibuildwheel">cibuildwheel</a> and the <a class="reference external" href="https://github.com/matthew-brett/multibuild">multibuild system</a> support building wheels for PyPy. <section id="what-is-pypy"> <h3>What is PyPy?</h3> PyPy is a Python interpreter, a drop-in replacement for CPython It's fast (<a class="reference external" href="https://speed.pypy.org">PyPy and CPython</a> performance comparison) due to its integrated tracing JIT compiler. We also welcome developers of other <a class="reference external" href="https://rpython.readthedocs.io/en/latest/examples.html">dynamic languages</a> to see what RPython can do for them. We provide binary builds for: <ul class="simple"> <li>x86 machines on most common operating systems (Linux 32/64 bits, Mac OS 64 bits, Windows 64 bits)</li> <li>64-bit ARM machines running Linux (<code class="docutils literal">aarch64</code>) and macos (<code class="docutils literal">macos_arm64</code>).</li> </ul> PyPy supports Windows 32-bit, Linux PPC64 big- and little-endian, Linux ARM 32 bit, RISC-V RV64IMAFD Linux, and s390x Linux but does not release binaries. Please reach out to us if you wish to sponsor binary releases for those platforms. Downstream packagers provide binary builds for debian, Fedora, conda, OpenBSD, FreeBSD, Gentoo, and more. </section> <section id="what-else-is-new"> <h3>What else is new?</h3> For more information about the 7.3.19 release, see the <a class="reference external" href="https://doc.pypy.org/en/latest/release-v7.3.19.html#changelog">full changelog</a>. Please update, and continue to help us make pypy better. Cheers, The PyPy Team </section> </section></description><category>release</category><guid>https://www.pypy.org/posts/2025/02/pypy-v7319-release.html</guid><pubDate>Wed, 26 Feb 2025 12:00:00 GMT</pubDate></item><item><title>Low Overhead Allocation Sampling with VMProf in PyPy's GC</title><link>https://www.pypy.org/posts/2025/02/pypy-gc-sampling.html</link><dc:creator>Christoph Jung</dc:creator><description><h3 id="introduction">Introduction</h3> There are many time-based statistical profilers around (like VMProf or py-spy just to name a few). They allow the user to pick a trade-off between profiling precision and runtime overhead. On the other hand there are memory profilers such as <a href="https://github.com/bloomberg/memray">memray</a>. They can be handy for finding leaks or for discovering functions that allocate a lot of memory. Memory profilers typlically save every single allocation a program does. This results in precise profiling, but larger overhead. In this post we describe our experimental approach to low overhead statistical memory profiling. Instead of saving every single allocation a program does, it only saves every nth allocated byte. We have tightly integrated VMProf and the PyPy Garbage Collector to achieve this. The main technical insight is that the check whether an allocation should be sampled can be made free. This is done by folding it into the bump pointer allocator check that the PyPy’s GC uses to find out if it should start a minor collection. In this way the fast path with and without memory sampling are exactly the same. <h3 id="background">Background</h3> To get an insight how the profiler and GC interact, lets take a brief look at both of them first. <h4 id="vmprof">VMProf</h4> <a href="https://github.com/vmprof/vmprof-python">VMProf</a> is a statistical time-based profiler for PyPy. VMProf samples the stack of currently running Python functions a certain user-configured number of times per second. By adjusting this number, the overhead of profiling can be modified to pick the correct trade-off between overhead and precision of the profile. In the resulting profile, functions with huge runtime stand out the most, functions with shorter runtime less so. If you want to get a little more introduction to VMProf and how to use it with PyPy, you may look at <a href="https://pypy.org/posts/2024/05/vmprof-firefox-converter.html">this blog post</a> <h4 id="pypys-gc">PyPy’s GC</h4> PyPy uses a generational incremental copying collector. That means there are two spaces for allocated objects, the nursery and the old-space. Freshly allocated objects will be allocated into the nursery. When the nursery is full at some point, it will be collected and all objects that survive will be tenured i.e. moved into the old-space. The old-space is much larger than the nursery and is collected less frequently and <a href="https://www.pypy.org/posts/2024/03/fixing-bug-incremental-gc.html">incrementally</a> (not completely collected in one go, but step-by-step). The old space collection is not relevant for the rest of the post though. We will now take a look at nursery allocations and how the nursery is collected. <h4 id="bump-pointer-allocation-in-the-nursery">Bump Pointer Allocation in the Nursery</h4> The nursery (a small continuous memory area) utilizes two pointers to keep track from where on the nursery is free and where it ends. They are called <code>nursery_free</code> and <code>nursery_top</code>. When memory is allocated, the GC checks if there is enough space in the nursery left. If there is enough space, the <code>nursery_free</code> pointer will be returned as the start address for the newly allocated memory, and <code>nursery_free</code> will be moved forward by the amount of allocated memory. <img src="https://www.pypy.org/images/2025_02_allocation_sampling_images/nursery_allocation.svg"> <div class="code"><pre class="code literal-block">def allocate(totalsize): # Save position, where the object will be allocated to as result result = gc.nursery_free # Move nursery_free pointer forward by totalsize gc.nursery_free = result + totalsize # Check if this allocation would exceed the nursery if gc.nursery_free &gt; gc.nursery_top: # If it does =&gt; collect the nursery and allocate afterwards result = collect_and_reserve(totalsize) # result is a pointer into the nursery, obj will be allocated there return result def collect_and_reserve(size_of_allocation): # do a minor collection and return the start of the nursery afterwards minor_collection() return gc.nursery_free </pre></div> Understanding this is crucial for our allocation sampling approach, so let us go through this step-by-step. We already saw an example on how an allocation into a non-full nursery will look like. But what happens, if the nursery is (too) full? <img src="https://www.pypy.org/images/2025_02_allocation_sampling_images/nursery_full.svg"> As soon as an object doesn't fit into the nursery anymore, it will be collected. A nursery collection will move all surviving objects into the old-space, so that the nursery is free afterwards, and the requested allocation can be made. <img src="https://www.pypy.org/images/2025_02_allocation_sampling_images/nursery_collected.svg"> (Note that this is still a bit of a simplification.) <h3 id="sampling-approach">Sampling Approach</h3> The last section described how the nursery allocation works normally. Now we'll talk how we integrate the new allocation sampling approach into it. To decide whether the GC should trigger a sample, the sampling logic is integrated into the bump pointer allocation logic. Usually, when there is not enough space in the nursery left to fulfill an allocation request, the nursery will be collected and the allocation will be done afterwards. We reuse that mechanism for sampling, by introducing a new pointer called <code>sample_point</code> that is calculated by <code>sample_point = nursery_free + sample_n_bytes</code> where <code>sample_n_bytes</code> is the number of bytes allocated before a sample is made (i.e. our sampling rate). Imagine we'd have a nursery of 2MB and want to sample every 512KB allocated, then you could imagine our nursery looking like that: <img src="https://www.pypy.org/images/2025_02_allocation_sampling_images/nursery_sampling.svg"> We use the sample point as <code>nursery_top</code>, so that allocating a chunk of 512KB would exceed the nursery top and start a nursery collection. But of course we don't want to do a minor collection just then, so before starting a collection, we need to check if the nursery is actually full or if that is just an exceeded sample point. The latter will then trigger a VMprof stack sample. Afterwards we don't actually do a minor collection, but change <code>nursery_top</code> and immediately return to the caller. The last picture is a conceptual simplification. Only one sampling point exists at any given time. After we created the sampling point, it will be used as nursery top, if exceeded at some point, we will just add <code>sample_n_bytes</code> to that sampling point, i.e. move it forward. Here's how the updated <code>collect_and_reserve</code> function looks like: <div class="code"><pre class="code literal-block">def collect_and_reserve(size_of_allocation): # Check if we exceeded a sample point or if we need to do a minor collection if gc.nursery_top == gc.sample_point: # One allocation could exceed multiple sample points # Sample, move sample_point forward vmprof.sample_now() gc.sample_point += sample_n_bytes # Set sample point as new nursery_top if it fits into the nursery if sample_point &lt;= gc.real_nursery_top: gc.nursery_top = sample_point # Or use the real nursery top if it does not fit else: gc.nursery_top = gc.real_nursery_top # Is there enough memory left inside the nursery if gc.nursery_free + size_of_allocation &lt;= gc.nursery_top: # Yes =&gt; move nursery_free forward gc.nursery_free += size_of_allocation return gc.nursery_free # We did not exceed a sampling point and must do a minor collection, or # we exceeded a sample point but we needed to do a minor collection anyway minor_collection() return gc.nursery_free </pre></div> <h3 id="why-is-the-overhead-low">Why is the Overhead ‘low’</h3> The most important property of our approach is that the bump-pointer fast path is not changed at all. If sampling is turned off, the slow path in <code>collect_and_reserve</code> has three extra instructions for the if at the beginning, but are only a very small amount of overhead, compared to doing a minor collection. When sampling is on, the extra logic in <code>collect_and_reserve</code> gets executed. Every time an allocation exceeds the <code>sample_point</code>, <code>collect_and_reserve</code> will sample the Python functions currently executing. The resulting overhead is directly controlled by <code>sample_n_bytes</code>. After sampling, the <code>sample_point</code> and <code>nursery_top</code> must be set accordingly. This will be done once after sampling in <code>collect_and_reserve</code>. At some point a nursery collection will free the nursery and set the new <code>sample_point</code> afterwards. That means that the overhead mostly depends on the sampling rate and the rate at which the user program allocates memory, as the combination of those two factors determines the amount of samples. Since the sampling rate can be adjusted from as low as 64 Byte to a theoretical maximum of ~4 GB (at the moment), the tradeoff between number of samples (i.e. profiling precision) and overhead can be completely adjusted. We also suspect linkage between user program stack depth and overhead (a deeper stack takes longer to walk, leading to higher overhead), especially when walking the C call stack to. <h3 id="sampling-rates-bigger-than-the-nursery-size">Sampling rates bigger than the nursery size</h3> The nursery usually has a size of a few megabytes, but profiling long-runningor larger applications with tons of allocations could result in very high number of samples per second (and thus overhead). To combat that it is possible to use sampling rates higher than the nursery size. The sampling point is not limited by the nursery size, but if it is 'outside' the nursery (e.g. because <code>sample_n_bytes</code> is set to twice the nursery size) it won't be used as <code>nursery_top</code> until it 'fits' into the nursery. <img src="https://www.pypy.org/images/2025_02_allocation_sampling_images/nursery_sampling_larger_than_nursery.svg"> After every nursery collection, we'd usually set the <code>sample_point</code> to <code>nursery_free + sample_n_bytes</code>, but if it is larger than the nursery, then the amount of collected memory during the last nursery collection is subtracted from <code>sample_point</code>. <img src="https://www.pypy.org/images/2025_02_allocation_sampling_images/nursery_sampling_larger_than_nursery_post_minor.svg"> At some point the <code>sample_point</code> will be smaller than the nursery size, then it will be used as <code>nursery_top</code> again to trigger a sample when exceeded. <h3 id="differences-to-time-based-sampling">Differences to Time-Based Sampling</h3> As mentioned in the introduction, time-based sampling ‘hits’ functions with high runtime, and allocation-sampling ‘hits’ functions allocating much memory. But are those always different functions? The answer is: sometimes. There can be functions allocating lots of memory, that do not have a (relative) high runtime. Another difference to time-based sampling is that the profiling overhead does not solely depend on the sampling rate (if we exclude a potential stack-depth - overhead correlation for now) but also on the amount of memory the user code allocates. Let us look at an example: If we’d sample every 1024 Byte and some program A allocates 3 MB and runs for 5 seconds, and program B allocates 6 MB but also runs for 5 seconds, there will be ~3000 samples when profiling A, but ~6000 samples when profiling B. That means we cannot give a ‘standard’ sampling rate like time-based profilers use to do (e.g. vmprof uses ~1000 samples/s for time sampling), as the number of resulting samples, and thus overhead, depends on sampling rate and amount of memory allocated by the program. For testing and benchmarking, we usually started with a sampling rate of 128Kb and then halved or doubled that (multiple times) depending on sample counts, our need for precision (and size of the profile). <h3 id="evaluation">Evaluation</h3> <h4 id="overhead">Overhead</h4> Now let us take a look at the allocation sampling overhead, by profiling some benchmarks. The x-axis shows the sampling rate, while the y-axis shows the overhead, which is computed as <code>runtime_with_sampling / runtime_without_sampling</code>. All benchmarks were executed five times on a PyPy with JIT and native profiling enabled, so that every dot in the plot is one run of a benchmark. <img src="https://www.pypy.org/images/2025_02_allocation_sampling_images/as_overhead.png"> As you probably expected, the Overhead drops with higher allocation sampling rates. Reaching from as high as ~390% for 32kb allocation sampling to as low as &lt; 10% for 32mb. Let me give one concrete example: One run of the microbenchmark at 32kb sampling took 15.596 seconds and triggered 822050 samples. That makes a ridiculous amount of <code>822050 / 15.596 = ~52709</code> samples per second. There is probably no need for that amount of samples per second, so that for 'real' application profiling a much higher sampling rate would be sufficient. Let us compare that to time sampling. This time we ran those benchmarks with 100, 1000 and 2000 samples per second. <img src="https://www.pypy.org/images/2025_02_allocation_sampling_images/ts_overhead.png"> The overhead varies with the sampling rate. Both with allocation and time sampling, you can reach any amount of overhead and any level of profiling precision you want. The best approach probably is to just try out a sampling rate and choose what gives you the right tradeoff between precision and overhead (and disk usage). The benchmarks used are: microbenchmark <ul> <li><a href="https://github.com/Cskorpion/microbenchmark">https://github.com/Cskorpion/microbenchmark</a></li> <li><code>pypy microbench.py 65536</code></li> </ul> gcbench <ul> <li><a href="https://github.com/pypy/pypy/blob/main/rpython/translator/goal/gcbench.py">https://github.com/pypy/pypy/blob/main/rpython/translator/goal/gcbench.py</a></li> <li>print statements removed</li> <li><code>pypy gcbench.py 1</code></li> </ul> pypy translate step <ul> <li>first step of the pypy translation (annotation step)</li> <li><code>pypy path/to/rpython --opt=0 --cc=gcc --dont-write-c-files --gc=incminimark --annotate path/to/pypy/goal/targetpypystandalone.py</code></li> </ul> interpreter pystone <ul> <li>pystone benchmark on top of an interpreted pypy on top of a translated pypy</li> <li><code>pypy path/to/pypy/bin/pyinteractive.py -c "import test.pystone; test.pystone.main(1)"</code></li> </ul> All benchmarks executed on: <ul> <li>Kubuntu 24.04</li> <li>AMD Ryzen 7 5700U</li> <li>24gb DDR4 3200MHz (dual channel)</li> <li> SSD benchmarking at read: 1965 MB/s, write: 227 MB/s <ul> <li>Sequential 1MB 1 Thread 8 Queues</li> </ul> </li> <li> Self built PyPy with allocation sampling features <ul> <li><a href="https://github.com/Cskorpion/pypy/tree/gc_allocation_sampling_u_2.7">https://github.com/Cskorpion/pypy/tree/gc_allocation_sampling_u_2.7</a></li> </ul> </li> <li> Modified VMProf with allocation sampling support <ul> <li><a href="https://github.com/Cskorpion/vmprof-python/tree/pypy_gc_allocation_sampling">https://github.com/Cskorpion/vmprof-python/tree/pypy_gc_allocation_sampling</a></li> </ul> </li> </ul> <h4 id="example">Example</h4> We have also modified <a href="https://github.com/Cskorpion/vmprof-firefox-converter/tree/allocation_sampling">vmprof-firefox-converter</a> to show the allocation samples in the Firefor Profiler UI. With the techniques from this post, the output looks like this: <img src="https://www.pypy.org/images/2025_02_allocation_sampling_images/allocation_sampling_call_tree.png"> While this view is interesting, it would be even better if we could also see what types of objects are being allocated in these functions. We will take about how to do this in a future blog post. <h3 id="conclusion">Conclusion</h3> In this blog post we introduced allocation sampling for PyPy by going through the technical aspects and the corresponding overhead. In a future blog post, we are going to dive into the actual usage of allocation sampling with VMProf, and show an example case study. That will be accompanied by some new improvements and additional features, like extracting the type of an object that triggered a sample. So far all this work is still experimental and happening on PyPy branches but we hope to get the technique stable enough to merge it to main and ship it with PyPy eventually. -- Christoph Jung and CF Bolz-Tereick</description><category>gc</category><category>profiling</category><category>vmprof</category><guid>https://www.pypy.org/posts/2025/02/pypy-gc-sampling.html</guid><pubDate>Tue, 25 Feb 2025 10:16:00 GMT</pubDate></item><item><title>PyPy v7.3.18 release</title><link>https://www.pypy.org/posts/2025/02/pypy-v7318-release.html</link><dc:creator>mattip</dc:creator><description><section id="pypy-v7-3-18-release-of-python-2-7-3-10-and-3-11-beta"> <h2>PyPy v7.3.18: release of python 2.7, 3.10 and 3.11 beta</h2> The PyPy team is proud to release version 7.3.18 of PyPy. This release includes a python 3.11 interpreter. We are labelling it "beta" because it is the first one. In the next release we will drop 3.10 and remove the "beta" label. There are a particularly large set of bugfixes in this release thanks to @devdanzin using fusil on the 3.10 builds, originally written by Victor Stinner. Other significant changes: <ul class="simple"> <li>We have updated libffi shipped in our portable builds. We also now statically link to libffi where possible which reduces the number of shared object dependencies.</li> <li>We have added code to be able to show the native function names when profiling with VMProf. So far only Linux supports this feature.</li> <li>We have added a <a class="reference external" href="https://peps.python.org/pep-0768/">PEP 768</a>-inspired remote debugging facility.</li> <li>The HPy backend has been updated to latest HPy HEAD</li> </ul> The release includes three different interpreters: <ul class="simple"> <li>PyPy2.7, which is an interpreter supporting the syntax and the features of Python 2.7 including the stdlib for CPython 2.7.18+ (the <code class="docutils literal">+</code> is for backported security updates)</li> <li>PyPy3.10, which is an interpreter supporting the syntax and the features of Python 3.10, including the stdlib for CPython 3.10.16.</li> <li>PyPy3.11, which is an interpreter supporting the syntax and the features of Python 3.11, including the stdlib for CPython 3.11.11.</li> </ul> The interpreters are based on much the same codebase, thus the triple release. This is a micro release, all APIs are compatible with the other 7.3 releases. It follows after 7.3.17 release on August 28, 2024. We recommend updating. You can find links to download the releases here: <blockquote> <a class="reference external" href="https://pypy.org/download.html">https://pypy.org/download.html</a> </blockquote> We would like to thank our donors for the continued support of the PyPy project. If PyPy is not quite good enough for your needs, we are available for <a class="reference external" href="https://www.pypy.org/pypy-sponsors.html">direct consulting</a> work. If PyPy is helping you out, we would love to hear about it and encourage submissions to our <a class="reference external" href="https://pypy.org/blog">blog</a> via a pull request to <a class="reference external" href="https://github.com/pypy/pypy.org">https://github.com/pypy/pypy.org</a> We would also like to thank our contributors and encourage new people to join the project. PyPy has many layers and we need help with all of them: bug fixes, <a class="reference external" href="https://doc.pypy.org/">PyPy</a> and <a class="reference external" href="https://rpython.readthedocs.org">RPython</a> documentation improvements, or general <a class="reference external" href="https://doc.pypy.org/en/latest/project-ideas.html">help</a> with making RPython's JIT even better. If you are a python library maintainer and use C-extensions, please consider making a <a class="reference external" href="https://hpyproject.org/">HPy</a> / <a class="reference external" href="https://cffi.readthedocs.io">CFFI</a> / <a class="reference external" href="https://cppyy.readthedocs.io">cppyy</a> version of your library that would be performant on PyPy. In any case, both <a class="reference external" href="https://github.com/joerick/cibuildwheel">cibuildwheel</a> and the <a class="reference external" href="https://github.com/matthew-brett/multibuild">multibuild system</a> support building wheels for PyPy. <section id="vmprof-native-symbol-names"> <h3>VMProf Native Symbol Names</h3> When running VMProf profiling with native profiling enabled, PyPy did so far not produce function names for C functions. The output looked like this: <pre class="literal-block">pypy -m vmprof ~/projects/gitpypy/lib-python/2.7/test/pystone.py Pystone(1.1) time for 50000 passes = 0.0109887 This machine benchmarks at 4.55011e+06 pystones/second vmprof output: %: name: location: 100.0% entry_point &lt;builtin&gt;/app_main.py:874 100.0% run_command_line &lt;builtin&gt;/app_main.py:601 100.0% run_toplevel &lt;builtin&gt;/app_main.py:93 100.0% _run_module_as_main /home/user/bin/pypy-c-jit-170203-99a72243b541-linux64/lib-python/2.7/runpy.py:150 100.0% _run_code /home/user/bin/pypy-c-jit-170203-99a72243b541-linux64/lib-python/2.7/runpy.py:62 100.0% &lt;module&gt; /home/user/bin/pypy-c-jit-170203-99a72243b541-linux64/site-packages/vmprof/__main__.py:1 100.0% main /home/user/bin/pypy-c-jit-170203-99a72243b541-linux64/site-packages/vmprof/__main__.py:30 100.0% run_path /home/user/bin/pypy-c-jit-170203-99a72243b541-linux64/lib-python/2.7/runpy.py:238 100.0% _run_module_code /home/user/bin/pypy-c-jit-170203-99a72243b541-linux64/lib-python/2.7/runpy.py:75 100.0% &lt;module&gt; /home/user/projects/gitpypy/lib-python/2.7/test/pystone.py:3 100.0% main /home/user/projects/gitpypy/lib-python/2.7/test/pystone.py:60 100.0% pystones /home/user/projects/gitpypy/lib-python/2.7/test/pystone.py:67 100.0% Proc0 /home/user/projects/gitpypy/lib-python/2.7/test/pystone.py:79 76.9% &lt;unknown code&gt; 69.2% &lt;unknown code&gt; 53.8% &lt;unknown code&gt; 53.8% &lt;unknown code&gt; 46.2% &lt;unknown code&gt; 46.2% &lt;unknown code&gt; 38.5% &lt;unknown code&gt; 38.5% Proc8 /home/user/projects/gitpypy/lib-python/2.7/test/pystone.py:212 30.8% &lt;unknown code&gt; ...</pre> We can now symbolify these C functions and give function names and which shared library they come from, at least on Linux: <pre class="literal-block">Pystone(1.1) time for 50000 passes = 0.218967 This machine benchmarks at 228345 pystones/second vmprof output: %: name: location: 100.0% entry_point &lt;builtin&gt;/app_main.py:889 100.0% run_command_line &lt;builtin&gt;/app_main.py:616 100.0% run_toplevel &lt;builtin&gt;/app_main.py:95 100.0% _run_module_as_main /home/user/projects/gitpypy/lib-python/2.7/runpy.py:150 100.0% _run_code /home/user/projects/gitpypy/lib-python/2.7/runpy.py:62 100.0% &lt;module&gt; /home/user/projects/gitpypy/site-packages/vmprof/__main__.py:1 100.0% main /home/user/projects/gitpypy/site-packages/vmprof/__main__.py:30 100.0% run_module /home/user/projects/gitpypy/lib-python/2.7/runpy.py:179 100.0% _run_module_code /home/user/projects/gitpypy/lib-python/2.7/runpy.py:75 100.0% &lt;module&gt; /home/user/projects/gitpypy/lib-python/2.7/test/pystone.py:3 100.0% main /home/user/projects/gitpypy/lib-python/2.7/test/pystone.py:60 100.0% pystones /home/user/projects/gitpypy/lib-python/2.7/test/pystone.py:67 100.0% Proc0 /home/user/projects/gitpypy/lib-python/2.7/test/pystone.py:79 95.5% n:pypy_g_execute_frame:0:pypy-c 91.4% n:pypy_g_PyFrame_dispatch:0:pypy-c 63.8% n:pypy_g_PyFrame_dispatch_bytecode:0:pypy-c 49.8% Proc1 /home/user/projects/gitpypy/lib-python/2.7/test/pystone.py:137 17.6% copy /home/user/projects/gitpypy/lib-python/2.7/test/pystone.py:53 13.6% n:pypy_g_PyFrame_CALL_FUNCTION:0:pypy-c 10.4% Proc8 /home/user/projects/gitpypy/lib-python/2.7/test/pystone.py:212 8.6% n:pypy_g_STORE_ATTR_slowpath:0:pypy-c</pre> This becomes even more useful when using the <a class="reference external" href="https://github.com/Cskorpion/vmprof-firefox-converter/">VMProf Firefox converter</a>, which uses the Firefox Profiler Web UI to visualize profiling output: <img alt="/images/2025-vmprof-firefox.png" src="https://www.pypy.org/images/2025-vmprof-firefox.png"> </section> <section id="what-is-pypy"> <h3>What is PyPy?</h3> PyPy is a Python interpreter, a drop-in replacement for CPython It's fast (<a class="reference external" href="https://speed.pypy.org">PyPy and CPython</a> performance comparison) due to its integrated tracing JIT compiler. We also welcome developers of other <a class="reference external" href="https://rpython.readthedocs.io/en/latest/examples.html">dynamic languages</a> to see what RPython can do for them. We provide binary builds for: <ul class="simple"> <li>x86 machines on most common operating systems (Linux 32/64 bits, Mac OS 64 bits, Windows 64 bits)</li> <li>64-bit ARM machines running Linux (<code class="docutils literal">aarch64</code>) and macos (<code class="docutils literal">macos_arm64</code>).</li> </ul> PyPy supports Windows 32-bit, Linux PPC64 big- and little-endian, Linux ARM 32 bit, RISC-V RV64IMAFD Linux, and s390x Linux but does not release binaries. Please reach out to us if you wish to sponsor binary releases for those platforms. Downstream packagers provide binary builds for debian, Fedora, conda, OpenBSD, FreeBSD, Gentoo, and more. </section> <section id="what-else-is-new"> <h3>What else is new?</h3> For more information about the 7.3.18 release, see the <a class="reference external" href="https://doc.pypy.org/en/latest/release-v7.3.18.html#changelog">full changelog</a>. Please update, and continue to help us make pypy better. Cheers, The PyPy Team </section> </section></description><category>release</category><guid>https://www.pypy.org/posts/2025/02/pypy-v7318-release.html</guid><pubDate>Thu, 06 Feb 2025 12:00:00 GMT</pubDate></item><item><title>Musings on Tracing in PyPy</title><link>https://www.pypy.org/posts/2025/01/musings-tracing.html</link><dc:creator>CF Bolz-Tereick</dc:creator><description>Last summer, <a href="https://cs.brown.edu/~sk/">Shriram Krishnamurthi</a> <a href="https://twitter.com/ShriramKMurthi/status/1818009884484583459">asked on Twitter</a>: <blockquote> "I'm curious what the current state of tracing JITs is. They used to be all the rage for a while, then I though I heard they weren't so effective, then I haven't heard of them at all. Is the latter because they are ubiquitous, or because they proved to not work so well?" </blockquote> I replied with my personal (pretty subjective) opinions about the question in a lengthy Twitter thread (which also spawned an even lengthier discussion). I wanted to turn what I wrote there into a blog post to make it more widely available (Twitter is no longer easily consumable without an account), and also because I'm mostly not using Twitter anymore. The blog post i still somewhat terse, I've written a small background section and tried to at least add links to further information. Please ask in the comments if something is particularly unclear. <h3 id="background">Background</h3> I'll explain a few of the central terms of the rest of the post. JIT compilers are compilers that do their work at runtime, interleaved (or concurrent with) the execution of the program. There are (at least) two common general styles of JIT compiler architectures. The most common one is that of a method-based JIT, which will compile one method or function at a time. Then there are tracing JIT compilers, which generate code by tracing the execution of the user's program. They often focus on loops as their main unit of compilation. Then there is the distinction between a "regular" JIT compiler and that of a meta-JIT. A regular JIT is built to compile one specific source language to machine code. A meta-JIT is a framework for building JIT compilers for a variety of different languages, reusing as much machinery as possible between the different implementation. <h3 id="personal-and-project-context">Personal and Project Context</h3> Some personal context: my perspective is informed by nearly <a href="https://mail.python.org/archives/list/pypy-dev@python.org/thread/TZM37YJ733G445R6JGTV26333RQEPLRX/">two decades</a> of work on PyPy. PyPy's implementation language, <a href="https://rpython.readthedocs.io/">RPython</a>, has support for a meta-JIT, which allows it to reuse its JIT infrastructure for the various Python versions that we support (currently we do releases of PyPy2.7 and PyPy3.10 together). Our meta-JIT infrastructure has been used for some experimental different languages like: <ul> <li>PyPy's <a href="https://pypy.org/posts/2010/11/pypy-14-ouroboros-in-practice-5437628000869417542.html#more-highlights">regular expression engine</a></li> <li><a href="https://github.com/SOM-st/PySOM">RPySom</a>, a tiny Smalltalk</li> <li><a href="https://github.com/topazproject/topaz">Ruby</a></li> <li><a href="https://github.com/hippyvm/hippyvm">PHP</a></li> <li><a href="https://dl.acm.org/doi/10.1145/1836089.1836102">Prolog</a>,</li> <li><a href="https://dl.acm.org/doi/10.1145/2784731.2784740">Racket</a>,</li> <li>a <a href="https://drops.dagstuhl.de/storage/00lipics/lipics-vol056-ecoop2016/LIPIcs.ECOOP.2016.4/LIPIcs.ECOOP.2016.4.pdf">database (SQLite)</a></li> <li><a href="https://www.youtube.com/watch?v=fZj3uljJl_k">Lox</a>, the language of <a href="https://craftinginterpreters.com/">Crafting Interpreters</a></li> <li>an <a href="https://docs.pydrofoil.org/en/latest/">ARM and RISC-V emulator</a></li> <li>and many more</li> </ul> Those implementations had various degrees of maturity and many of them are research software and aren't maintained any more. PyPy gives itself the goal to try to be extremely compatible with all the quirks of the Python language. We don't change the Python language to make things easier to compile and we support the introspection and debugging features of Python. We try very hard to have no opinions on language design. The CPython core developers come up with the semantics, we somehow deal with them. <h3 id="meta-tracing">Meta-tracing</h3> PyPy started using a <a href="https://en.wikipedia.org/wiki/Tracing_just-in-time_compilation">tracing JIT</a> approach not because we thought method-based just-in-time compilers are bad. Historically we <a href="https://foss.heptapod.net/pypy/extradoc/-/blob/branch/extradoc/eu-report/D08.2_JIT_Compiler_Architecture-2007-05-01.pdf">had tried</a> to implement a method-based meta-JIT that was using partial evaluation (we wrote three or four method-based prototypes that all weren't as good as we hoped). After all those <a href="https://pypy.org/posts/2008/10/sprint-discussions-jit-generator-3301578822967655604.html">experiments failed</a> we switched to the <a href="https://dl.acm.org/doi/10.1145/1565824.1565827">tracing approach</a>, and only at this point did our meta-JIT start producing interesting performance. In the meta-JIT context tracing has good properties, because tracing has relatively understandable behavior and its easy(ish) to tweak how things work <a href="https://dl.acm.org/doi/10.1145/2069172.2069181">with extra annotations in the interpreter source</a>. Another reason why meta-tracing often works well for PyPy is that it can often slice through the complicated layers of Python quite effectively and remove a lot of overhead. Python is often described as simple, but I think that's actually a misconception. On the implementation level it's a very big and complicated language, and it is also continuously getting new features every year (the language is quite a bit more complicated than Javascript, for example<a class="footnote-ref" href="https://www.pypy.org/posts/2025/01/musings-tracing.html#fn:help">1</a>). <h4 id="truffle">Truffle</h4> Later <a href="https://dl.acm.org/doi/abs/10.1145/2509578.2509581">Truffle</a> came along and made a method-based meta-JIT using partial evaluation work. However Truffle (and <a href="https://www.oracle.com/java/graalvm/">Graal</a>) has had significantly more people working on it and much more money invested. In addition, it at first required a quite specific style of <a href="https://dl.acm.org/doi/10.1145/2384577.2384587">AST-based interpreters</a> (in the last few years they have also added support for bytecode-based interpreters). It's still my impression that getting similar results with Truffle is <a href="https://stefan-marr.de/downloads/oopsla15-marr-ducasse-meta-tracing-vs-partial-evaluation.pdf">more work for language implementers</a> than with RPython, and the <a href="https://arxiv.org/pdf/1602.00602">warmup</a> of Truffle can often pretty bad. But Truffle is definitely an existence proof that meta-JITs don't have to be based on tracing. <h3 id="tracing-the-good">Tracing, the good</h3> Let's now actually get to he heart of Shriram's question and discuss some of the advantages of tracing that go beyond the ease of using tracing for a meta-JIT. Tracing allows for doing very aggressive <a href="https://www.cs.fsu.edu/~xyuan/INTERACT-15/papers/paper11.pdf">partial inlining</a>, Following just the hot path through lots of layers of abstraction is obviously often really useful for generating fast code. It's definitely possible to achieve the same effect in a method-based context with <a href="https://dl.acm.org/doi/pdf/10.1145/117954.117955">path splitting</a>. But it requires a lot more implementation work and is not trivial, because the path <a href="https://dl.acm.org/doi/10.1145/504282.504295">execution counts</a> of inlined functions can often be very call-site dependent. Tracing, on the other hand, gives you call-site dependent path splitting "for free". (The aggressive partial inlining and path splitting is even more important in the meta-tracing context of PyPy, where some of inlined layers are a part of the language runtime, and where rare corner cases that are never executed in practice are everywhere.) Another advantage of tracing is that it makes a number of optimizations really easy to implement, because there are (to first approximation) no control flow merges. This makes all the optimizations that we do (super-)<a href="https://en.wikipedia.org/wiki/Optimizing_compiler#Local_vs._global_scope">local optimizations</a>, that operate on a single (very long) basic block. This allows the JIT to do the optimizations in exactly one forwards and one backwards pass. An example is our <a href="https://dl.acm.org/doi/10.1145/1929501.1929508">allocation removal</a>/partial escape analysis pass, which is <a href="https://pypy.org/posts/2022/10/toy-optimizer-allocation-removal.html">quite simple</a>, whereas the <a href="https://ssw.jku.at/Teaching/PhDTheses/Stadler/Thesis_Stadler_14.pdf">version for general control flow</a> has a lot more complexity, particularly in its handling of loops. This ease of implementation of optimizations allowed us to implement some pretty decent optimizations. Our allocation removal, the way PyPy's JIT can reason about the heap, about dictionary accesses, about properties of functions of the runtime, about the range and <a href="https://pypy.org/posts/2024/08/toy-knownbits.html">known bits of integer variables</a> is all quite solid. <h3 id="tracing-the-bad">Tracing, the bad</h3> Tracing also comes with a significant number of downsides. Probably the biggest one is that it tends to have big performance cliffs (PyPy certainly has them, and other tracing JITs such as TraceMonkey had them too). In my experience the 'good' cases of tracing are really good, but if something goes wrong you are annoyed and performance can become a lot slower. With a simple method JIT the performance is often much more "even". Another set of downsides is that tracing has a number of corner cases and "weird" behaviour in certain situations. Questions such as: - When do you stop inlining? - What happens when you <a href="https://mail.python.org/archives/list/pypy-dev@python.org/thread/GQQ7ABUFHGEAHWN7RQZM6D54CDROQINR/">trace recursion</a>? - What happens if your traces are <a href="https://pypy.org/posts/2021/09/jit-auto-generated-code.html">consistently too long, even without inlining</a>? - and so on... Some of these problems can be solved by adding heuristics to the tracing JIT, but doing so loses a lot of the simplicity of tracing again. There are also some classes of programs that tend to generally perform quite poorly when they are executed by a tracing JIT, bytecode interpreters in particularly, and other extremely unpredictably branchy code. This is because the core assumption of the tracing jit "loops take similar control flow paths" is just really wrong in the case of interpreters. <h3 id="discussion">Discussion</h3> The Twitter thread spawned quite a bit of discussion, please look at the original thread for all of the comments. Here are three that I wanted to highlight: <blockquote> "This is a really great summary. Meta-tracing is probably the one biggest success story. I think it has to do with how big and branchy the bytecode implementations are for typical dynamic languages; the trace captures latent type feedback naturally. There is an upper limit, tho." </blockquote> <a href="https://twitter.com/TitzerBL/status/1818385622203298265">Ben Titzer</a> I agree with this completely. The complexity of Python bytecodes is a big factor for why meta tracing works well for us. But also in Python there are many builtin types (collection types, types that form the <a href="https://en.wikipedia.org/wiki/Metaobject#Metaobject_protocol">meta-object protocol</a> of Python, standard library modules implemented in C/RPython) and tracing operations on them is very important too, for good performance. <hr> <blockquote> "I think Mozilla had a blog post talking more about the difficulty with TraceMonkey, could only find this one: https://blog.mozilla.org/nnethercote/category/jagermonkey/" </blockquote> <a href="https://twitter.com/smarr/status/1818600052752797990">Stefan Marr</a> <blockquote> "imo doing tracing for JS is really hard mode, because the browser is so incredibly warmup-sensitive. IIRC tracemonkey used a really low loop trip count (single-digit?) to decide when to start tracing (pypy uses &gt;1000). the JS interpreters of the time were also quite slow." </blockquote> <a href="https://twitter.com/cfbolz/status/1818609594219811245">me</a> In the meantime there were some more reminiscences about tracing in Javascript by <a href="https://www.youtube.com/live/_VF3pISRYRc?t=24797s">Shu-Yu Guo in a panel discussion</a> and by <a href="https://kfogel.org/notice/AngH0uqyJl231yLLOa">Jason Orendorff on Mastodon</a>. <hr> <blockquote> "There are a number of corner cases you have to deal with in a tracing JIT. It's unfortunately not as simple and easy as the initial papers would have you believe. One example is how would you deal with a loop inside a loop? Is your tracing now recursive? There's been some research work on trace stitching to deal with trace explosion but it does add complexity. With the increase in complexity, I think most industrial VM developers would rather pick tried-and-true method-based JITs that are well understood." </blockquote> <a href="https://twitter.com/Love2Code/status/1818292516753383644">Maxime Chevalier</a> <h3 id="conclusion">Conclusion</h3> Given access to enough developers and in the context of "normal" jitting (ie not meta-jitting) it's very unclear to me that you should use tracing. It makes more sense to rather spend effort on a solid control-flow-graph-based baseline and then try to get some of the good properties of tracing on top (path splitting, partial inlining, partial escape analysis, etc). For PyPy with its meta-JIT (and the fact that we don't have particularly much funding nor people) I still think tracing was/is a relatively pragmatic choice. When I talked with <a href="https://samth.github.io/">Sam Tobin-Hochstadt</a> about this topic recently he characterized it like this: "tracing is a labor-saving device for compiler authors". Performance-wise PyPy is still quite hard to beat in the cases where it works well (i.e. pure Python code that doesn't use too many C modules, which are <a href="https://pypy.org/posts/2018/09/inside-cpyext-why-emulating-cpython-c-8083064623681286567.html">supported but slow in PyPy</a>). In general, there are very few JITs for Python (particularly with the constraint of not being "allowed" to change the language), the most competitive other ones are <a href="https://www.graalvm.org/python/">GraalPy</a>, also based on a meta-JIT approach. Instagram is running on <a href="https://github.com/facebookincubator/cinder/">Cinder</a> and also CPython has <a href="https://tonybaloney.github.io/posts/python-gets-a-jit.html">grown a JIT recently</a> which was part of the recent <a href="https://docs.python.org/3.13/whatsnew/3.13.html#an-experimental-just-in-time-jit-compiler">3.13 release, but only as an off-by-default build option</a>, so I'm very excited about how Python's performance will develop in the next years! <div class="footnote"> <hr> <ol> <li id="fn:help"> (A side point: people who haven't worked on Python tend to underestimate its complexity and pace of development. A pet peeve of mine is C++ compiler devs/static analysis/Javascript people/other well-meaning communities coming with statements like "why don't you just..." 🤷‍♀️) <a class="footnote-backref" href="https://www.pypy.org/posts/2025/01/musings-tracing.html#fnref:help" title="Jump back to footnote 1 in the text">↩</a> </li> </ol> </div></description><guid>https://www.pypy.org/posts/2025/01/musings-tracing.html</guid><pubDate>Sun, 05 Jan 2025 17:01:09 GMT</pubDate></item><item><title>Towards PyPy3.11 - an update</title><link>https://www.pypy.org/posts/2025/01/towards-pypy311-an-update.html</link><dc:creator>mattip</dc:creator><description>We<a class="footnote-ref" href="https://www.pypy.org/posts/2025/01/towards-pypy311-an-update.html#fn:0">1</a> are steadily working towards a Python 3.11 interpreter, which will be part of the upcoming PyPy 7.3.18 release. Along with that, we also recently updated <a href="https://speed.pypy.org">speed.pypy.org</a> to compare PyPy's performance to CPython 3.11 (it used to be CPython 3.7).  <h2 id="why-is-there-no-pypy-for-python-311">Why is there no PyPy for Python 3.11?</h2> TL;DR: we are working on it and hopefully will have a beta version soon We started by merging the <a href="https://peps.python.org/pep-0654/">exception groups</a> work by Nico Rittinghaus, merging the CPython stdlib for Python 3.11.9, and updating the regex engine to handle atomic groupings. I think these were the largest changes needed to support Python3.11, maybe I missed something else? We then created a <a href="https://github.com/pypy/pypy/milestone/15">milestone</a> with many of the changes that might not be caught in testing. As of today that milestone stands at 24/40 issues closed, 16 open. This is in addition to the <a href="https://github.com/pypy/pypy/milestone/22">v7.3.18 milestone</a>, which has 11/23 issues closed, 12 open. We also updated our infrastructure to run nightly <a href="https://buildbot.pypy.org/summary?branch=py3.11">buildbot test of the py3.11</a> branch, including adding py3.11 to our benchmarking site <a href="https://speed.pypy.org/">speed.pypy.org</a>. Then the real work started: fixing these milestone issues and failing stdlib tests. Some of the changes were cosmetic changes to error messages, some were more involved changes to the interpreter to behave more like CPython. For instance, <code>hex(x)</code> where <code>x</code> is an int calls <code>long.format(x, "#x")</code> on CPython where in PyPy we used <code>x.__format__("#x")</code>. This subtle difference caused a failure in the <code>repr</code> of <code>IntEnum</code>. Tracking down problems like these takes time. We are now down to about 250 failing stdlib tests, with thousands passing. For comparison, PyPy3.10, first released in June 2023, still has around 100 failing stdlib tests. <h3 id="c-extension-support">C-extension support</h3> PyPy supports the Python C-API via a cpyext compatibility layer. We "mangle" the CPython symbols to add an extra <code>Py</code> to prevent loading CPython c-extension modules into PyPy, since the ABI is different. So a function like <code>PyLong_FromLong</code> will be exported from the c shared object as <code>PyPyLong_FromLong</code>. One of my long-standing goals is to remove this mangling, but it then requires that our c declarations, inlined functions, and macros in the headers match 1:1 the CPython headers. We can get by with not declaring and implementing parts of the interfaces, but what is declared must be identical. This is a long-term project, with each release the headers get closer to the CPython versions. I hope to concentrate on the PyUnicode interfaces for this release. Note that the time to do this is before a new version release. Once the version is released, we cannot change the headers significantly. <h3 id="so-what-is-left">So what is left?</h3> Summarizing the milestones and other things to do: <ul> <li>Update to the recently released 3.11.11 stdlib</li> <li>Make sure <a href="https://vmprof.readthedocs.io/en/latest/">vmprof</a> works</li> <li>Update the <code>time</code> module to use more <code>MONOTONIC_CLOCK</code>, implement <code>time.sleep</code> differently, and clean up the many duplicate <code>time</code> like interfaces we have across the codebase. We have the <code>time</code> module, some time routines in the <code>_threading</code> module<code>and RPython threading support in the</code>rpython/rlib` code. We should also make sure we are using 64-bit time interfaces.</li> <li>Decide whether zero-cost exceptions gain us in performance and whether we should implement them even if they do not improve performance.</li> <li>Update our <a href="http://hpyproject.org/">hpy</a> backend to the latest HEAD, which would allow running the <a href="https://github.com/hpyproject/numpy-hpy/tree/graal-team/hpy#readme">hpy numpy fork</a></li> <li>Reintegrate the pure-python pyrepl libbrary from CPython 3.13.</li> </ul> What else did I forget? <h2 id="why-did-the-benchmark-results-get-worse-on-speedpypyorg">Why did the benchmark results get worse on speed.pypy.org?</h2> TL;DR: running a benchmark site is hard. Something changed in the benchmark runner, and suddenly benchmarks got 10-15% slower. PyPy run an instance of <a href="https://github.com/python/codespeed/tree/speed.pypy.org">codepseed</a> with a very old <a href="https://foss.heptapod.net/pypy/benchmarks">benchmarking suite</a> that can still run Python2 (remember: that is the language of <a href="https://rpython.readthedocs.io/en/latest/">RPython</a> underlying PyPy). The site runs on PSA infrastructure and the benchmarking machine is generously sponsored by <a href="https://baroquesoftware.com/">Baroque Software</a>. On Nov 9, there was a sudden jump in benchmarking times. For instance <a href="https://speed.pypy.org/timeline/#/?exe=21,8&amp;base=10+2622&amp;ben=float&amp;revs=50&amp;equid=off&amp;quarts=on&amp;extr=on&amp;env=3">here</a> is the result for the <code>float</code> benchmark. This happened across various benchmark runs: both PyPy2.7 and PyPy3.11alpha, with and without the JIT. After spending some time rerunning various benchmarks, I could only conclude the machine itself had gotten slower, maybe due to some security update in the linux kernel, maybe some change in the hosting platform. This "broke" the front-page comparsison: suddenly "latest" is much slower than the historic benchmarks run previous to the changes in the machine. That page also recently (as of last week) uses CPython 3.11 as a baseline for comparison, where previously it used CPython3.7. It is common knowledge that the newer CPython versions are faster, and we see this now quite clearly. Diving into individual benchmarks, we can see that ones where PyPy-with-a-jit was comparable to CPython3.7 seem to be the ones that CPython3.11 improved greatly. Looking at a <a href="https://speed.pypy.org/comparison/?exe=22%2BL%2Bpy3.11%2C2%2B2360%2C2%2B3893&amp;ben=1%2C34%2C58%2C63%2C60%2C27%2C61%2C2%2C25%2C57%2C3%2C46%2C4%2C5%2C41%2C42%2C22%2C44%2C6%2C59%2C39%2C7%2C8%2C65%2C45%2C23%2C62%2C66%2C24%2C9%2C10%2C47%2C48%2C49%2C50%2C51%2C11%2C12%2C13%2C40%2C14%2C69%2C15%2C70%2C67%2C68%2C64%2C35%2C36%2C37%2C38%2C16%2C52%2C54%2C55%2C53%2C56%2C28%2C30%2C32%2C29%2C33%2C17%2C18%2C19%2C20%2C43&amp;env=3&amp;hor=true&amp;bas=2%2B2360&amp;chart=normal+bars">comparison of the runs</a> this can be seen in benchmarks like deltablue and the sqlalchemy family. So the new graph has more lines that extend past 1.5 than the old graph. New graph with PyPy3.11 <img alt="new graph" src="https://www.pypy.org/images/2024-12-new-graph.png"> Older graph with PyPy3.10 <img alt="old graph" src="https://www.pypy.org/images/2024-12-old-graph.png">. <div class="footnote"> <hr> <ol> <li id="fn:0"> These days most of the work is done by <a href="https://cfbolz.de/">CF Bolz-Tereick</a> and <a href="https://github.com/mattip">me</a> on a volunteer basis. Want to get involved? Reach out, we would love to expand the team. Have an idea for funding the work? Fantastic, let's talk. <a class="footnote-backref" href="https://www.pypy.org/posts/2025/01/towards-pypy311-an-update.html#fnref:0" title="Jump back to footnote 1 in the text">↩</a> </li> </ol> </div></description><category>release</category><guid>https://www.pypy.org/posts/2025/01/towards-pypy311-an-update.html</guid><pubDate>Sat, 04 Jan 2025 13:29:11 GMT</pubDate></item><item><title>Guest Post: Final Encoding in RPython Interpreters</title><link>https://www.pypy.org/posts/2024/11/guest-post-final-encoding-in-rpython.html</link><dc:creator>Corbin</dc:creator><description><h3 id="introduction">Introduction</h3> This post started as a quick note summarizing a recent experiment I carried out upon a small RPython interpreter by rewriting it in an uncommon style. It is written for folks who have already written some RPython and want to take a deeper look at interpreter architecture. Some experiments are about finding solutions to problems. This experiment is about taking a solution which is already well-understood and applying it in the context of RPython to find a new approach. As we will see, there is no real change in functionality or the number of clauses in the interpreter; it's more like a comparison between endo- and exoskeletons, a different arrangement of equivalent bones and plates. <h3 id="overview">Overview</h3> An RPython interpreter for a programming language generally does three or four things, in order: <ol> <li>Read and parse input programs</li> <li>Encode concrete syntax as abstract syntax</li> <li>Optionally, optimize or reduce the abstract syntax</li> <li>Evaluate the abstract syntax: read input data, compute, print output data, etc.</li> </ol> Today we'll look at abstract syntax. Most programming languages admit a <a href="https://en.wikipedia.org/wiki/Parse_tree">concrete parse tree</a> which is readily abstracted to provide an <a href="https://en.wikipedia.org/wiki/Abstract_syntax_tree">abstract syntax tree</a> (AST). The AST is usually encoded with the initial style of encoding. An initial encoding can be transformed into any other encoding for the same AST, looks like a hierarchy of classes, and is implemented as a static structure on the heap. In contrast, there is also a final encoding. A final encoding can be transformed into by any other encoding, looks like an interface for the actions of the interpreter, and is implemented as an unwinding structure on the stack. From the RPython perspective, Python builtin modules like <code>os</code> or <code>sys</code> are final encodings for features of the operating system; the underlying implementation is different when translated or untranslated, but the interface used to access those features does not change. In RPython, an initial encoding is built from a hierarchy of classes. Each class represents a type of tree nodes, corresponding to a parser production in the concrete parse tree. Each class instance therefore represents an individual tree node. The fields of a class, particularly those filled during <code>.__init__()</code>, store pre-computed properties of each node; methods can be used to compute node properties on demand. This seems like an obvious and simple approach; what other approaches could there be? We need an example. <h3 id="final-encoding-of-brainfuck">Final Encoding of Brainfuck</h3> We will consider <a href="https://esolangs.org/wiki/Brainfuck">Brainfuck</a>, a simple Turing-complete programming language. An example Brainfuck program might be: <div class="code"><pre class="code literal-block">[-] </pre></div> This program is built from a loop and a decrement, and sets a cell to zero. In an initial encoding which follows the <a href="https://esolangs.org/wiki/Algebraic_Brainfuck">algebraic semantics of Brainfuck</a>, the program could be expressed by applying class constructors to build a structure on the heap: <div class="code"><pre class="code literal-block">Loop(Plus(-1)) </pre></div> A final encoding is similar, except that class constructors are replaced by methods, the structure is built on the stack, and we are parameterized over the choice of class: <div class="code"><pre class="code literal-block">lambda cls: cls.loop(cls.plus(-1)) </pre></div> In ordinary Python, transforming between these would be trivial, and mostly is a matter of passing around the appropriate class. Indeed, initial and final encodings are equivalent; we'll return to that fact later. However, in RPython, all of the types must line up, and classes must be determined before translation. We'll need to monomorphize our final encodings, using some RPython tricks later on. Before that, let's see what an actual Brainfuck interface looks like, so that we can cover all of the difficulties with final encoding. Before we embark, please keep in mind that local code doesn't know what <code>cls</code> is. There's no type-safe way to inspect an arbitrary semantic domain. In the initial-encoded version, we can ask <code>isinstance(bf, Loop)</code> to see whether an AST node is a loop, but there simply isn't an equivalent for final-encoded ASTs. So, there is an implicit challenge to think about: how do we evaluate a program in an arbitrary semantic domain? For bonus points, how do we optimize a program without inspecting the types of its AST nodes? What follows is a dissection of <a href="https://github.com/rpypkgs/rpypkgs/blob/d439d225b79ac82e009a5f1cd1c670f00356464c/bf/bf.py">this</a> module at the given revision. Readers may find it satisfying to read the entire interpreter top to bottom first; it is less than 300 lines. <h4 id="core-functionality">Core Functionality</h4> Final encoding is given as methods on an interface. These five methods correspond precisely to the summands of the algebra of Brainfuck. <div class="code"><pre class="code literal-block">class BF(object): # Other methods elided def plus(self, i): pass def right(self, i): pass def input(self): pass def output(self): pass def loop(self, bfs): pass </pre></div> Note that the <code>.loop()</code> method takes another program as an argument. Initial-encoded ASTs have other initial-encoded ASTs as fields on class instances; final-encoded ASTs have other final-encoded ASTs as parameters to interface methods. RPython infers all of the types, so the reader has to know that <code>i</code> is usually an integer while <code>bfs</code> is a sequence of Brainfuck operations. We're using a class to implement this functionality. Later, we'll treat it as a mixin, rather than a superclass, to avoid typing problems. <h4 id="monoid">Monoid</h4> In order to optimize input programs, we'll need to represent the underlying <a href="https://en.wikipedia.org/wiki/Monoid">monoid</a> of Brainfuck programs. To do this, we add the signature for a monoid: <div class="code"><pre class="code literal-block">class BF(object): # Other methods elided def unit(self): pass def join(self, l, r): pass </pre></div> This is technically a <a href="https://en.wikipedia.org/wiki/Magma_(algebra)">unital magma</a>, since RPython doesn't support algebraic laws, but we will enforce the algebraic laws later on during optimization. We also want to make use of the folklore that <a href="https://en.wikipedia.org/wiki/Free_monoid">free monoids</a> are lists, allowing callers to pass a list of actions which we'll reduce with recursion: <div class="code"><pre class="code literal-block">class BF(object): # Other methods elided def joinList(self, bfs): if not bfs: return self.unit() elif len(bfs) == 1: return bfs[0] elif len(bfs) == 2: return self.join(bfs[0], bfs[1]) else: i = len(bfs) &gt;&gt; 1 return self.join(self.joinList(bfs[:i]), self.joinList(bfs[i:])) </pre></div> <code>.joinList()</code> is a little bulky to implement, but Wirth's principle applies: the interpreter is shorter with it than without it. <h4 id="idioms">Idioms</h4> Finally, our interface includes a few high-level idioms, like the zero program shown earlier, which are defined in terms of low-level behaviors. In an initial encoding, these could be defined as module-level functions; here, we define them on the mixin class <code>BF</code>. <div class="code"><pre class="code literal-block">class BF(object): # Other methods elided def zero(self): return self.loop(self.plus(-1)) def move(self, i): return self.scalemove(i, 1) def move2(self, i, j): return self.scalemove2(i, 1, j, 1) def scalemove(self, i, s): return self.loop(self.joinList([ self.plus(-1), self.right(i), self.plus(s), self.right(-i)])) def scalemove2(self, i, s, j, t): return self.loop(self.joinList([ self.plus(-1), self.right(i), self.plus(s), self.right(j - i), self.plus(t), self.right(-j)])) </pre></div> <h3 id="interface-oriented-architecture">Interface-oriented Architecture</h3> <h4 id="applying-interfaces">Applying Interfaces</h4> Now, we hack at RPython's object model until everything translates. First, consider the task of pretty-printing. For Brainfuck, we'll simply regurgitate the input program as a Python string: <div class="code"><pre class="code literal-block">class AsStr(object): import_from_mixin(BF) def unit(self): return "" def join(self, l, r): return l + r def plus(self, i): return '+' * i if i &gt; 0 else '-' * -i def right(self, i): return '&gt;' * i if i &gt; 0 else '&lt;' * -i def loop(self, bfs): return '[' + bfs + ']' def input(self): return ',' def output(self): return '.' </pre></div> Via <code>rlib.objectmodel.import_from_mixin</code>, no stressing with covariance of return types is required. Instead, we shift from a Java-esque view of classes and objects, to an OCaml-ish view of prebuilt classes and constructors. <code>AsStr</code> is monomorphic, and any caller of it will have to create their own covariance somehow. For example, here are the first few lines of the parsing function: <div class="code"><pre class="code literal-block">@specialize.argtype(1) def parse(s, domain): ops = [domain.unit()] # Parser elided to preserve the reader's attention </pre></div> By invoking <code>rlib.objectmodel.specialize.argtype</code>, we make copies of the parsing function, up to one per call site, based on our choice of semantic domain. <a href="https://okmij.org/ftp/tagless-final/">Oleg</a> calls these "symantics" but I prefer "domain" in code. Also, note how the parsing stack starts with the unit of the monoid, which corresponds to the empty input string; the parser will repeatedly use the monoidal join to build up a parsed expression without inspecting it. Here's a small taste of that: <div class="code"><pre class="code literal-block">while i &lt; len(s): char = s[i] if char == '+': ops[-1] = domain.join(ops[-1], domain.plus(1)) elif char == '-': ops[-1] = domain.join(ops[-1], domain.plus(-1)) # and so on </pre></div> The reader may feel justifiably mystified; what breaks if we don't add these magic annotations? Well, the translator will throw <code>UnionError</code> because the low-level types don't match. RPython only wants to make one copy of functions like <code>parse()</code> in its low-level representation, and each copy of <code>parse()</code> will be compiled to monomorphic machine code. In this interpreter, in order to support parsing to an optimized string and also parsing to an evaluator, we need two copies of <code>parse()</code>. It is okay to not fully understand this at first. <h4 id="composing-interfaces">Composing Interfaces</h4> Earlier, we noted that an interpreter can optionally optimize input programs after parsing. To support this, we'll precompose a <a href="https://en.wikipedia.org/wiki/Peephole_optimization">peephole optimizer</a> onto an arbitrary domain. We could also postcompose with a parser instead, but that sounds more difficult. Here are the relevant parts: <div class="code"><pre class="code literal-block">def makePeephole(cls): domain = cls() def stripDomain(bfs): return domain.joinList([t[0] for t in bfs]) class Peephole(object): import_from_mixin(BF) def unit(self): return [] def join(self, l, r): return l + r # Actual definition elided... for now... return Peephole, stripDomain </pre></div> Don't worry about the actual optimization yet. What's important here is the pattern of initialization of semantic domains. <code>makePeephole</code> is an <a href="https://en.wikipedia.org/wiki/Standard_ML">SML</a>-style functor on semantic domains: given a final encoding of Brainfuck, it produces another final encoding of Brainfuck which incorporates optimizations. The helper <code>stripDomain</code> is a finalizer which performs the extraction from the optimizer's domain to the underlying <code>cls</code> that was passed in at translation time. For example, let's optimize pretty-printing: <div class="code"><pre class="code literal-block">AsStr, finishStr = makePeephole(AsStr) </pre></div> Now, it only takes one line to parse and print an optimized AST without ever building it on the heap. To be pedantic, fragments of the output string will be heap-allocated, but the AST's node structure will only ever be stack-allocated. Further, to be shallow, the parser is written to prevent malicious input from causing a stack overflow, and this forces it to maintain a heap-allocated RPython list of intermediate operations inside loops. <div class="code"><pre class="code literal-block">print finishStr(parse(text, AsStr())) </pre></div> <h3 id="performance">Performance</h3> But is it fast? Yes. It's faster than the prior version, which was initial-encoded, and also faster than Andrew Brown's classic version (<a href="https://pypy.org/posts/2011/04/tutorial-writing-interpreter-with-pypy-3785910476193156295.html">part 1</a>, <a href="https://pypy.org/posts/2011/04/tutorial-part-2-adding-jit-8121732841568309472.html">part 2</a>). Since Brown's interpreter does not perform much optimization, we will focus on how final encoding can outperform initial encoding. <h4 id="jit">JIT</h4> First, why is it faster than the same interpreter with initial encoding? Well, it still has initial encoding from the JIT's perspective! There is an <code>Op</code> class with a hierarchy of subclasses implementing individual behaviors. A sincere tagless-final student, or those who remember <a href="https://pyvideo.org/pycon-us-2012/stop-writing-classes.html">Stop Writing Classes (2012, Pycon US)</a>, will recognize that the following classes could be plain functions, and should think of the classes as a concession to RPython's lack of support for lambdas with closures rather than an initial encoding. We aren't ever going to directly typecheck any <code>Op</code>, but the JIT will generate typechecking guards anyway, so we effectively get a fully-promoted AST inlined into each JIT trace. First, some simple behaviors: <div class="code"><pre class="code literal-block">class Op(object): _immutable_ = True class _Input(Op): _immutable_ = True def runOn(self, tape, position): tape[position] = ord(os.read(0, 1)[0]) return position Input = _Input() class _Output(Op): _immutable_ = True def runOn(self, tape, position): os.write(1, chr(tape[position])) return position Output = _Output() class Add(Op): _immutable_ = True _immutable_fields_ = "imm", def __init__(self, imm): self.imm = imm def runOn(self, tape, position): tape[position] += self.imm return position </pre></div> The JIT does technically have less information than before; it no longer knows that a sequence of immutable operations is immutable enough to be worth unrolling, but a bit of <code>rlib.jit.unroll_safe</code> fixes that: <div class="code"><pre class="code literal-block">class Seq(Op): _immutable_ = True _immutable_fields_ = "ops[*]", def __init__(self, ops): self.ops = ops @unroll_safe def runOn(self, tape, position): for op in self.ops: position = op.runOn(tape, position) return position </pre></div> Finally, the JIT entry point is at the head of each loop, just like with prior interpreters. Since Brainfuck doesn't support mid-loop jumps, there's no penalty for only allowing merge points at the head of the loop. <div class="code"><pre class="code literal-block">class Loop(Op): _immutable_ = True _immutable_fields_ = "op", def __init__(self, op): self.op = op def runOn(self, tape, position): op = self.op while tape[position]: jitdriver.jit_merge_point(op=op, position=position, tape=tape) position = op.runOn(tape, position) return position </pre></div> That's the end of the implicit challenge. There's no secret to it; just evaluate the AST. Here's part of the semantic domain for evaluation, as well as the "functor" to optimize it. In <code>AsOps.join()</code> are the only <code>isinstance()</code> calls in the entire interpreter! This is acceptable because <code>Seq</code> is effectively a type wrapper for an RPython list, so that a list of operations is also an operation; its list is initial-encoded and available for inspection. <div class="code"><pre class="code literal-block">class AsOps(object): import_from_mixin(BF) def unit(self): return Shift(0) def join(self, l, r): if isinstance(l, Seq) and isinstance(r, Seq): return Seq(l.ops + r.ops) elif isinstance(l, Seq): return Seq(l.ops + [r]) elif isinstance(r, Seq): return Seq([l] + r.ops) return Seq([l, r]) # Other methods elided! AsOps, finishOps = makePeephole(AsOps) </pre></div> And finally here is the actual top-level code to evaluate the input program. As before, once everything is composed, the actual invocation only takes one line. <div class="code"><pre class="code literal-block">tape = bytearray("\x00" * cells) finishOps(parse(text, AsOps())).runOn(tape, 0) </pre></div> <h4 id="peephole-optimization">Peephole Optimization</h4> Our peephole optimizer is an <a href="https://en.wikipedia.org/wiki/Abstract_interpretation">abstract interpreter</a> with one instruction of lookahead/rewrite buffer. It implements the aforementioned algebraic laws of the Brainfuck monoid. It also implements idiom recognition for loops. First, the abstract interpreter. The abstract domain has six elements: <div class="code"><pre class="code literal-block">class AbstractDomain(object): pass meh, aLoop, aZero, theIdentity, anAdd, aRight = [AbstractDomain() for _ in range(6)] </pre></div> We'll also tag everything with an integer, so that <code>anAdd</code> or <code>aRight</code> can be exact annotations. This is the actual <code>Peephole.join()</code> method: <div class="code"><pre class="code literal-block">def join(self, l, r): if not l: return r rv = l[:] bfHead, adHead, immHead = rv.pop() for bf, ad, imm in r: if ad is theIdentity: continue elif adHead is aLoop and ad is aLoop: continue elif adHead is theIdentity: bfHead, adHead, immHead = bf, ad, imm elif adHead is anAdd and ad is aZero: bfHead, adHead, immHead = bf, ad, imm elif adHead is anAdd and ad is anAdd: immHead += imm if immHead: bfHead = domain.plus(immHead) elif rv: bfHead, adHead, immHead = rv.pop() else: bfHead = domain.unit() adHead = theIdentity elif adHead is aRight and ad is aRight: immHead += imm if immHead: bfHead = domain.right(immHead) elif rv: bfHead, adHead, immHead = rv.pop() else: bfHead = domain.unit() adHead = theIdentity else: rv.append((bfHead, adHead, immHead)) bfHead, adHead, immHead = bf, ad, imm rv.append((bfHead, adHead, immHead)) return rv </pre></div> If this were to get much longer, then <a href="https://pypy.org/posts/2024/10/jit-peephole-dsl.html">implementing a DSL</a> would be worth it, but this is a short-enough method to inline. The abstract interpretation is assumed by induction for the left-hand side of the join, save for the final instruction, which is loaded into a rewrite register. Each instruction on the right-hand side is inspected exactly once. The logic for <code>anAdd</code> followed by <code>anAdd</code> is exactly the same as for <code>aRight</code> followed by <code>aRight</code> because they both have underlying <a href="https://en.wikipedia.org/wiki/Abelian_group">Abelian groups</a> given by the integers. The rewrite register is carefully pushed onto and popped off from the left-hand side in order to cancel out <code>theIdentity</code>, which itself is merely a unifier for <code>anAdd</code> or <code>aRight</code> of 0. Note that we generate a lot of garbage. For example, parsing a string of n '+' characters will cause the peephole optimizer to allocate n instances of the underlying <code>domain.plus()</code> action, from <code>domain.plus(1)</code> up to <code>domain.plus(n)</code>. An older initial-encoded version of this interpreter used <a href="https://en.wikipedia.org/wiki/Hash_consing">hash consing</a> to avoid ever building an op more than once, even loops. It appears more efficient to generate lots of immutable garbage than to repeatedly hash inputs and search mutable hash tables, at least for optimizing Brainfuck incrementally during parsing. Finally, let's look at idiom recognition. RPython lists are initial-coded, so we can dispatch based on the length of the list, and then inspect the abstract domains of each action. <div class="code"><pre class="code literal-block">def isConstAdd(bf, i): return bf[1] is anAdd and bf[2] == i def oppositeShifts(bf1, bf2): return bf1[1] is bf2[1] is aRight and bf1[2] == -bf2[2] def oppositeShifts2(bf1, bf2, bf3): return (bf1[1] is bf2[1] is bf3[1] is aRight and bf1[2] + bf2[2] + bf3[2] == 0) def loop(self, bfs): if len(bfs) == 1: bf, ad, imm = bfs[0] if ad is anAdd and imm in (1, -1): return [(domain.zero(), aZero, 0)] elif len(bfs) == 4: if (isConstAdd(bfs[0], -1) and bfs[2][1] is anAdd and oppositeShifts(bfs[1], bfs[3])): return [(domain.scalemove(bfs[1][2], bfs[2][2]), aLoop, 0)] if (isConstAdd(bfs[3], -1) and bfs[1][1] is anAdd and oppositeShifts(bfs[0], bfs[2])): return [(domain.scalemove(bfs[0][2], bfs[1][2]), aLoop, 0)] elif len(bfs) == 6: if (isConstAdd(bfs[0], -1) and bfs[2][1] is bfs[4][1] is anAdd and oppositeShifts2(bfs[1], bfs[3], bfs[5])): return [(domain.scalemove2(bfs[1][2], bfs[2][2], bfs[1][2] + bfs[3][2], bfs[4][2]), aLoop, 0)] if (isConstAdd(bfs[5], -1) and bfs[1][1] is bfs[3][1] is anAdd and oppositeShifts2(bfs[0], bfs[2], bfs[4])): return [(domain.scalemove2(bfs[0][2], bfs[1][2], bfs[0][2] + bfs[2][2], bfs[3][2]), aLoop, 0)] return [(domain.loop(stripDomain(bfs)), aLoop, 0)] </pre></div> This ends the bonus question. How do we optimize an unknown semantic domain? We must maintain an abstract context which describes elements of the domain. In initial encoding, we ask an AST about itself. In final encoding, we already know everything relevant about the AST. The careful reader will see that I didn't really answer that opening question in the JIT section. Because the JIT still ranges over the same operations as before, it can't really be slower; but why is it now faster? Because the optimizer is now slightly better in a few edge cases. It performs the same optimizations as before, but the rigor of abstract interpretation causes it to emit slightly better operations to the JIT backend. Concretely, improving the optimizer can shorten pretty-printed programs. The <a href="https://bbgauge.info/">Busy Beaver Gauge</a> measures the length of programs which search for solutions to mathematical problems. After implementing and debugging the final-encoded interpreter, I found that two of my entries on the <a href="https://bbgauge.info/brainfuck.html">Busy Beaver Gauge for Brainfuck</a> had become shorter by about 2%. (Most other entries are already hand-optimized according to the standard algebra and have no optimization opportunities.) <h3 id="discussion">Discussion</h3> Given that initial and final encodings are equivalent, and noting that RPython's toolchain is written to prefer initial encodings, what did we actually gain? Did we gain anything? One obvious downside to final encoding in RPython is interpreter size. The example interpreter shown here is a rewrite of an initial-encoded interpreter which can be seen <a href="https://github.com/rpypkgs/rpypkgs/blob/659c8a26d428a1e04fdff614b28e464a50d4647b/bf/bf.py">here</a> for comparison. Final encoding adds about 20% more code in this case. Final encoding is not necessarily more code than initial encoding, though. All AST encodings in interpreters are subject to the <a href="https://en.wikipedia.org/wiki/Expression_problem">Expression Problem</a>, which states that there is generally a quadratic amount of code required to implement multiple behaviors for an AST with multiple types of nodes; specifically, n behaviors for m types of nodes require n × m methods. Initial encodings improve the cost of adding new types of nodes; final encodings improve the cost of adding new behaviors. Final encoding may tend to win in large codebases for mature languages, where the language does not change often but new behaviors are added frequently and maintained for long periods. Optimizations in final encoding require a bit of planning. The abstract-interpretation approach is solid but relies upon the monoid and its algebraic laws. In the worst case, an entire class hierarchy could be required to encode the abstraction. It is remarkable to find a 2% improvement in residual program size merely by reimplementing an optimizer as an abstract interpreter respecting the algebraic laws. This could be the most important lesson for compiler engineers, if it happens to generalize. Final encoding was popularized via the tagless-final movement in OCaml and Scala, including famously in a series of tutorials by <a href="https://okmij.org/ftp/tagless-final/">Kiselyov et al</a>. A "tag", in this jargon, is a runtime identifier for an object's type or class; a tagless encoding effectively doesn't allow <code>isinstance()</code> at all. In the above presentation, tags could be hacked in, but were not materially relevant to most steps. Tags were required for the final evaluation step, though, and the tagless-final insight is that certain type systems can express type-safe evaluation without those tags. We won't go further in this direction because tags also communicate valuable information to the JIT. <h4 id="summarizing-table">Summarizing Table</h4> <table> <thead> <tr> <th>Initial Encoding</th> <th>Final Encoding</th> </tr> </thead> <tbody> <tr> <td>hierarchy of classes</td> <td>signature of interfaces</td> </tr> <tr> <td>class constructors</td> <td>method calls</td> </tr> <tr> <td>built on the heap</td> <td>built on the stack</td> </tr> <tr> <td>traversals allocate stack</td> <td>traversals allocate heap</td> </tr> <tr> <td>tags are available with <code>isinstance()</code></td> <td>tags are only available through hacks</td> </tr> <tr> <td>cost of adding a new AST node: one class</td> <td>cost of adding a new AST node: one method on every other class</td> </tr> <tr> <td>cost of adding a new behavior: one method on every other class</td> <td>cost of adding a new behavior: one class</td> </tr> </tbody> </table> <h3 id="credits">Credits</h3> Thanks to folks in <code>#pypy</code> on Libera Chat: arigato for the idea, larstiq for pushing me to write it up, and cfbolz and mattip for reviewing and finding mistakes. The original IRC discussion leading to this blog post is available <a href="https://gist.github.com/MostAwesomeDude/fd86ad2d2e38af7aa67b6e548aabe008">here</a>. This interpreter is part of the <a href="https://github.com/rpypkgs/rpypkgs">rpypkgs</a> suite, a Nix flake for RPython interpreters. Readers with Nix installed can run this interpreter directly from the flake: <div class="code"><pre class="code literal-block">$ nix-prefetch-url https://github.com/MG-K/pypy-tutorial-ko/raw/refs/heads/master/mandel.b $ nix run github:rpypkgs/rpypkgs#bf -- /nix/store/ngnphbap9ncvz41d0fkvdh61n7j2bg21-mandel.b </pre></div></description><category>guestpost</category><guid>https://www.pypy.org/posts/2024/11/guest-post-final-encoding-in-rpython.html</guid><pubDate>Thu, 14 Nov 2024 08:42:36 GMT</pubDate></item><item><title>A DSL for Peephole Transformation Rules of Integer Operations in the PyPy JIT</title><link>https://www.pypy.org/posts/2024/10/jit-peephole-dsl.html</link><dc:creator>CF Bolz-Tereick</dc:creator><description>As is probably apparent from the sequence of blog posts about the topic in the last year, I have been thinking about and working on integer optimizations in the JIT compiler a lot. This work was mainly motivated by <a class="reference external" href="https://docs.pydrofoil.org/en/latest/">Pydrofoil</a>, where integer operations matter a lot more than for your typical Python program. In this post I'll describe my most recent change, which is a new small domain specific language that I implemented to specify peephole optimizations on integer operations in the JIT. It uses pattern matching to specify how (sequences of) integer operations should be simplified and optimized. The rules are then compiled to RPython code that then becomes part of the JIT's optimization passes. To make it less likely to introduce incorrect optimizations into the JIT, the rules are automatically proven correct with Z3 as part of the build process (for a more hands-on intro to how that works you can look at the <a class="reference external" href="https://pypy.org/posts/2024/08/toy-knownbits.html#proving-correctness-of-the-transfer-functions-with-z3">knownbits</a> post). In this blog post I want to motivate why I introduced the DSL and give an introduction to how it works. <section id="motivation"> <h2>Motivation</h2> This summer, after I wrote my <a class="reference external" href="https://www.pypy.org/posts/2024/07/mining-jit-traces-missing-optimizations-z3.html">scripts to mine JIT traces for missed optimization</a> opportunities, I started implementing a few of the integer peephole rewrite that the script identified. Unfortunately, doing so led to the problem that the way we express these rewrites up to now is very imperative and verbose. Here's a snippet of RPython code that shows some rewrites for integer multiplication (look at the comments to see what the different parts actually do). You don't need to understand the code in detail, but basically it's in very imperative style and there's quite a lot of boilerplate. <div class="code"><pre class="code python"><a id="rest_code_eee775dbffd04b1385c373f787204896-1" name="rest_code_eee775dbffd04b1385c373f787204896-1" href="https://www.pypy.org/posts/2024/10/jit-peephole-dsl.html#rest_code_eee775dbffd04b1385c373f787204896-1"></a>def optimize_INT_MUL(self, op): <a id="rest_code_eee775dbffd04b1385c373f787204896-2" name="rest_code_eee775dbffd04b1385c373f787204896-2" href="https://www.pypy.org/posts/2024/10/jit-peephole-dsl.html#rest_code_eee775dbffd04b1385c373f787204896-2"></a> arg0 = get_box_replacement(op.getarg(0)) <a id="rest_code_eee775dbffd04b1385c373f787204896-3" name="rest_code_eee775dbffd04b1385c373f787204896-3" href="https://www.pypy.org/posts/2024/10/jit-peephole-dsl.html#rest_code_eee775dbffd04b1385c373f787204896-3"></a> b0 = self.getintbound(arg0) <a id="rest_code_eee775dbffd04b1385c373f787204896-4" name="rest_code_eee775dbffd04b1385c373f787204896-4" href="https://www.pypy.org/posts/2024/10/jit-peephole-dsl.html#rest_code_eee775dbffd04b1385c373f787204896-4"></a> arg1 = get_box_replacement(op.getarg(1)) <a id="rest_code_eee775dbffd04b1385c373f787204896-5" name="rest_code_eee775dbffd04b1385c373f787204896-5" href="https://www.pypy.org/posts/2024/10/jit-peephole-dsl.html#rest_code_eee775dbffd04b1385c373f787204896-5"></a> b1 = self.getintbound(arg1) <a id="rest_code_eee775dbffd04b1385c373f787204896-6" name="rest_code_eee775dbffd04b1385c373f787204896-6" href="https://www.pypy.org/posts/2024/10/jit-peephole-dsl.html#rest_code_eee775dbffd04b1385c373f787204896-6"></a> <a id="rest_code_eee775dbffd04b1385c373f787204896-7" name="rest_code_eee775dbffd04b1385c373f787204896-7" href="https://www.pypy.org/posts/2024/10/jit-peephole-dsl.html#rest_code_eee775dbffd04b1385c373f787204896-7"></a> if b0.known_eq_const(1): <a id="rest_code_eee775dbffd04b1385c373f787204896-8" name="rest_code_eee775dbffd04b1385c373f787204896-8" href="https://www.pypy.org/posts/2024/10/jit-peephole-dsl.html#rest_code_eee775dbffd04b1385c373f787204896-8"></a> # 1 * x == x <a id="rest_code_eee775dbffd04b1385c373f787204896-9" name="rest_code_eee775dbffd04b1385c373f787204896-9" href="https://www.pypy.org/posts/2024/10/jit-peephole-dsl.html#rest_code_eee775dbffd04b1385c373f787204896-9"></a> self.make_equal_to(op, arg1) <a id="rest_code_eee775dbffd04b1385c373f787204896-10" name="rest_code_eee775dbffd04b1385c373f787204896-10" href="https://www.pypy.org/posts/2024/10/jit-peephole-dsl.html#rest_code_eee775dbffd04b1385c373f787204896-10"></a> elif b1.known_eq_const(1): <a id="rest_code_eee775dbffd04b1385c373f787204896-11" name="rest_code_eee775dbffd04b1385c373f787204896-11" href="https://www.pypy.org/posts/2024/10/jit-peephole-dsl.html#rest_code_eee775dbffd04b1385c373f787204896-11"></a> # x * 1 == x <a id="rest_code_eee775dbffd04b1385c373f787204896-12" name="rest_code_eee775dbffd04b1385c373f787204896-12" href="https://www.pypy.org/posts/2024/10/jit-peephole-dsl.html#rest_code_eee775dbffd04b1385c373f787204896-12"></a> self.make_equal_to(op, arg0) <a id="rest_code_eee775dbffd04b1385c373f787204896-13" name="rest_code_eee775dbffd04b1385c373f787204896-13" href="https://www.pypy.org/posts/2024/10/jit-peephole-dsl.html#rest_code_eee775dbffd04b1385c373f787204896-13"></a> elif b0.known_eq_const(0) or b1.known_eq_const(0): <a id="rest_code_eee775dbffd04b1385c373f787204896-14" name="rest_code_eee775dbffd04b1385c373f787204896-14" href="https://www.pypy.org/posts/2024/10/jit-peephole-dsl.html#rest_code_eee775dbffd04b1385c373f787204896-14"></a> # 0 * x == x * 0 == 0 <a id="rest_code_eee775dbffd04b1385c373f787204896-15" name="rest_code_eee775dbffd04b1385c373f787204896-15" href="https://www.pypy.org/posts/2024/10/jit-peephole-dsl.html#rest_code_eee775dbffd04b1385c373f787204896-15"></a> self.make_constant_int(op, 0) <a id="rest_code_eee775dbffd04b1385c373f787204896-16" name="rest_code_eee775dbffd04b1385c373f787204896-16" href="https://www.pypy.org/posts/2024/10/jit-peephole-dsl.html#rest_code_eee775dbffd04b1385c373f787204896-16"></a> else: <a id="rest_code_eee775dbffd04b1385c373f787204896-17" name="rest_code_eee775dbffd04b1385c373f787204896-17" href="https://www.pypy.org/posts/2024/10/jit-peephole-dsl.html#rest_code_eee775dbffd04b1385c373f787204896-17"></a> for lhs, rhs in [(arg0, arg1), (arg1, arg0)]: <a id="rest_code_eee775dbffd04b1385c373f787204896-18" name="rest_code_eee775dbffd04b1385c373f787204896-18" href="https://www.pypy.org/posts/2024/10/jit-peephole-dsl.html#rest_code_eee775dbffd04b1385c373f787204896-18"></a> lh_info = self.getintbound(lhs) <a id="rest_code_eee775dbffd04b1385c373f787204896-19" name="rest_code_eee775dbffd04b1385c373f787204896-19" href="https://www.pypy.org/posts/2024/10/jit-peephole-dsl.html#rest_code_eee775dbffd04b1385c373f787204896-19"></a> if lh_info.is_constant(): <a id="rest_code_eee775dbffd04b1385c373f787204896-20" name="rest_code_eee775dbffd04b1385c373f787204896-20" href="https://www.pypy.org/posts/2024/10/jit-peephole-dsl.html#rest_code_eee775dbffd04b1385c373f787204896-20"></a> x = lh_info.get_constant_int() <a id="rest_code_eee775dbffd04b1385c373f787204896-21" name="rest_code_eee775dbffd04b1385c373f787204896-21" href="https://www.pypy.org/posts/2024/10/jit-peephole-dsl.html#rest_code_eee775dbffd04b1385c373f787204896-21"></a> if x &amp; (x - 1) == 0: <a id="rest_code_eee775dbffd04b1385c373f787204896-22" name="rest_code_eee775dbffd04b1385c373f787204896-22" href="https://www.pypy.org/posts/2024/10/jit-peephole-dsl.html#rest_code_eee775dbffd04b1385c373f787204896-22"></a> # x * (2 ** c) == x &lt;&lt; c <a id="rest_code_eee775dbffd04b1385c373f787204896-23" name="rest_code_eee775dbffd04b1385c373f787204896-23" href="https://www.pypy.org/posts/2024/10/jit-peephole-dsl.html#rest_code_eee775dbffd04b1385c373f787204896-23"></a> new_rhs = ConstInt(highest_bit(lh_info.get_constant_int())) <a id="rest_code_eee775dbffd04b1385c373f787204896-24" name="rest_code_eee775dbffd04b1385c373f787204896-24" href="https://www.pypy.org/posts/2024/10/jit-peephole-dsl.html#rest_code_eee775dbffd04b1385c373f787204896-24"></a> op = self.replace_op_with(op, rop.INT_LSHIFT, args=[rhs, new_rhs]) <a id="rest_code_eee775dbffd04b1385c373f787204896-25" name="rest_code_eee775dbffd04b1385c373f787204896-25" href="https://www.pypy.org/posts/2024/10/jit-peephole-dsl.html#rest_code_eee775dbffd04b1385c373f787204896-25"></a> self.optimizer.send_extra_operation(op) <a id="rest_code_eee775dbffd04b1385c373f787204896-26" name="rest_code_eee775dbffd04b1385c373f787204896-26" href="https://www.pypy.org/posts/2024/10/jit-peephole-dsl.html#rest_code_eee775dbffd04b1385c373f787204896-26"></a> return <a id="rest_code_eee775dbffd04b1385c373f787204896-27" name="rest_code_eee775dbffd04b1385c373f787204896-27" href="https://www.pypy.org/posts/2024/10/jit-peephole-dsl.html#rest_code_eee775dbffd04b1385c373f787204896-27"></a> elif x == -1: <a id="rest_code_eee775dbffd04b1385c373f787204896-28" name="rest_code_eee775dbffd04b1385c373f787204896-28" href="https://www.pypy.org/posts/2024/10/jit-peephole-dsl.html#rest_code_eee775dbffd04b1385c373f787204896-28"></a> # x * -1 == -x <a id="rest_code_eee775dbffd04b1385c373f787204896-29" name="rest_code_eee775dbffd04b1385c373f787204896-29" href="https://www.pypy.org/posts/2024/10/jit-peephole-dsl.html#rest_code_eee775dbffd04b1385c373f787204896-29"></a> op = self.replace_op_with(op, rop.INT_NEG, args=[rhs]) <a id="rest_code_eee775dbffd04b1385c373f787204896-30" name="rest_code_eee775dbffd04b1385c373f787204896-30" href="https://www.pypy.org/posts/2024/10/jit-peephole-dsl.html#rest_code_eee775dbffd04b1385c373f787204896-30"></a> self.optimizer.send_extra_operation(op) <a id="rest_code_eee775dbffd04b1385c373f787204896-31" name="rest_code_eee775dbffd04b1385c373f787204896-31" href="https://www.pypy.org/posts/2024/10/jit-peephole-dsl.html#rest_code_eee775dbffd04b1385c373f787204896-31"></a> return <a id="rest_code_eee775dbffd04b1385c373f787204896-32" name="rest_code_eee775dbffd04b1385c373f787204896-32" href="https://www.pypy.org/posts/2024/10/jit-peephole-dsl.html#rest_code_eee775dbffd04b1385c373f787204896-32"></a> else: <a id="rest_code_eee775dbffd04b1385c373f787204896-33" name="rest_code_eee775dbffd04b1385c373f787204896-33" href="https://www.pypy.org/posts/2024/10/jit-peephole-dsl.html#rest_code_eee775dbffd04b1385c373f787204896-33"></a> # x * (1 &lt;&lt; y) == x &lt;&lt; y <a id="rest_code_eee775dbffd04b1385c373f787204896-34" name="rest_code_eee775dbffd04b1385c373f787204896-34" href="https://www.pypy.org/posts/2024/10/jit-peephole-dsl.html#rest_code_eee775dbffd04b1385c373f787204896-34"></a> shiftop = self.optimizer.as_operation(get_box_replacement(lhs), rop.INT_LSHIFT) <a id="rest_code_eee775dbffd04b1385c373f787204896-35" name="rest_code_eee775dbffd04b1385c373f787204896-35" href="https://www.pypy.org/posts/2024/10/jit-peephole-dsl.html#rest_code_eee775dbffd04b1385c373f787204896-35"></a> if shiftop is None: <a id="rest_code_eee775dbffd04b1385c373f787204896-36" name="rest_code_eee775dbffd04b1385c373f787204896-36" href="https://www.pypy.org/posts/2024/10/jit-peephole-dsl.html#rest_code_eee775dbffd04b1385c373f787204896-36"></a> continue <a id="rest_code_eee775dbffd04b1385c373f787204896-37" name="rest_code_eee775dbffd04b1385c373f787204896-37" href="https://www.pypy.org/posts/2024/10/jit-peephole-dsl.html#rest_code_eee775dbffd04b1385c373f787204896-37"></a> if not shiftop.getarg(0).is_constant() or shiftop.getarg(0).getint() != 1: <a id="rest_code_eee775dbffd04b1385c373f787204896-38" name="rest_code_eee775dbffd04b1385c373f787204896-38" href="https://www.pypy.org/posts/2024/10/jit-peephole-dsl.html#rest_code_eee775dbffd04b1385c373f787204896-38"></a> continue <a id="rest_code_eee775dbffd04b1385c373f787204896-39" name="rest_code_eee775dbffd04b1385c373f787204896-39" href="https://www.pypy.org/posts/2024/10/jit-peephole-dsl.html#rest_code_eee775dbffd04b1385c373f787204896-39"></a> shiftvar = get_box_replacement(shiftop.getarg(1)) <a id="rest_code_eee775dbffd04b1385c373f787204896-40" name="rest_code_eee775dbffd04b1385c373f787204896-40" href="https://www.pypy.org/posts/2024/10/jit-peephole-dsl.html#rest_code_eee775dbffd04b1385c373f787204896-40"></a> shiftbound = self.getintbound(shiftvar) <a id="rest_code_eee775dbffd04b1385c373f787204896-41" name="rest_code_eee775dbffd04b1385c373f787204896-41" href="https://www.pypy.org/posts/2024/10/jit-peephole-dsl.html#rest_code_eee775dbffd04b1385c373f787204896-41"></a> if shiftbound.known_nonnegative() and shiftbound.known_lt_const(LONG_BIT): <a id="rest_code_eee775dbffd04b1385c373f787204896-42" name="rest_code_eee775dbffd04b1385c373f787204896-42" href="https://www.pypy.org/posts/2024/10/jit-peephole-dsl.html#rest_code_eee775dbffd04b1385c373f787204896-42"></a> op = self.replace_op_with( <a id="rest_code_eee775dbffd04b1385c373f787204896-43" name="rest_code_eee775dbffd04b1385c373f787204896-43" href="https://www.pypy.org/posts/2024/10/jit-peephole-dsl.html#rest_code_eee775dbffd04b1385c373f787204896-43"></a> op, rop.INT_LSHIFT, args=[rhs, shiftvar]) <a id="rest_code_eee775dbffd04b1385c373f787204896-44" name="rest_code_eee775dbffd04b1385c373f787204896-44" href="https://www.pypy.org/posts/2024/10/jit-peephole-dsl.html#rest_code_eee775dbffd04b1385c373f787204896-44"></a> self.optimizer.send_extra_operation(op) <a id="rest_code_eee775dbffd04b1385c373f787204896-45" name="rest_code_eee775dbffd04b1385c373f787204896-45" href="https://www.pypy.org/posts/2024/10/jit-peephole-dsl.html#rest_code_eee775dbffd04b1385c373f787204896-45"></a> return <a id="rest_code_eee775dbffd04b1385c373f787204896-46" name="rest_code_eee775dbffd04b1385c373f787204896-46" href="https://www.pypy.org/posts/2024/10/jit-peephole-dsl.html#rest_code_eee775dbffd04b1385c373f787204896-46"></a> return self.emit(op) </pre></div> Adding more rules to these functions is very tedious and gets super confusing when the functions get bigger. In addition I am always worried about making mistakes when writing this kind of code, and there is no feedback at all about which of these rules are actually applied a lot in real programs. Therefore I decided to write a small domain specific language with the goal of expressing these rules in a more declarative way. In the rest of the post I'll describe the DSL (most of that description is adapted from the <a class="reference external" href="https://rpython.readthedocs.io/en/latest/jit/ruleopt.html">documentation</a> about it that I wrote). </section> <section id="the-peephole-rule-dsl"> <h2>The Peephole Rule DSL</h2> <section id="simple-transformation-rules"> <h3>Simple transformation rules</h3> The rules in the DSL specify how integer operation can be transformed into cheaper other integer operations. A rule always consists of a name, a pattern, and a target. Here's a simple rule: <pre class="literal-block">add_zero: int_add(x, 0) =&gt; x</pre> The name of the rule is <code class="docutils literal">add_zero</code>. It matches operations in the trace of the form <code class="docutils literal">int_add(x, 0)</code>, where <code class="docutils literal">x</code> will match anything and <code class="docutils literal">0</code> will match only the constant zero. After the <code class="docutils literal">=&gt;</code> arrow is the target of the rewrite, i.e. what the operation is rewritten to, in this case <code class="docutils literal">x</code>. The rule language has a list of which of the operations are commutative, so <code class="docutils literal">add_zero</code> will also optimize <code class="docutils literal">int_add(0, x)</code> to <code class="docutils literal">x</code>. Variables in the pattern can repeat: <pre class="literal-block">sub_x_x: int_sub(x, x) =&gt; 0</pre> This rule matches against <code class="docutils literal">int_sub</code> operations where the two arguments are the same (either the same box, or the same constant). Here's a rule with a more complicated pattern: <pre class="literal-block">sub_add: int_sub(int_add(x, y), y) =&gt; x</pre> This pattern matches <code class="docutils literal">int_sub</code> operations, where the first argument was produced by an <code class="docutils literal">int_add</code> operation. In addition, one of the arguments of the addition has to be the same as the second argument of the subtraction. The constants <code class="docutils literal">MININT</code>, <code class="docutils literal">MAXINT</code> and <code class="docutils literal">LONG_BIT</code> (which is either 32 or 64, depending on which platform the JIT is built for) can be used in rules, they behave like writing numbers but allow bit-width-independent formulations: <pre class="literal-block">is_true_and_minint: int_is_true(int_and(x, MININT)) =&gt; int_lt(x, 0)</pre> It is also possible to have a pattern where some arguments needs to be a constant, without specifying which constant. Those patterns look like this: <pre class="literal-block">sub_add_consts: int_sub(int_add(x, C1), C2) # incomplete # more goes here =&gt; int_sub(x, C)</pre> Variables in the pattern that start with a <code class="docutils literal">C</code> match against constants only. However, in this current form the rule is incomplete, because the variable <code class="docutils literal">C</code> that is being used in the target operation is not defined anywhere. We will see how to compute it in the next section. </section> <section id="computing-constants-and-other-intermediate-results"> <h3>Computing constants and other intermediate results</h3> Sometimes it is necessary to compute intermediate results that are used in the target operation. To do that, there can be extra assignments between the rule head and the rule target.: <pre class="literal-block">sub_add_consts: int_sub(int_add(x, C1), C2) # incomplete C = C1 + C2 =&gt; int_sub(x, C)</pre> The right hand side of such an assignment is a subset of Python syntax, supporting arithmetic using <code class="docutils literal">+</code>, <code class="docutils literal">-</code>, <code class="docutils literal">*</code>, and certain helper functions. However, the syntax allows you to be explicit about unsignedness for some operations. E.g. <code class="docutils literal">&gt;&gt;u</code> exists for unsigned right shifts (and I plan to add <code class="docutils literal">&gt;u</code>, <code class="docutils literal">&gt;=u</code>, <code class="docutils literal">&lt;u</code>, <code class="docutils literal">&lt;=u</code> for comparisons). Here's an example of a rule that uses <code class="docutils literal">&gt;&gt;u</code>: <pre class="literal-block">urshift_lshift_x_c_c: uint_rshift(int_lshift(x, C), C) mask = (-1 &lt;&lt; C) &gt;&gt;u C =&gt; int_and(x, mask)</pre> </section> <section id="checks"> <h3>Checks</h3> Some rewrites are only true under certain conditions. For example, <code class="docutils literal">int_eq(x, 1)</code> can be rewritten to <code class="docutils literal">x</code>, if <code class="docutils literal">x</code> is known to store a boolean value. This can be expressed with checks: <pre class="literal-block">eq_one: int_eq(x, 1) check x.is_bool() =&gt; x</pre> A check is followed by a boolean expression. The variables from the pattern can be used as <code class="docutils literal">IntBound</code> instances in checks (and also in assignments) to find out what the abstract interpretation of the JIT knows about the value of a trace variable (<code class="docutils literal">IntBound</code> is the name of the abstract domain that the JIT uses for integers, despite the fact that it also stores <a class="reference external" href="https://www.pypy.org/posts/2024/08/toy-knownbits.html">knownbits</a> information nowadays). Here's another example: <pre class="literal-block">mul_lshift: int_mul(x, int_lshift(1, y)) check y.known_ge_const(0) and y.known_le_const(LONG_BIT) =&gt; int_lshift(x, y)</pre> It expresses that <code class="docutils literal">x * (1 &lt;&lt; y)</code> can be rewritten to <code class="docutils literal">x &lt;&lt; y</code> but checks that <code class="docutils literal">y</code> is known to be between <code class="docutils literal">0</code> and <code class="docutils literal">LONG_BIT</code>. Checks and assignments can be repeated and combined with each other: <pre class="literal-block">mul_pow2_const: int_mul(x, C) check C &gt; 0 and C &amp; (C - 1) == 0 shift = highest_bit(C) =&gt; int_lshift(x, shift)</pre> In addition to calling methods on <code class="docutils literal">IntBound</code> instances, it's also possible to access their attributes, like in this rule: <pre class="literal-block">and_x_c_in_range: int_and(x, C) check x.lower &gt;= 0 and x.upper &lt;= C &amp; ~(C + 1) =&gt; x</pre> </section> <section id="rule-ordering-and-liveness"> <h3>Rule Ordering and Liveness</h3> The generated optimizer code will give preference to applying rules that produce a constant or a variable as a rewrite result. Only if none of those match do rules that produce new result operations get applied. For example, the rules <code class="docutils literal">sub_x_x</code> and <code class="docutils literal">sub_add</code> are tried before trying <code class="docutils literal">sub_add_consts</code>, because the former two rules optimize to a constant and a variable respectively, while the latter produces a new operation as the result. The rule <code class="docutils literal">sub_add_consts</code> has a possible problem, which is that if the intermediate result of the <code class="docutils literal">int_add</code> operation in the rule head is used by some other operations, then the <code class="docutils literal">sub_add_consts</code> rule does not actually reduce the number of operations (and might actually make things slightly worse due to increased register pressure). However, currently it would be extremely hard to take that kind of information into account in the optimization pass of the JIT, so we optimistically apply the rules anyway. </section> <section id="checking-rule-coverage"> <h3>Checking rule coverage</h3> Every rewrite rule should have at least one unit test where it triggers. To ensure this, the <a class="reference external" href="https://github.com/pypy/pypy/blob/d92d0bfd38318ede1cbaadadafd77da69d431fad/rpython/jit/metainterp/optimizeopt/test/test_optimizeintbound.py">unit test file that mainly checks integer optimizations</a> in the JIT has an assert at the end of a test run, that every rule fired at least once. </section> <section id="printing-rule-statistics"> <h3>Printing rule statistics</h3> The JIT can print statistics about which rule fired how often in the <code class="docutils literal">jit-intbounds-stats</code> logging category, using the <a class="reference external" href="https://rpython.readthedocs.io/en/latest/logging.html">PYPYLOG</a> mechanism. For example, to print the category to stdout at the end of program execution, run PyPy like this: <pre class="literal-block">PYPYLOG=jit-intbounds-stats:- pypy ...</pre> The output of that will look something like this: <pre class="literal-block">int_add add_reassoc_consts 2514 add_zero 107008 int_sub sub_zero 31519 sub_from_zero 523 sub_x_x 3153 sub_add_consts 159 sub_add 55 sub_sub_x_c_c 1752 sub_sub_c_x_c 0 sub_xor_x_y_y 0 sub_or_x_y_y 0 int_mul mul_zero 0 mul_one 110 mul_minus_one 0 mul_pow2_const 1456 mul_lshift 0 ...</pre> </section> <section id="termination-and-confluence"> <h3>Termination and Confluence</h3> Right now there are unfortunately no checks that the rules actually rewrite operations towards "simpler" forms. There is no cost model, and also nothing that prevents you from writing a rule like this: <pre class="literal-block">neg_complication: int_neg(x) # leads to infinite rewrites =&gt; int_mul(-1, x)</pre> Doing this would lead to endless rewrites if there is also another rule that turns multiplication with -1 into negation. There is also no checking for <a class="reference external" href="https://en.wikipedia.org/wiki/Confluence_(abstract_rewriting)">confluence</a> (yet?), i.e. the property that all rewrites starting from the same input trace always lead to the same output trace, no matter in which order the rules are applied. </section> <section id="proofs"> <h3>Proofs</h3> It is very easy to write a peephole rule that is not correct in all corner cases. Therefore all the rules are proven correct with Z3 before compiled into actual JIT code, by default. When the proof fails, a (hopefully minimal) counterexample is printed. The counterexample consists of values for all the inputs that fulfil the checks, values for the intermediate expressions, and then two different values for the source and the target operations. E.g. if we try to add the incorrect rule: <pre class="literal-block">mul_is_add: int_mul(a, b) =&gt; int_add(a, b)</pre> We get the following counterexample as output: <pre class="literal-block">Could not prove correctness of rule 'mul_is_add' in line 1 counterexample given by Z3: counterexample values: a: 0 b: 1 operation int_mul(a, b) with Z3 formula a*b has counterexample result vale: 0 BUT target expression: int_add(a, b) with Z3 formula a + b has counterexample value: 1</pre> If we add conditions, they are taken into account and the counterexample will fulfil the conditions: <pre class="literal-block">mul_is_add: int_mul(a, b) check a.known_gt_const(1) and b.known_gt_const(2) =&gt; int_add(a, b)</pre> This leads to the following counterexample: <pre class="literal-block">Could not prove correctness of rule 'mul_is_add' in line 46 counterexample given by Z3: counterexample values: a: 2 b: 3 operation int_mul(a, b) with Z3 formula a*b has counterexample result vale: 6 BUT target expression: int_add(a, b) with Z3 formula a + b has counterexample value: 5</pre> Some <code class="docutils literal">IntBound</code> methods cannot be used in Z3 proofs because their <a class="reference external" href="https://www.pypy.org/posts/2024/08/toy-knownbits.html#cases-where-this-style-of-z3-proof-doesnt-work).">control flow is too complex</a>. If that is the case, they can have Z3-equivalent formulations defined (in every case this is done, it's a potential proof hole if the Z3 friendly reformulation and the real implementation differ from each other, therefore extra care is required to make very sure they are equivalent). It's possible to skip the proof of individual rules entirely by adding <code class="docutils literal">SORRY_Z3</code> to its body (but we should try not to do that too often): <pre class="literal-block">eq_different_knownbits: int_eq(x, y) SORRY_Z3 check x.known_ne(y) =&gt; 0</pre> </section> <section id="checking-for-satisfiability"> <h3>Checking for satisfiability</h3> In addition to checking whether the rule yields a correct optimization, we also check whether the rule can ever apply. This ensures that there are some runtime values that would fulfil all the checks in a rule. Here's an example of a rule violating this: <pre class="literal-block">never_applies: int_is_true(x) check x.known_lt_const(0) and x.known_gt_const(0) # impossible condition, always False =&gt; x</pre> Right now the error messages if this goes wrong are not completely easy to understand. I hope to be able to improve this later: <pre class="literal-block">Rule 'never_applies' cannot ever apply in line 1 Z3 did not manage to find values for variables x such that the following condition becomes True: And(x &lt;= x_upper, x_lower &lt;= x, If(x_upper &lt; 0, x_lower &gt; 0, x_upper &lt; 0))</pre> </section> <section id="implementation-notes"> <h3>Implementation Notes</h3> The implementation of the DSL is done in a relatively ad-hoc manner. It is parsed using <a class="reference external" href="https://rply.readthedocs.io/">rply</a>, there's a small type checker that tries to find common problems in how the rules are written. Z3 is used via the Python API, like in the previous blog posts that are using it. The pattern matching RPython code is generated using an approach inspired by Luc Maranget's paper <a class="reference external" href="http://moscova.inria.fr/~maranget/papers/ml05e-maranget.pdf">Compiling Pattern Matching to Good Decision Trees</a>. See <a class="reference external" href="https://compiler.club/compiling-pattern-matching/">this blog post</a> for an approachable introduction. </section> </section> <section id="conclusion"> <h2>Conclusion</h2> Now that I've described the DSL, here are the rules that are equivalent to the imperative code in the motivation section: <pre class="literal-block">mul_zero: int_mul(x, 0) =&gt; 0 mul_one: int_mul(x, 1) =&gt; x mul_minus_one: int_mul(x, -1) =&gt; int_neg(x) mul_pow2_const: int_mul(x, C) check C &gt; 0 and C &amp; (C - 1) == 0 shift = highest_bit(C) =&gt; int_lshift(x, shift) mul_lshift: int_mul(x, int_lshift(1, y)) check y.known_ge_const(0) and y.known_le_const(LONG_BIT) =&gt; int_lshift(x, y)</pre> The current status of the DSL is that it got merged to PyPy's main branch. I rewrote a part of the integer rewrites <a class="reference external" href="https://github.com/pypy/pypy/blob/d92d0bfd38318ede1cbaadadafd77da69d431fad/rpython/jit/metainterp/ruleopt/real.rules">into the DSL</a>, but some are still in the old imperative style (mostly for complicated reasons, the easily ported ones are all done). Since I've only been porting optimizations that had existed prior to the existence of the DSL, performance numbers of benchmarks didn't change. There are a number of features that are still missing and some possible extensions that I plan to work on in the future: <ul class="simple"> <li>All the integer operations that the DSL handles so far are the variants that do not check for overflow (or where overflow was proven to be impossible to happen). In regular Python code the overflow-checking variants <cite>int_add_ovf</cite> etc are much more common, but the DSL doesn't support them yet. I plan to fix this, but don't completely understand how the correctness proofs for them should be done correctly.</li> <li>A related problem is that I don't understand what it means for a rewrite to be correct if some of the operations are only defined for a subset of the input values. E.g. division isn't defined if the divisor is zero. In theory, a division operation in the trace should always be preceded by a check that the divisor isn't zero. But sometimes other optimization move the check around and the connection to the division gets lost or muddled. What optimizations can we still safely perform on the division? There's lots of prior work on this question, but I still don't understand what the correct approach in our context would be.</li> <li>Ordering comparisons like <code class="docutils literal">int_lt</code>, <code class="docutils literal">int_le</code> and their unsigned variants are not ported to the DSL yet. Comparisons are an area where the JIT is not super good yet at optimizing away operations. This is a pretty big topic and I've started a project with Nico Rittinghaus to try to improve the situation a bit more generally.</li> <li>A more advanced direction of work would be to implement a simplified form of <a class="reference external" href="https://egraphs-good.github.io/">e-graphs</a> (or <a class="reference external" href="https://vimeo.com/843540328">ae-graphs</a>). The JIT has like half of an e-graph data structure already, and we probably can't afford a full one in terms of compile time costs, but maybe we can have two thirds or something?</li> </ul> </section> <section id="acknowledgements"> <h2>Acknowledgements</h2> Thank you to <a class="reference external" href="https://bernsteinbear.com/">Max Bernstein</a> and <a class="reference external" href="https://martinfriedrichberger.net/">Martin Berger</a> for super helpful feedback on drafts of the post! </section></description><category>jit</category><category>z3</category><guid>https://www.pypy.org/posts/2024/10/jit-peephole-dsl.html</guid><pubDate>Wed, 23 Oct 2024 15:00:00 GMT</pubDate></item><item><title>Guest Post: How PortaOne uses PyPy for high-performance processing, connecting over 1B of phone calls every month</title><link>https://www.pypy.org/posts/2024/08/portaone.html</link><dc:creator>The PyPy Team</dc:creator><description>The PyPy project is always happy to hear about industrial use and deployments of PyPy. For the <a href="https://www.pypy.org/posts/2024/03/fixing-bug-incremental-gc.html">GC bug finding</a> task earlier this year, we collaborated with PortaOne and we're super happy that Serhii Titov, head of the QA department at PortaOne, was up to writing this guest post to describe their use and experience with the project. <hr> <h3 id="what-does-portaone-do">What does PortaOne do?</h3> We at <a href="https://www.portaone.com/">PortaOne Inc.</a> allow telecom operators to launch new services (or provide existing services more efficiently) using our VoIP platform (PortaSIP) and our real-time charging system (PortaBilling), which provides additional features for cloud PBX, such as call transfer, queues, interactive voice response (IVR) and more. At this moment our support team manages several thousand servers with our software installed in 100 countries, through which over 500 telecommunication service providers connect millions of end users every day. The unique thing about PortaOne is that we supply the source code of our product to our customers - something unheard of in the telecom world! Thus we attract "telco innovators", who use our APIs to build around the system and the source code to create unique tweaks of functionality, which produces amazing products. At the core of PortaSIP is the middle-ware component (the proper name for it is "B2BUA", but that probably does not say much to anyone outside of experts in VoIP), which implements the actual handling of SIP calls, messages, etc. and all added features (for instance, trying to send a call via telco operators through which the cost per minute is lower). It has to be fast (since even a small delay in establishing a call is noticed by a customer), reliable (everyone hates when a call drops or cannot be completed) and yet easily expandable with new functionality. This is why we decided to use Python as opposed to C/C++ or similar programming languages, which are often used in telecom equipment. The B2BUA component is a batch of similar Python processes that are looped inside a <a href="https://docs.python.org/3.10/library/asyncore.html"><code>asyncore.dispatcher</code></a> wrapper. The load balancing between these Python processes is done by our stateless SIP proxy server written in C++. All our sockets are served by this B2BUA. We have our custom client-wrappers around <code>pymysql</code>, <code>redis</code>, <code>cassandra-driver</code> and <code>requests</code> to communicate with external services. Some of the Python processes use <a href="https://cffi.readthedocs.io/en/stable/"><code>cffi</code></a> wrappers around C-code to improve their performance (examples: an Oracle DB driver, a client to a radius server, a custom C logger). The I/O operations that block the main thread of the Python processes are processed in sub-threads. We have custom wrappers around <code>threading.Thread</code> and also <code>asyncore.dispatcher</code>. The results of such operations are returned to the main thread. <h3 id="improving-our-performance-with-pypy">Improving our performance with PyPy</h3> We started with CPython and then in 2014 switched to PyPy because it was faster. Here's an exact quote from our first testing notes: "PyPy gives significant performance boost, ~50%". Nowadays, after years of changes in all the software involved, PyPy still gives us +50% boost compared to CPython. Taking care of real time traffic for so many people around the globe is something we're really proud of. I hope the PyPy team can be proud of it as well, as the PyPy product is a part of this solution. <h3 id="finding-a-garbage-collector-bug-stage-1-the-gc-hooks">Finding a garbage collector bug: stage 1, the GC hooks</h3> However our path with PyPy wasn't perfectly smooth. There were very rare cases of crashes on PyPy that we weren't able to catch. That's because to make coredump useful we needed to switch to PyPy with debug, but we cannot let it run in that mode on a production system for an extended period of time, and we did not have any STR (steps-to-reproduce) to make PyPy crash again in our lab. That's why we kept (and still keep) both interpreters installed just in case, and we would switch to CPython if we noticed it happening. At the time of updating PyPy from 3.5 to 3.6 our QA started noticing those crashes more often, but we still had no luck with STR or collecting proper coredumps with debug symbols. Then it became even worse after our development played with the <a href="https://doc.pypy.org/en/latest/gc_info.html">Garbage Collector's options</a> to increase performance of our middleware component. The crashes started to affect our regular performance testing (controlled by QA manager Yevhenii Bovda). At that point it was decided that we can no longer live like that and so we started an intense investigation. During the first stage of our investigation (following the best practice of troubleshooting) we narrowed down the issue as much as we could. So, it was not our code, it was definitely somewhere in PyPy. Eventually our SIP software engineer <a href="https://github.com/Yevhenii-Yatchenko">Yevhenii Yatchenko</a> found out that this bug is connected with the use of our <a href="https://doc.pypy.org/en/latest/gc_info.html#gc-hooks">custom hooks in the GC</a>. Yevhenii created ticket <a href="https://github.com/pypy/pypy/issues/4899">#4899</a> and within 2-3 days we got a fix from a <a href="https://github.com/cfbolz">member of the PyPy team</a>, in true open-source fashion. <h3 id="finding-a-garbage-collector-bug-stage-2-the-real-bug">Finding a garbage collector bug: stage 2, the real bug</h3> Then came stage 2. In parallel with the previous ticket, Yevhenii created <a href="https://github.com/pypy/pypy/issues/4900">#4900</a> that we still see failing with coredumps quite often, and they are not connected to GC custom hooks. In a nutshell, it took us dozens of back and forward emails, three Zoom sessions and four versions of a patch to solve the issue. During the last iteration we got a new set of options to try and a new version of the patch. Surprisingly, that helped! What a relief! So, the next logical step was to remove all debug options and run PyPy only with the patch. Unfortunately, it started to fail again and we came to the obvious conclusion that what will help us is not a patch, but one of options we were testing out. At that point we found out that <a href="https://doc.pypy.org/en/latest/gc_info.html#environment-variables"><code>PYPY_GC_MAX_PINNED=0</code></a> is a necessary and sufficient condition to solve our issue. This points to another bug in the garbage collector, somehow related to object pinning. Here's our current state: we have to add <code>PYPY_GC_MAX_PINNED=0</code>, but we do not face the crashes anymore. <h3 id="conclusion-and-next-steps">Conclusion and next steps</h3> Gratitude is extended to Carl for his invaluable assistance in resolving the nasty bugss, because it seems we're the only ones who suffered from the last one and we really did not want to fall back to CPython due to its performance disadvantage. Serhii Titov, head of the QA department at PortaOne Inc. P.S. If you are a perfectionist and at this point you have mixed feelings and you are still bothered by the question "But there might still be a bug in the GC, what about that?" - Carl has some ideas about it and he will sort it out (we will help with the testing/verification part).</description><category>casestudy</category><category>guestpost</category><guid>https://www.pypy.org/posts/2024/08/portaone.html</guid><pubDate>Thu, 29 Aug 2024 09:00:00 GMT</pubDate></item><item><title>PyPy v7.3.17 release</title><link>https://www.pypy.org/posts/2024/08/pypy-v7317-release.html</link><dc:creator>mattip</dc:creator><description><section id="pypy-v7-3-17-release-of-python-2-7-and-3-10"> <h2>PyPy v7.3.17: release of python 2.7 and 3.10</h2> The PyPy team is proud to release version 7.3.17 of PyPy. This release includes a new <a class="reference internal" href="https://www.pypy.org/posts/2024/08/pypy-v7317-release.html#risc-v-jit-backend">RISC-V JIT backend</a>, an <a class="reference internal" href="https://www.pypy.org/posts/2024/08/pypy-v7317-release.html#improved-repl">improved REPL</a> based on work by the CPython team, and <a class="reference internal" href="https://www.pypy.org/posts/2024/08/pypy-v7317-release.html#better-jit-optimizations">better JIT optimizations</a> of integer operations. Special shout-outs to <a class="reference external" href="https://github.com/loganchien">Logan Chien</a> for the <a class="reference external" href="https://github.com/pypy/pypy/pull/5002">RISC-V backend work</a>, to <a class="reference external" href="https://github.com/nirit100">Nico Rittinghaus</a> for better integer optimization in the JIT, and the CPython team that has worked on the repl. The release includes two different interpreters: <ul class="simple"> <li>PyPy2.7, which is an interpreter supporting the syntax and the features of Python 2.7 including the stdlib for CPython 2.7.18+ (the <code class="docutils literal">+</code> is for backported security updates)</li> <li>PyPy3.10, which is an interpreter supporting the syntax and the features of Python 3.10, including the stdlib for CPython 3.10.14.</li> </ul> The interpreters are based on much the same codebase, thus the dual release. This is a micro release, all APIs are compatible with the other 7.3 releases. It follows after 7.3.16 release on April 23, 2024. We recommend updating. You can find links to download the releases here: <blockquote> <a class="reference external" href="https://pypy.org/download.html">https://pypy.org/download.html</a> </blockquote> We would like to thank our donors for the continued support of the PyPy project. If PyPy is not quite good enough for your needs, we are available for <a class="reference external" href="https://www.pypy.org/pypy-sponsors.html">direct consulting</a> work. If PyPy is helping you out, we would love to hear about it and encourage submissions to our <a class="reference external" href="https://pypy.org/blog">blog</a> via a pull request to <a class="reference external" href="https://github.com/pypy/pypy.org">https://github.com/pypy/pypy.org</a> We would also like to thank our contributors and encourage new people to join the project. PyPy has many layers and we need help with all of them: bug fixes, <a class="reference external" href="https://doc.pypy.org/">PyPy</a> and <a class="reference external" href="https://rpython.readthedocs.org">RPython</a> documentation improvements, or general <a class="reference external" href="https://doc.pypy.org/en/latest/project-ideas.html">help</a> with making RPython's JIT even better. If you are a python library maintainer and use C-extensions, please consider making a <a class="reference external" href="https://hpyproject.org/">HPy</a> / <a class="reference external" href="https://cffi.readthedocs.io">CFFI</a> / <a class="reference external" href="https://cppyy.readthedocs.io">cppyy</a> version of your library that would be performant on PyPy. In any case, both <a class="reference external" href="https://github.com/joerick/cibuildwheel">cibuildwheel</a> and the <a class="reference external" href="https://github.com/matthew-brett/multibuild">multibuild system</a> support building wheels for PyPy. <section id="risc-v-backend-for-the-jit"> <h3>RISC-V backend for the JIT</h3> PyPy's JIT has added support for generating 64-bit RISC-V machine code at runtime (RV64-IMAD, specifically). So far we are not releasing binaries for any RISC-V platforms, but there are <a class="reference external" href="https://rpython.readthedocs.io/en/latest/riscv.html">instructions</a> on how to cross-compile binaries. </section> <section id="repl-improvements"> <h3>REPL Improvements</h3> The biggest user-visible change of the release is new features in the repl of PyPy3.10. CPython 3.13 has adopted and extended PyPy's pure-Python repl, adding a number of features and fixing a number or bugs in the process. We have backported and added the following features: <ul class="simple"> <li>Prompts and tracebacks use terminal colors, as well as <a class="reference external" href="https://gist.github.com/egmontkob/eb114294efbcd5adb1944c9f3cb5feda">terminal hyperlinks</a> for file names.</li> <li><a class="reference external" href="https://en.wikipedia.org/wiki/Bracketed-paste">Bracketed paste</a> enable pasting several lines of input into the terminal without auto-indentation getting in the way.</li> <li>A special interactive help browser (F1), history browser (F2), explicit paste mode (F3).</li> <li>Support for Ctrl-&lt;left/right&gt; to jump over whole words at a time.</li> </ul> See the <a class="reference external" href="https://docs.python.org/3.13/whatsnew/3.13.html#a-better-interactive-interpreter">CPython documentation for further details</a>. Thanks to Łukasz Langa, Pablo Galindo Salgado and the other CPython devs involved in this work. </section> <section id="better-jit-optimizations-of-integer-operations"> <h3>Better JIT optimizations of integer operations</h3> The optimizers of PyPy's JIT have become much better at reasoning about and optimizing integer operations. This is done with a new <a class="reference external" href="https://pypy.org/posts/2024/08/toy-knownbits.html">"knownbits" abstract domain</a>. In many programs that do bit-manipulation of integers, some of the bits of the integer variables of the program can be statically known. Here's a simple example: <div class="code"><pre class="code python"><a id="rest_code_bf930ab507984607a20043060866170d-1" name="rest_code_bf930ab507984607a20043060866170d-1" href="https://www.pypy.org/posts/2024/08/pypy-v7317-release.html#rest_code_bf930ab507984607a20043060866170d-1"></a>x = a | 1 <a id="rest_code_bf930ab507984607a20043060866170d-2" name="rest_code_bf930ab507984607a20043060866170d-2" href="https://www.pypy.org/posts/2024/08/pypy-v7317-release.html#rest_code_bf930ab507984607a20043060866170d-2"></a>... <a id="rest_code_bf930ab507984607a20043060866170d-3" name="rest_code_bf930ab507984607a20043060866170d-3" href="https://www.pypy.org/posts/2024/08/pypy-v7317-release.html#rest_code_bf930ab507984607a20043060866170d-3"></a>if x &amp; 1: <a id="rest_code_bf930ab507984607a20043060866170d-4" name="rest_code_bf930ab507984607a20043060866170d-4" href="https://www.pypy.org/posts/2024/08/pypy-v7317-release.html#rest_code_bf930ab507984607a20043060866170d-4"></a> ... <a id="rest_code_bf930ab507984607a20043060866170d-5" name="rest_code_bf930ab507984607a20043060866170d-5" href="https://www.pypy.org/posts/2024/08/pypy-v7317-release.html#rest_code_bf930ab507984607a20043060866170d-5"></a>else: <a id="rest_code_bf930ab507984607a20043060866170d-6" name="rest_code_bf930ab507984607a20043060866170d-6" href="https://www.pypy.org/posts/2024/08/pypy-v7317-release.html#rest_code_bf930ab507984607a20043060866170d-6"></a> ... </pre></div> With the new abstract domain, the JIT can optimize the <code class="docutils literal">if</code>-condition to <code class="docutils literal">True</code>, because it already knows that the lowest bit of <code class="docutils literal">x</code> must be set. This optimization applies to all Python-integers that fit into a machine word (PyPy optimistically picks between two different representations for <code class="docutils literal">int</code>, depending on the size of the value). Unfortunately there is very little impact of this change on almost all Python code, because intensive bit-manipulation is rare in Python. However, the change leads to significant performance improvements in <a class="reference external" href="https://docs.pydrofoil.org/en/latest/">Pydrofoil</a> (the RPython-based RISC-V/ARM emulators that are automatically generated from high-level <a class="reference external" href="https://github.com/rems-project/sail/">Sail</a> specifications of the respective ISAs, and that use the RPython JIT to improve performance). </section> <section id="pypy-versions-and-speed-pypy-org"> <h3>PyPy versions and speed.pypy.org</h3> The keen-eyed will have noticed no mention of Python version 3.9 in the releases above. Typically we will maintain only one version of Python3, but due to PyPy3.9 support on conda-forge we maintained multiple versions from the first release of PyPy3.10 in PyPy v7.3.12 (Dec 2022). Conda-forge is <a class="reference external" href="https://pypy.org/posts/2024/08/conda-forge-proposes-dropping-support-for-pypy.html">sunsetting its PyPy support</a>, which means we can drop PyPy3.9. Since that was the major driver of benchmarks at <a class="reference external" href="https://speed.pypy.org">https://speed.pypy.org</a>, we revamped the site to showcase PyPy3.9, PyPy3.10, and various versions of cpython on the home page. For historical reasons, the "baseline" for comparison is still cpython 3.7.19. We will keep the buildbots building PyPY3.9 until the end of August, these builds will still be available on the <a class="reference external" href="https://buildbot.pypy.org/nightly/">nightly builds</a> tab of the buildbot. </section> <section id="what-is-pypy"> <h3>What is PyPy?</h3> PyPy is a Python interpreter, a drop-in replacement for CPython It's fast (<a class="reference external" href="https://speed.pypy.org">PyPy and CPython</a> performance comparison) due to its integrated tracing JIT compiler. We also welcome developers of other <a class="reference external" href="https://rpython.readthedocs.io/en/latest/examples.html">dynamic languages</a> to see what RPython can do for them. We provide binary builds for: <ul class="simple"> <li>x86 machines on most common operating systems (Linux 32/64 bits, Mac OS 64 bits, Windows 64 bits)</li> <li>64-bit ARM machines running Linux (<code class="docutils literal">aarch64</code>) and macos (<code class="docutils literal">macos_arm64</code>).</li> </ul> PyPy supports Windows 32-bit, Linux PPC64 big- and little-endian, Linux ARM 32 bit, RISC-V RV64IMAFD Linux, and s390x Linux but does not release binaries. Please reach out to us if you wish to sponsor binary releases for those platforms. Downstream packagers provide binary builds for debian, Fedora, conda, OpenBSD, FreeBSD, Gentoo, and more. </section> <section id="what-else-is-new"> <h3>What else is new?</h3> For more information about the 7.3.17 release, see the <a class="reference external" href="https://doc.pypy.org/en/latest/release-v7.3.17.html#changelog">full changelog</a>. Please update, and continue to help us make pypy better. Cheers, The PyPy Team </section> </section></description><category>release</category><guid>https://www.pypy.org/posts/2024/08/pypy-v7317-release.html</guid><pubDate>Wed, 28 Aug 2024 12:22:08 GMT</pubDate></item><item><title>Conda-forge proposes sunsetting support for PyPy</title><link>https://www.pypy.org/posts/2024/08/conda-forge-proposes-dropping-support-for-pypy.html</link><dc:creator>mattip</dc:creator><description>Conda-forge has kindly been providing support for PyPy since 2019. The conda-forge team has been very patient and generous with resources, but it seems the uptake of PyPy has not justified the effort. Major packages still are not <a href="https://conda-forge.org/status/migration/?name=pypy38">available on PyPy</a>, others find it hard to <a href="https://github.com/conda-forge/numpy-feedstock/pull/310">update versions</a>. We don't get much feedback at all about people using PyPy, and even less about PyPy on conda-forge. The conda-forge team has proposed <a href="https://github.com/conda-forge/conda-forge.github.io/pull/2259">sunsetting PyPy</a> going forward, which means current packages would remain but no new packages would be built. If you have an opinion, you can comment on that PR, or on this blog post. Since conda-forge supports PyPy3.9 but not PyPy3.10, we have continued releasing PyPy3.9 even though we typically support only one version of PyPy3. With the sunsetting proposal, we will not release any more updates to PyPy3.9. I opened a <a href="https://github.com/orgs/pypy/discussions/4998">poll</a> about the intention to drop PyPy3.9. If you have an opinion, please chime in.</description><category>conda-forge</category><guid>https://www.pypy.org/posts/2024/08/conda-forge-proposes-dropping-support-for-pypy.html</guid><pubDate>Fri, 09 Aug 2024 06:27:41 GMT</pubDate></item></channel></rss>

CINXE.COM

PyPy