============================================================================ PyPy v7.3.18: release of python 2.7, 3.10 and 3.11 beta, released 2025-02-06 ============================================================================ .. updated to 30e1d327015bf68 The PyPy team is proud to release version 7.3.18 of PyPy. This release includes a python 3.11 interpreter. We are labelling it "beta" because it is the first one. In the next release we will drop 3.10 and remove the "beta" label. There are a particularly large set of bugfixes in this release thanks to @devdanzin using fusil on the 3.10 builds, originally written by Victor Stinner. Other significant changes: - We have updated libffi shipped in our portable builds. We also now statically link to libffi where possible which reduces the number of shared object dependencies. - We have added code to be able to show the native function names when profiling with VMProf. So far only Linux supports this feature. - We have added a `PEP 768`_-inspired remote debugging facility. See :doc:`remotedebugging`. - The HPy backend has been updated to latest HPy HEAD The release includes three different interpreters: - PyPy2.7, which is an interpreter supporting the syntax and the features of Python 2.7 including the stdlib for CPython 2.7.18+ (the ``+`` is for backported security updates) - PyPy3.10, which is an interpreter supporting the syntax and the features of Python 3.10, including the stdlib for CPython 3.10.16. - PyPy3.11, which is an interpreter supporting the syntax and the features of Python 3.11, including the stdlib for CPython 3.11.11. The interpreters are based on much the same codebase, thus the triple release. This is a micro release, all APIs are compatible with the other 7.3 releases. It follows after 7.3.17 release on August 28, 2024. We recommend updating. You can find links to download the releases here: https://pypy.org/download.html We would like to thank our donors for the continued support of the PyPy project. If PyPy is not quite good enough for your needs, we are available for `direct consulting`_ work. If PyPy is helping you out, we would love to hear about it and encourage submissions to our blog_ via a pull request to https://github.com/pypy/pypy.org We would also like to thank our contributors and encourage new people to join the project. PyPy has many layers and we need help with all of them: bug fixes, `PyPy`_ and `RPython`_ documentation improvements, or general `help`_ with making RPython's JIT even better. If you are a python library maintainer and use C-extensions, please consider making a HPy_ / CFFI_ / cppyy_ version of your library that would be performant on PyPy. In any case, both `cibuildwheel`_ and the `multibuild system`_ support building wheels for PyPy. .. _`PyPy`: index.html .. _`RPython`: https://rpython.readthedocs.org .. _`help`: project-ideas.html .. _CFFI: https://cffi.readthedocs.io .. _cppyy: https://cppyy.readthedocs.io .. _`multibuild system`: https://github.com/matthew-brett/multibuild .. _`cibuildwheel`: https://github.com/joerick/cibuildwheel .. _blog: https://pypy.org/blog .. _HPy: https://hpyproject.org/ .. _direct consulting: https://www.pypy.org/pypy-sponsors.html VMProf Native Symbol Names =========================== When running VMProf profiling with native profiling enabled, PyPy did so far not produce function names for C functions. The output looked like this:: pypy -m vmprof ~/projects/gitpypy/lib-python/2.7/test/pystone.py Pystone(1.1) time for 50000 passes = 0.0109887 This machine benchmarks at 4.55011e+06 pystones/second vmprof output: %: name: location: 100.0% entry_point /app_main.py:874 100.0% run_command_line /app_main.py:601 100.0% run_toplevel /app_main.py:93 100.0% _run_module_as_main /home/user/bin/pypy-c-jit-170203-99a72243b541-linux64/lib-python/2.7/runpy.py:150 100.0% _run_code /home/user/bin/pypy-c-jit-170203-99a72243b541-linux64/lib-python/2.7/runpy.py:62 100.0% /home/user/bin/pypy-c-jit-170203-99a72243b541-linux64/site-packages/vmprof/__main__.py:1 100.0% main /home/user/bin/pypy-c-jit-170203-99a72243b541-linux64/site-packages/vmprof/__main__.py:30 100.0% run_path /home/user/bin/pypy-c-jit-170203-99a72243b541-linux64/lib-python/2.7/runpy.py:238 100.0% _run_module_code /home/user/bin/pypy-c-jit-170203-99a72243b541-linux64/lib-python/2.7/runpy.py:75 100.0% /home/user/projects/gitpypy/lib-python/2.7/test/pystone.py:3 100.0% main /home/user/projects/gitpypy/lib-python/2.7/test/pystone.py:60 100.0% pystones /home/user/projects/gitpypy/lib-python/2.7/test/pystone.py:67 100.0% Proc0 /home/user/projects/gitpypy/lib-python/2.7/test/pystone.py:79 76.9% 69.2% 53.8% 53.8% 46.2% 46.2% 38.5% 38.5% Proc8 /home/user/projects/gitpypy/lib-python/2.7/test/pystone.py:212 30.8% ... We can now symbolify these C functions and give function names and which shared library they come from, at least on Linux:: Pystone(1.1) time for 50000 passes = 0.218967 This machine benchmarks at 228345 pystones/second vmprof output: %: name: location: 100.0% entry_point /app_main.py:889 100.0% run_command_line /app_main.py:616 100.0% run_toplevel /app_main.py:95 100.0% _run_module_as_main /home/user/projects/gitpypy/lib-python/2.7/runpy.py:150 100.0% _run_code /home/user/projects/gitpypy/lib-python/2.7/runpy.py:62 100.0% /home/user/projects/gitpypy/site-packages/vmprof/__main__.py:1 100.0% main /home/user/projects/gitpypy/site-packages/vmprof/__main__.py:30 100.0% run_module /home/user/projects/gitpypy/lib-python/2.7/runpy.py:179 100.0% _run_module_code /home/user/projects/gitpypy/lib-python/2.7/runpy.py:75 100.0% /home/user/projects/gitpypy/lib-python/2.7/test/pystone.py:3 100.0% main /home/user/projects/gitpypy/lib-python/2.7/test/pystone.py:60 100.0% pystones /home/user/projects/gitpypy/lib-python/2.7/test/pystone.py:67 100.0% Proc0 /home/user/projects/gitpypy/lib-python/2.7/test/pystone.py:79 95.5% n:pypy_g_execute_frame:0:pypy-c 91.4% n:pypy_g_PyFrame_dispatch:0:pypy-c 63.8% n:pypy_g_PyFrame_dispatch_bytecode:0:pypy-c 49.8% Proc1 /home/user/projects/gitpypy/lib-python/2.7/test/pystone.py:137 17.6% copy /home/user/projects/gitpypy/lib-python/2.7/test/pystone.py:53 13.6% n:pypy_g_PyFrame_CALL_FUNCTION:0:pypy-c 10.4% Proc8 /home/user/projects/gitpypy/lib-python/2.7/test/pystone.py:212 8.6% n:pypy_g_STORE_ATTR_slowpath:0:pypy-c This becomes even more useful when using the `VMProf Firefox converter`_, which uses the Firefox Profiler Web UI to visualize profiling output: .. image:: image/2025-vmprof-firefox.png .. _`VMProf Firefox converter`: https://github.com/Cskorpion/vmprof-firefox-converter/ What is PyPy? ============= PyPy is a Python interpreter, a drop-in replacement for CPython It's fast (`PyPy and CPython`_ performance comparison) due to its integrated tracing JIT compiler. We also welcome developers of other `dynamic languages`_ to see what RPython can do for them. We provide binary builds for: * **x86** machines on most common operating systems (Linux 32/64 bits, Mac OS 64 bits, Windows 64 bits) * 64-bit **ARM** machines running Linux (``aarch64``) and macos (``macos_arm64``). PyPy supports Windows 32-bit, Linux PPC64 big- and little-endian, Linux ARM 32 bit, RISC-V RV64IMAFD Linux, and s390x Linux but does not release binaries. Please reach out to us if you wish to sponsor binary releases for those platforms. Downstream packagers provide binary builds for debian, Fedora, conda, OpenBSD, FreeBSD, Gentoo, and more. .. _`PyPy and CPython`: https://speed.pypy.org .. _`dynamic languages`: https://rpython.readthedocs.io/en/latest/examples.html Changelog ========= For all versions ---------------- - Update cffi backend to 1.18.0-dev - Refactor use of Python2 unicode in rpython to use only utf-8. This affects windows calls to ``rposix`` routines. Now all the system calls on windows should use the ``FunctionW`` variants instead of the ``FunctionA`` ones. - Update to vmprof-0.4.17 - Update vmprof native profiling to show more native symbols - Implement `PEP 768`_-inspired remote debugging facility. See :doc:`remotedebugging`. - Add many more int optimization rules. - Bump ``macosx-version-min`` to 10.13 on 2.7 to match 3.10, 3.11 Bugfixes ~~~~~~~~ - Make sure that tracing tail-recursive infinite recursion ends (:issue:`5021`) - Revive ``tools/gcdump.py`` which uses ``PYPYLOG`` - Fix ``socket.socket.sendto`` for ``AF_PACKET`` protocol (:issue:`5024`) - Fix ``inf``/``nan`` formatting with thousands separator (:issue:`5018`) - Fixup int/long confusion on 32-bit builds - Fix the ``gc.get_stats`` output to not show incorrect numbers (:issue:`5005`) - Use simple interactive console if ``stdin`` is closed (:issue:`2981`) - Use ``HOMEBREW_CELLAR`` to find ``tcl`` library on macOS (:issue:`5096`) - Don't segfault in unicodedb when looking up invalid codepoints (:issue:`5113`) - Fix segfault in ``pyexpat`` (:issue:`5112`) - Guard against list mutation in the list ``repr`` (:issue:`5117`) - Check input for divide-by-zero in ``__pypy__.intops`` (:issue:`5129`) - Check input for valid c in ``mulmod(a, b, c)``, ``mod``, and ``floordiv`` (:issue:`5128`) Speedups and enhancements ~~~~~~~~~~~~~~~~~~~~~~~~~ - Make the opencoder encoding support varsized ints. This shrinks the memory usage of the JIT and is supposed to support really long trace limits without recompiling pypy - Implement a ``try_cast_erased`` function in RPython - Copy CPython's ``threading`` implementation for windows - Only use ``largefile`` glibc interfaces on 32-bit build (:issue:`5071`) - Add a DSL for integer optimizations, use it to create some optimizations, see the blogpost_ - Optimize overflowing ``int*int`` multiplication that produces a ``long`` result - Resolve names of native functions in vmprof (:issue:`5114`) - Make internal streamio ``replace_crlf_with_lf`` do only one copy, not two since we often read programmatically generated python code. - Avoid keeping refs on the frame stack when calling functions - Fix ``reverse`` JIT unrolling, which was done too eagerly - Write ``abs(int)`` in a branch-free way, producing fewer JIT traces - On windows, use ``wchar_t`` for ``main(..., argv)``. Solves long standing issue around calling ``pypy.exe .py`` - Fix cffi backend for struct-in-a-struct (:issue:`python-cffi/cffi#147`) - Make ``str.format`` somewhat more jit friendly - Use ``BCryptGenRandom`` on windows in ``rurandom`` (:issue:`5039`) - If mode is 0700 in ``mkdir`` on windows, restrict the directory to the current user - Add support for unicode version 14 to RPython .. _blogpost: https://pypy.org/posts/2024/10/jit-peephole-dsl.html .. _`PEP 768`: https://peps.python.org/pep-0768/ Python 3.10 ----------- Bugfixes ~~~~~~~~ - Sync ``Python.h`` with upstream, add ``import.h`` (:issue:`5013`) - Prefer static sysconfigdata if it exists and do not create static sysconfigdata on portable builds (:issue:`5015`) - Fix python2-isms in ``complex``, in both str and format - Do not segfault in ``reversed.__setstate__`` (:issue:`5029`) - Fix weird edge case where a ``index`` of a ``memoryview`` releases the underlying buffer - Fix ``_ssl.read`` when ``get_shutdown`` is true, should no longer error - Always initialize locale by calling ``setlocale(LC_CTYPE, '')`` - Fix when metaclass ``__new__`` has extra args - Fix ``venv`` when src is a source build - Also create python.exe and python3.exe when creating a venv in a source build on windows - Sync ``'user_base'`` between ``site.py`` and ``sysconfig.py`` on windows (:issue:`5073`) - Allow unterminated string error to propagate in the tokenizer (:issue:`5076`) - Fix subtle problem with ``Py_SIZE(PyListObject)`` since it assumes ``PyListObject`` is a ``PyVarObject``. - Fix race in ``Thread.join()`` (:issue:`5080`) and ``threadpool`` stopping (:issue:`4994`) - Fix logic in packaging ``tklib`` for darwin (:issue:`5082`) - Fix an infinite loop in the jump threading optimization in the bytecode compiler (:issue:`5090`) - Make ``__doc__`` a proper descr on methods - Limit ``_string.formatter*`` functions to reject ``bytes`` (:issue:`5111`) - Remove internal calls of ``utf8(bytes)``, fix error handler, add test (:issue:`5110`, :issue:`5111`) - Make ``linecache.checkcache`` more resilient in the presence of ``__del__`` and multithreading (:issue:`5109`) - Remove extraneous exports from ``_dbm.__all__`` (:issue:`5115`) - Add missing ``_ensure_initialized`` in ``_curses.putp`` (:issue:`5116`) - Check ``self.ssl`` for pathological use of ``_ssl`` (:issue:`5124`) - Use ``os.fsencode`` for ``dbm.*.open`` (:issue:`5115`) - Check for NULL ssl certificate (:issue:`5120`) - Check before calling ``_dealloc_warn_w`` (:issue:`5123`) - Fix ``_curses`` tests and hide ``_mk_acs`` (:issue:`5122`) - Use ``pkg-config`` for cffi ``_tkinter``, fix tk/tcl_path for portable builds (:issue:`5064`, :issue:`5096`) - Test, fix infinite recursion when creating a ``pyobj`` from ``w_obj`` when the ``pyobj`` is a list subtype that overrides ``__len__`` in c - Be more careful in the order of ``StringIO.__init__`` since decoding can fail (:issue:`5126`) - Refactor hashlib ``_keccak_init`` to be a regular class method (:issue:`5127`) - Fix ``list.pop`` and ``list.insert`` to use ``__index__`` - Check code validity in ``_pickle_support.builtin_code`` (:issue:`5130`) - Check for bad result when calling ``nl_langinfo`` (:issue:`5132`) - Backport CPython fix to not write incomplete pyc files (:issue:`python/cpython#126606`) - Do not initialize values if stringio newline is wrong (:issue:`5140`) - Initialize buffer view ``readonly`` flag properly (:issue:`5136`) - Only increment ``pos`` in ``PyDict_Next`` after checking len (:issue:`5142`) - In builtin ``hex(x)``, ``oct(x)``, ``bin(x)``, use ``int.__format__`` not ``x.__format__`` - Invalidate the method cache in the presence of overridden mros (:issue:`5149`) - Fix new failure in ``lib-python/3/test/test_descr`` when ``mro()`` sets ``__bases__`` - Avoid segfault when creating memoryview of ctypes array with 0 shape (:issue:`5156`) - Fix signature of ``sendfunc`` - Backport the CPython fixes to ``pyrepl`` to PyPy (:issue:`4990`) - Fix ``win32console._write`` return value (:issue: `5139`) - Backport changes to ``site.py`` from CPython3.13 to get pyrepl's ``PYTHON_HISTORY`` working Speedups and enhancements ~~~~~~~~~~~~~~~~~~~~~~~~~ - Move ``_remove_dead_weakref`` to the _weakref module (:issue:`5068`) - Make ``unicodedata.normalize`` O(1) for ascii strings (:issue:`5052`) - Add ``PyContextVar_Reset``, ``Py_FatalError`` (:issue:`5081`) (:issue:`5086`) - Make ``Py_FatalError`` a macro that adds the current function name, like CPython - Many error message tweaks for test compliance with CPython - Make unmarshaling use ``unrolling_iterable`` instead of a function ptr table, which should speed it up slightly. - Add ``_ssl.keylog_filename`` which is useful for debugging ssl problems (:issue:`5141`) - Allocate less when using ``PyErr_NoMemory`` to raise an error rather than segfault - Add docstrings to tupleobject - Add audit events to ``syslog`` and ``_sqlite3``