Merge branch 'main' into support-halffloat-arrays/146238

Author: skirpichev

.github/workflows/build.yml

@@ -369,7 +369,7 @@ jobs:
           sudo xcode-select --switch /Applications/Xcode_15.4.app
 
       - name: Build and test
-        run: python3 Apple ci iOS --fast-ci --simulator 'iPhone SE (3rd generation),OS=17.5'
+        run: python3 Platforms/Apple ci iOS --fast-ci --simulator 'iPhone SE (3rd generation),OS=17.5'
 
   build-emscripten:
     name: 'Emscripten'

.pre-commit-config.yaml

@@ -3,9 +3,9 @@ repos:
     rev: a27a2e47c7751b639d2b5badf0ef6ff11fee893f  # frozen: v0.15.4
     hooks:
       - id: ruff-check
-        name: Run Ruff (lint) on Apple/
-        args: [--exit-non-zero-on-fix, --config=Apple/.ruff.toml]
-        files: ^Apple/
+        name: Run Ruff (lint) on Platforms/Apple/
+        args: [--exit-non-zero-on-fix, --config=Platforms/Apple/.ruff.toml]
+        files: ^Platforms/Apple/
       - id: ruff-check
         name: Run Ruff (lint) on Doc/
         args: [--exit-non-zero-on-fix]
@@ -39,9 +39,9 @@ repos:
         args: [--exit-non-zero-on-fix, --config=Tools/wasm/.ruff.toml]
         files: ^Tools/wasm/
       - id: ruff-format
-        name: Run Ruff (format) on Apple/
-        args: [--exit-non-zero-on-fix, --config=Apple/.ruff.toml]
-        files: ^Apple
+        name: Run Ruff (format) on Platforms/Apple/
+        args: [--exit-non-zero-on-fix, --config=Platforms/Apple/.ruff.toml]
+        files: ^Platforms/Apple/
       - id: ruff-format
         name: Run Ruff (format) on Doc/
         args: [--exit-non-zero-on-fix]

Doc/c-api/bytearray.rst

@@ -44,6 +44,10 @@ Direct API functions
 
    On failure, return ``NULL`` with an exception set.
 
+   .. note::
+      If the object implements the buffer protocol, then the buffer
+      must not be mutated while the bytearray object is being created.
+
 
 .. c:function:: PyObject* PyByteArray_FromStringAndSize(const char *string, Py_ssize_t len)
 
@@ -58,6 +62,10 @@ Direct API functions
 
    On failure, return ``NULL`` with an exception set.
 
+   .. note::
+      If the object implements the buffer protocol, then the buffer
+      must not be mutated while the bytearray object is being created.
+
 
 .. c:function:: Py_ssize_t PyByteArray_Size(PyObject *bytearray)
 
@@ -70,6 +78,9 @@ Direct API functions
    ``NULL`` pointer.  The returned array always has an extra
    null byte appended.
 
+   .. note::
+      It is not thread-safe to mutate the bytearray object while using the returned char array.
+
 
 .. c:function:: int PyByteArray_Resize(PyObject *bytearray, Py_ssize_t len)
 
@@ -89,6 +100,9 @@ These macros trade safety for speed and they don't check pointers.
 
    Similar to :c:func:`PyByteArray_AsString`, but without error checking.
 
+   .. note::
+      It is not thread-safe to mutate the bytearray object while using the returned char array.
+
 
 .. c:function:: Py_ssize_t PyByteArray_GET_SIZE(PyObject *bytearray)
 

Doc/c-api/bytes.rst

@@ -127,6 +127,10 @@ called with a non-bytes parameter.
    Return the bytes representation of object *o* that implements the buffer
    protocol.
 
+   .. note::
+      If the object implements the buffer protocol, then the buffer
+      must not be mutated while the bytes object is being created.
+
 
 .. c:function:: Py_ssize_t PyBytes_Size(PyObject *o)
 
@@ -185,13 +189,20 @@ called with a non-bytes parameter.
    created, the old reference to *bytes* will still be discarded and the value
    of *\*bytes* will be set to ``NULL``; the appropriate exception will be set.
 
+   .. note::
+      If *newpart* implements the buffer protocol, then the buffer
+      must not be mutated while the new bytes object is being created.
 
 .. c:function:: void PyBytes_ConcatAndDel(PyObject **bytes, PyObject *newpart)
 
    Create a new bytes object in *\*bytes* containing the contents of *newpart*
    appended to *bytes*.  This version releases the :term:`strong reference`
    to *newpart* (i.e. decrements its reference count).
 
+   .. note::
+      If *newpart* implements the buffer protocol, then the buffer
+      must not be mutated while the new bytes object is being created.
+
 
 .. c:function:: PyObject* PyBytes_Join(PyObject *sep, PyObject *iterable)
 
@@ -210,6 +221,9 @@ called with a non-bytes parameter.
 
    .. versionadded:: 3.14
 
+   .. note::
+      If *iterable* objects implement the buffer protocol, then the buffers
+      must not be mutated while the new bytes object is being created.
 
 .. c:function:: int _PyBytes_Resize(PyObject **bytes, Py_ssize_t newsize)
 

Doc/data/threadsafety.dat

@@ -66,10 +66,61 @@ PyList_Reverse:shared:
 # is a list
 PyList_SetSlice:shared:
 
-# Sort - per-object lock held; comparison callbacks may execute
-# arbitrary Python code
+# Sort - per-object lock held; the list is emptied before sorting
+# so other threads may observe an empty list, but they won't see the
+# intermediate states of the sort
 PyList_Sort:shared:
 
 # Extend - lock target list; also lock source when it is a
 # list, set, or dict
 PyList_Extend:shared:
+
+# Creation - pure allocation, no shared state
+PyBytes_FromString:atomic:
+PyBytes_FromStringAndSize:atomic:
+PyBytes_DecodeEscape:atomic:
+
+# Creation from formatting C primitives - pure allocation, no shared state
+PyBytes_FromFormat:atomic:
+PyBytes_FromFormatV:atomic:
+
+# Creation from object - uses buffer protocol so may call arbitrary code;
+# safe as long as the buffer is not mutated by another thread during the operation
+PyBytes_FromObject:shared:
+
+# Size - uses atomic load on free-threaded builds
+PyBytes_Size:atomic:
+PyBytes_GET_SIZE:atomic:
+
+# Raw data - no locking; mutating it is unsafe if the bytes object is shared between threads
+PyBytes_AsString:compatible:
+PyBytes_AS_STRING:compatible:
+PyBytes_AsStringAndSize:compatible:
+
+# Concatenation - uses buffer protocol; safe as long as buffer is not mutated by another thread during the operation
+PyBytes_Concat:shared:
+PyBytes_ConcatAndDel:shared:
+PyBytes_Join:shared:
+
+# Resizing - safe if the object is unique
+_PyBytes_Resize:distinct:
+
+# Repr - atomic as bytes are immutable
+PyBytes_Repr:atomic:
+
+# Creation from object - may call arbitrary code
+PyByteArray_FromObject:shared:
+
+# Creation - pure allocation, no shared state
+PyByteArray_FromStringAndSize:atomic:
+
+# Concatenation - uses buffer protocol; safe as long as buffer is not mutated by another thread during the operation
+PyByteArray_Concat:shared:
+
+# Size - uses atomic load on free-threaded builds
+PyByteArray_Size:atomic:
+PyByteArray_GET_SIZE:atomic:
+
+# Raw data - no locking; mutating it is unsafe if the bytearray object is shared between threads
+PyByteArray_AsString:compatible:
+PyByteArray_AS_STRING:compatible:

Doc/library/json.rst

@@ -264,7 +264,7 @@ Basic Usage
 
 .. function:: load(fp, *, cls=None, object_hook=None, parse_float=None, \
                    parse_int=None, parse_constant=None, \
-                   object_pairs_hook=None, **kw)
+                   object_pairs_hook=None, array_hook=None, **kw)
 
    Deserialize *fp* to a Python object
    using the :ref:`JSON-to-Python conversion table <json-to-py-table>`.
@@ -301,6 +301,15 @@ Basic Usage
       Default ``None``.
    :type object_pairs_hook: :term:`callable` | None
 
+   :param array_hook:
+      If set, a function that is called with the result of
+      any JSON array literal decoded with as a Python list.
+      The return value of this function will be used
+      instead of the :class:`list`.
+      This feature can be used to implement custom decoders.
+      Default ``None``.
+   :type array_hook: :term:`callable` | None
+
    :param parse_float:
       If set, a function that is called with
       the string of every JSON float to be decoded.
@@ -349,7 +358,10 @@ Basic Usage
       conversion length limitation <int_max_str_digits>` to help avoid denial
       of service attacks.
 
-.. function:: loads(s, *, cls=None, object_hook=None, parse_float=None, parse_int=None, parse_constant=None, object_pairs_hook=None, **kw)
+   .. versionchanged:: next
+      Added the optional *array_hook* parameter.
+
+.. function:: loads(s, *, cls=None, object_hook=None, parse_float=None, parse_int=None, parse_constant=None, object_pairs_hook=None, array_hook=None, **kw)
 
    Identical to :func:`load`, but instead of a file-like object,
    deserialize *s* (a :class:`str`, :class:`bytes` or :class:`bytearray`
@@ -367,7 +379,7 @@ Basic Usage
 Encoders and Decoders
 ---------------------
 
-.. class:: JSONDecoder(*, object_hook=None, parse_float=None, parse_int=None, parse_constant=None, strict=True, object_pairs_hook=None)
+.. class:: JSONDecoder(*, object_hook=None, parse_float=None, parse_int=None, parse_constant=None, strict=True, object_pairs_hook=None, array_hook=None)
 
    Simple JSON decoder.
 
@@ -412,6 +424,14 @@ Encoders and Decoders
    .. versionchanged:: 3.1
       Added support for *object_pairs_hook*.
 
+   *array_hook* is an optional function that will be called with the
+   result of every JSON array decoded as a list. The return value of
+   *array_hook* will be used instead of the :class:`list`. This feature can be
+   used to implement custom decoders.
+
+   .. versionchanged:: next
+      Added support for *array_hook*.
+
    *parse_float* is an optional function that will be called with the string of
    every JSON float to be decoded.  By default, this is equivalent to
    ``float(num_str)``.  This can be used to use another datatype or parser for

Doc/library/math.rst

@@ -848,8 +848,7 @@ Constants
 
    The :mod:`!math` module consists mostly of thin wrappers around the platform C
    math library functions.  Behavior in exceptional cases follows Annex F of
-   the C99 standard, if :attr:`sys.float_info.iec_60559` is true.
-   The current implementation will raise
+   the C99 standard where appropriate.  The current implementation will raise
    :exc:`ValueError` for invalid operations like ``sqrt(-1.0)`` or ``log(0.0)``
    (where C99 Annex F recommends signaling invalid operation or divide-by-zero),
    and :exc:`OverflowError` for results that overflow (for example,

Doc/library/struct.rst

@@ -254,7 +254,7 @@ platform-dependent.
 +--------+--------------------------+--------------------+----------------+------------+
 | ``N``  | :c:type:`size_t`         | integer            |                | \(3)       |
 +--------+--------------------------+--------------------+----------------+------------+
-| ``e``  | \(6)                     | float              | 2              | \(4)       |
+| ``e``  | :c:expr:`_Float16`       | float              | 2              | \(4), \(6) |
 +--------+--------------------------+--------------------+----------------+------------+
 | ``f``  | :c:expr:`float`          | float              | 4              | \(4)       |
 +--------+--------------------------+--------------------+----------------+------------+
@@ -328,7 +328,9 @@ Notes:
    revision of the `IEEE 754 standard <ieee 754 standard_>`_. It has a sign
    bit, a 5-bit exponent and 11-bit precision (with 10 bits explicitly stored),
    and can represent numbers between approximately ``6.1e-05`` and ``6.5e+04``
-   at full precision. This type is not widely supported by C compilers: on a
+   at full precision. This type is not widely supported by C compilers:
+   it's available as :c:expr:`_Float16` type, if the compiler supports the Annex H
+   of the C23 standard.  On a
    typical machine, an unsigned short can be used for storage, but not for math
    operations. See the Wikipedia page on the `half-precision floating-point
    format <half precision format_>`_ for more information.

Doc/library/sys.rst

@@ -694,16 +694,15 @@ always available. Unless explicitly noted otherwise, all variables are read-only
    A :term:`named tuple` holding information about the float type. It
    contains low level information about the precision and internal
    representation.  The values correspond to the various floating-point
-   constants defined by C implementation and in the standard header file
-   :file:`float.h` for the 'C' programming language; see Annex F and section
-   5.2.4.2.2 of the 1999 ISO/IEC C standard [C99]_, 'Characteristics of
-   floating types', for details.
+   constants defined in the standard header file :file:`float.h` for the 'C'
+   programming language; see section 5.2.4.2.2 of the 1999 ISO/IEC C standard
+   [C99]_, 'Characteristics of floating types', for details.
 
    .. list-table:: Attributes of the :data:`!float_info` :term:`named tuple`
       :header-rows: 1
 
       * - attribute
-        - C macro
+        - float.h macro
         - explanation
 
       * - .. attribute:: float_info.epsilon
@@ -772,12 +771,6 @@ always available. Unless explicitly noted otherwise, all variables are read-only
           All other values for :c:macro:`!FLT_ROUNDS` characterize
           implementation-defined rounding behavior.
 
-      * - .. attribute:: float_info.iec_60559
-        - :c:macro:`!__STDC_IEC_559__`
-        - A boolean, indicating support the IEC 60559 floating-point standard.
-          If true, the :class:`float` type characteristics and behavior matches
-          the IEC 60559 double format.
-
    The attribute :attr:`sys.float_info.dig` needs further explanation.  If
    ``s`` is any string representing a decimal number with at most
    :attr:`!sys.float_info.dig` significant digits, then converting ``s`` to a

Doc/whatsnew/3.15.rst

@@ -84,7 +84,8 @@ Summary -- Release highlights
   <whatsnew315-pybyteswriter>`
 * :ref:`The JIT compiler has been significantly upgraded <whatsnew315-jit>`
 * :ref:`Improved error messages <whatsnew315-improved-error-messages>`
-
+* :ref:`The official Windows 64-bit binaries now use the tail-calling interpreter
+  <whatsnew315-windows-tail-calling-interpreter>`
 
 New features
 ============
@@ -805,6 +806,17 @@ inspect
   for :func:`~inspect.getdoc`.
   (Contributed by Serhiy Storchaka in :gh:`132686`.)
 
+json
+----
+
+* Add the *array_hook* parameter to  :func:`~json.load` and
+  :func:`~json.loads` functions:
+  allow a callback for JSON literal array types to customize Python lists in
+  the resulting decoded object. Passing combined :class:`frozendict` to
+  *object_pairs_hook* param and :class:`tuple` to ``array_hook`` will yield a
+  deeply nested immutable Python structure representing the JSON data.
+  (Contributed by Joao S. O. Bueno in :gh:`146440`)
+
 
 locale
 ------
@@ -1032,11 +1044,6 @@ sys
 * Add :data:`sys.abi_info` namespace to improve access to ABI information.
   (Contributed by Klaus Zimmermann in :gh:`137476`.)
 
-* Add :data:`sys.float_info.iec_60559 <sys.float_info>`: a boolean flag,
-  indicating support the IEC 60559 floating-point standard (as specified by the
-  Annex F of C99).
-  (Contributed by Sergey B Kirpichev in :gh:`138580`.)
-
 
 tarfile
 -------
@@ -1292,18 +1299,6 @@ zlib
 Optimizations
 =============
 
-* Builds using Visual Studio 2026 (MSVC 18) may now use the new
-  :ref:`tail-calling interpreter <whatsnew314-tail-call-interpreter>`.
-  Results on Visual Studio 18.1.1 report between
-  `15-20% <https://github.com/faster-cpython/ideas/blob/main/results/5800X-msvc.pgo2-vs-msvc.pgo.tc.svg>`__
-  speedup on the geometric mean of pyperformance on Windows x86-64 over
-  the switch-case interpreter on an AMD Ryzen 7 5800X. We have
-  observed speedups ranging from 14% for large pure-Python libraries
-  to 40% for long-running small pure-Python scripts on Windows.
-  This was made possible by a new feature introduced in MSVC 18.
-  (Contributed by Chris Eibl, Ken Jin, and Brandt Bucher in :gh:`143068`.
-  Special thanks to the MSVC team including Hulon Jenkins.)
-
 * ``mimalloc`` is now used as the default allocator for
   for raw memory allocations such as via :c:func:`PyMem_RawMalloc`
   for better performance on :term:`free-threaded builds <free-threaded build>`.
@@ -1960,6 +1955,23 @@ Build changes
   and :option:`-X dev <-X>` is passed to the Python or Python is built in :ref:`debug mode <debug-build>`.
   (Contributed by Donghee Na in :gh:`141770`.)
 
+.. _whatsnew315-windows-tail-calling-interpreter:
+
+* 64-bit builds using Visual Studio 2026 (MSVC 18) may now use the new
+  :ref:`tail-calling interpreter <whatsnew314-tail-call-interpreter>`.
+  Results on Visual Studio 18.1.1 report between
+  `15-20% <https://github.com/faster-cpython/ideas/blob/main/results/5800X-msvc.pgo2-vs-msvc.pgo.tc.svg>`__
+  speedup on the geometric mean of pyperformance on Windows x86-64 over
+  the switch-case interpreter on an AMD Ryzen 7 5800X. We have
+  observed speedups ranging from 14% for large pure-Python libraries
+  to 40% for long-running small pure-Python scripts on Windows.
+  This was made possible by a new feature introduced in MSVC 18,
+  which the official Windows 64-bit binaries on python.org__ now use.
+  (Contributed by Chris Eibl, Ken Jin, and Brandt Bucher in :gh:`143068`.
+  Special thanks to Steve Dower, and the MSVC team including Hulon Jenkins.)
+
+  __ https://www.python.org/downloads/windows/
+
 
 Porting to Python 3.15
 ======================