From 8040796b1a2390b7cfc5b8a0f89e2aff28b909c8 Mon Sep 17 00:00:00 2001 From: Charlie Lin Date: Sun, 12 Apr 2026 11:04:54 -0400 Subject: [PATCH 01/21] Add reference to freethreading-stable-abi-howto --- Doc/howto/index.rst | 1 + 1 file changed, 1 insertion(+) diff --git a/Doc/howto/index.rst b/Doc/howto/index.rst index 81fc7e63f35bd79..8ae0b66b2750c10 100644 --- a/Doc/howto/index.rst +++ b/Doc/howto/index.rst @@ -60,6 +60,7 @@ Advanced development: * :ref:`curses-howto` * :ref:`freethreading-python-howto` * :ref:`freethreading-extensions-howto` +* :ref:`freethreading-stable-abi-howto` * :ref:`isolating-extensions-howto` * :ref:`python_2.3_mro` * :ref:`socket-howto` From 7175e142a27203ae7abcf631faeaf81dc336b935 Mon Sep 17 00:00:00 2001 From: Charlie Lin Date: Sun, 12 Apr 2026 11:14:53 -0400 Subject: [PATCH 02/21] Add documentation for Free Threading Stable ABI usage This document provides detailed instructions on how to use the Free Threading Stable ABI in CPython, including guidelines for module initialization, API usage, and thread safety considerations. --- Doc/howto/freethreading-stable-abi.rst | 429 +++++++++++++++++++++++++ 1 file changed, 429 insertions(+) create mode 100644 Doc/howto/freethreading-stable-abi.rst diff --git a/Doc/howto/freethreading-stable-abi.rst b/Doc/howto/freethreading-stable-abi.rst new file mode 100644 index 000000000000000..a569c49f6ea16f0 --- /dev/null +++ b/Doc/howto/freethreading-stable-abi.rst @@ -0,0 +1,429 @@ +.. highlight:: c + +.. _freethreading-stable-abi-howto: + +****************************************** +How to Use Free Threading Stable ABI +****************************************** + +Starting with the 3.15 release, CPython has support for compiling extensions targeting +a unique kind of Stable ABI with interpreters having the :term:`global interpreter lock` (GIL) disabled. +For 3.14 and 3.13, continue compiling with the version-specific ABI. This document describes how +to adapt C API extensions to support free threading. + +Identifying the Free-Threaded Build in C +======================================== + +The CPython C API exposes the ``Py_GIL_DISABLED`` macro: in the free-threaded +build it's defined to ``1``, and in the regular build it's not defined. +You can use it to enable code that only runs under the free-threaded build:: + + #ifdef Py_GIL_DISABLED + /* code that only runs in the free-threaded build */ + #endif + +.. note:: + + On Windows, this macro is not defined automatically, but must be specified + to the compiler when building. The :func:`sysconfig.get_config_var` function + can be used to determine whether the current running interpreter had the + macro defined. + + +Module Initialization +===================== + +Extension modules need to explicitly indicate that they support running with +the GIL disabled; otherwise importing the extension will raise a warning and +enable the GIL at runtime. + +There are two ways to indicate that an extension module supports running with +the GIL disabled depending on whether the extension uses multi-phase or +single-phase initialization. + +Multi-Phase Initialization +.......................... + +Extensions that use :ref:`multi-phase initialization ` +(functions like :c:func:`PyModuleDef_Init`, +:c:func:`PyModExport_* ` export hook, +:c:func:`PyModule_FromSlotsAndSpec`) should add a +:c:data:`Py_mod_gil` slot in the module definition. +If your extension supports older versions of CPython, +you should guard the slot with a :c:data:`PY_VERSION_HEX` check. + +:: + + static struct PyModuleDef_Slot module_slots[] = { + ... + #if PY_VERSION_HEX >= 0x030D0000 + {Py_mod_gil, Py_MOD_GIL_NOT_USED}, + #endif + {0, NULL} + }; + + +Single-Phase Initialization +........................... + +Extensions that use legacy :ref:`single-phase initialization ` +(that is, :c:func:`PyModule_Create`) should call :c:func:`PyUnstable_Module_SetGIL` to +indicate that they support running with the GIL disabled. The function is +only defined in the free-threaded build, so you should guard the call with +``#ifdef Py_GIL_DISABLED`` to avoid compilation errors in the regular build. + +:: + + static struct PyModuleDef moduledef = { + PyModuleDef_HEAD_INIT, + ... + }; + + PyMODINIT_FUNC + PyInit_mymodule(void) + { + PyObject *m = PyModule_Create(&moduledef); + if (m == NULL) { + return NULL; + } + #ifdef Py_GIL_DISABLED + PyUnstable_Module_SetGIL(m, Py_MOD_GIL_NOT_USED); + #endif + return m; + } + + +General API Guidelines +====================== + +Most of the C API is thread-safe, but there are some exceptions. + +* **Struct Fields**: Accessing fields in Python C API objects or structs + directly is not thread-safe if the field may be concurrently modified. +* **Macros**: Accessor macros like :c:macro:`PyList_GET_ITEM`, + :c:macro:`PyList_SET_ITEM`, and macros like + :c:macro:`PySequence_Fast_GET_SIZE` that use the object returned by + :c:func:`PySequence_Fast` do not perform any error checking or locking. + These macros are not thread-safe if the container object may be modified + concurrently. +* **Borrowed References**: C API functions that return + :term:`borrowed references ` may not be thread-safe if + the containing object is modified concurrently. See the section on + :ref:`borrowed references ` for more information. + + +Container Thread Safety +....................... + +Containers like :c:struct:`PyListObject`, +:c:struct:`PyDictObject`, and :c:struct:`PySetObject` perform internal locking +in the free-threaded build. For example, the :c:func:`PyList_Append` will +lock the list before appending an item. + +.. _PyDict_Next: + +``PyDict_Next`` +''''''''''''''' + +A notable exception is :c:func:`PyDict_Next`, which does not lock the +dictionary. You should use :c:macro:`Py_BEGIN_CRITICAL_SECTION` to protect +the dictionary while iterating over it if the dictionary may be concurrently +modified:: + + Py_BEGIN_CRITICAL_SECTION(dict); + PyObject *key, *value; + Py_ssize_t pos = 0; + while (PyDict_Next(dict, &pos, &key, &value)) { + ... + } + Py_END_CRITICAL_SECTION(); + + +Borrowed References +=================== + +.. _borrowed-references: + +Some C API functions return :term:`borrowed references `. +These APIs are not thread-safe if the containing object is modified +concurrently. For example, it's not safe to use :c:func:`PyList_GetItem` +if the list may be modified concurrently. + +The following table lists some borrowed reference APIs and their replacements +that return :term:`strong references `. + ++-----------------------------------+-----------------------------------+ +| Borrowed reference API | Strong reference API | ++===================================+===================================+ +| :c:func:`PyList_GetItem` | :c:func:`PyList_GetItemRef` | ++-----------------------------------+-----------------------------------+ +| :c:func:`PyList_GET_ITEM` | :c:func:`PyList_GetItemRef` | ++-----------------------------------+-----------------------------------+ +| :c:func:`PyDict_GetItem` | :c:func:`PyDict_GetItemRef` | ++-----------------------------------+-----------------------------------+ +| :c:func:`PyDict_GetItemWithError` | :c:func:`PyDict_GetItemRef` | ++-----------------------------------+-----------------------------------+ +| :c:func:`PyDict_GetItemString` | :c:func:`PyDict_GetItemStringRef` | ++-----------------------------------+-----------------------------------+ +| :c:func:`PyDict_SetDefault` | :c:func:`PyDict_SetDefaultRef` | ++-----------------------------------+-----------------------------------+ +| :c:func:`PyDict_Next` | none (see :ref:`PyDict_Next`) | ++-----------------------------------+-----------------------------------+ +| :c:func:`!PyWeakref_GetObject` | :c:func:`PyWeakref_GetRef` | ++-----------------------------------+-----------------------------------+ +| :c:func:`!PyWeakref_GET_OBJECT` | :c:func:`PyWeakref_GetRef` | ++-----------------------------------+-----------------------------------+ +| :c:func:`PyImport_AddModule` | :c:func:`PyImport_AddModuleRef` | ++-----------------------------------+-----------------------------------+ +| :c:func:`PyCell_GET` | :c:func:`PyCell_Get` | ++-----------------------------------+-----------------------------------+ + +Not all APIs that return borrowed references are problematic. For +example, :c:func:`PyTuple_GetItem` is safe because tuples are immutable. +Similarly, not all uses of the above APIs are problematic. For example, +:c:func:`PyDict_GetItem` is often used for parsing keyword argument +dictionaries in function calls; those keyword argument dictionaries are +effectively private (not accessible by other threads), so using borrowed +references in that context is safe. + +Some of these functions were added in Python 3.13. You can use the +`pythoncapi-compat `_ package +to provide implementations of these functions for older Python versions. + + +.. _free-threaded-memory-allocation: + +Memory Allocation APIs +====================== + +Python's memory management C API provides functions in three different +:ref:`allocation domains `: "raw", "mem", and "object". +For thread-safety, the free-threaded build requires that only Python objects +are allocated using the object domain, and that all Python objects are +allocated using that domain. This differs from the prior Python versions, +where this was only a best practice and not a hard requirement. + +.. note:: + + Search for uses of :c:func:`PyObject_Malloc` in your + extension and check that the allocated memory is used for Python objects. + Use :c:func:`PyMem_Malloc` to allocate buffers instead of + :c:func:`PyObject_Malloc`. + + +Thread State and GIL APIs +========================= + +Python provides a set of functions and macros to manage thread state and the +GIL, such as: + +* :c:func:`PyGILState_Ensure` and :c:func:`PyGILState_Release` +* :c:func:`PyEval_SaveThread` and :c:func:`PyEval_RestoreThread` +* :c:macro:`Py_BEGIN_ALLOW_THREADS` and :c:macro:`Py_END_ALLOW_THREADS` + +These functions should still be used in the free-threaded build to manage +thread state even when the :term:`GIL` is disabled. For example, if you +create a thread outside of Python, you must call :c:func:`PyGILState_Ensure` +before calling into the Python API to ensure that the thread has a valid +Python thread state. + +You should continue to call :c:func:`PyEval_SaveThread` or +:c:macro:`Py_BEGIN_ALLOW_THREADS` around blocking operations, such as I/O or +lock acquisitions, to allow other threads to run the +:term:`cyclic garbage collector `. + + +Protecting Internal Extension State +=================================== + +Your extension may have internal state that was previously protected by the +GIL. You may need to add locking to protect this state. The approach will +depend on your extension, but some common patterns include: + +* **Caches**: global caches are a common source of shared state. Consider + using a lock to protect the cache or disabling it in the free-threaded build + if the cache is not critical for performance. +* **Global State**: global state may need to be protected by a lock or moved + to thread local storage. C11 and C++11 provide the ``thread_local`` or + ``_Thread_local`` for + `thread-local storage `_. + + +Critical Sections +================= + +.. _critical-sections: + +In the free-threaded build, CPython provides a mechanism called "critical +sections" to protect data that would otherwise be protected by the GIL. +While extension authors may not interact with the internal critical section +implementation directly, understanding their behavior is crucial when using +certain C API functions or managing shared state in the free-threaded build. + +What Are Critical Sections? +........................... + +Conceptually, critical sections act as a deadlock avoidance layer built on +top of simple mutexes. Each thread maintains a stack of active critical +sections. When a thread needs to acquire a lock associated with a critical +section (e.g., implicitly when calling a thread-safe C API function like +:c:func:`PyDict_SetItem`, or explicitly using macros), it attempts to acquire +the underlying mutex. + +Using Critical Sections +....................... + +The primary APIs for using critical sections are: + +* :c:macro:`Py_BEGIN_CRITICAL_SECTION` and :c:macro:`Py_END_CRITICAL_SECTION` - + For locking a single object + +* :c:macro:`Py_BEGIN_CRITICAL_SECTION2` and :c:macro:`Py_END_CRITICAL_SECTION2` + - For locking two objects simultaneously + +These macros must be used in matching pairs and must appear in the same C +scope, since they establish a new local scope. These macros are no-ops in +non-free-threaded builds, so they can be safely added to code that needs to +support both build types. + +A common use of a critical section would be to lock an object while accessing +an internal attribute of it. For example, if an extension type has an internal +count field, you could use a critical section while reading or writing that +field:: + + // read the count, returns new reference to internal count value + PyObject *result; + Py_BEGIN_CRITICAL_SECTION(obj); + result = Py_NewRef(obj->count); + Py_END_CRITICAL_SECTION(); + return result; + + // write the count, consumes reference from new_count + Py_BEGIN_CRITICAL_SECTION(obj); + obj->count = new_count; + Py_END_CRITICAL_SECTION(); + + +How Critical Sections Work +.......................... + +Unlike traditional locks, critical sections do not guarantee exclusive access +throughout their entire duration. If a thread would block while holding a +critical section (e.g., by acquiring another lock or performing I/O), the +critical section is temporarily suspended—all locks are released—and then +resumed when the blocking operation completes. + +This behavior is similar to what happens with the GIL when a thread makes a +blocking call. The key differences are: + +* Critical sections operate on a per-object basis rather than globally + +* Critical sections follow a stack discipline within each thread (the "begin" and + "end" macros enforce this since they must be paired and within the same scope) + +* Critical sections automatically release and reacquire locks around potential + blocking operations + +Deadlock Avoidance +.................. + +Critical sections help avoid deadlocks in two ways: + +1. If a thread tries to acquire a lock that's already held by another thread, + it first suspends all of its active critical sections, temporarily releasing + their locks + +2. When the blocking operation completes, only the top-most critical section is + reacquired first + +This means you cannot rely on nested critical sections to lock multiple objects +at once, as the inner critical section may suspend the outer ones. Instead, use +:c:macro:`Py_BEGIN_CRITICAL_SECTION2` to lock two objects simultaneously. + +Note that the locks described above are only :c:type:`PyMutex` based locks. +The critical section implementation does not know about or affect other locking +mechanisms that might be in use, like POSIX mutexes. Also note that while +blocking on any :c:type:`PyMutex` causes the critical sections to be +suspended, only the mutexes that are part of the critical sections are +released. If :c:type:`PyMutex` is used without a critical section, it will +not be released and therefore does not get the same deadlock avoidance. + +Important Considerations +........................ + +* Critical sections may temporarily release their locks, allowing other threads + to modify the protected data. Be careful about making assumptions about the + state of the data after operations that might block. + +* Because locks can be temporarily released (suspended), entering a critical + section does not guarantee exclusive access to the protected resource + throughout the section's duration. If code within a critical section calls + another function that blocks (e.g., acquires another lock, performs blocking + I/O), all locks held by the thread via critical sections will be released. + This is similar to how the GIL can be released during blocking calls. + +* Only the lock(s) associated with the most recently entered (top-most) + critical section are guaranteed to be held at any given time. Locks for + outer, nested critical sections might have been suspended. + +* You can lock at most two objects simultaneously with these APIs. If you need + to lock more objects, you'll need to restructure your code. + +* While critical sections will not deadlock if you attempt to lock the same + object twice, they are less efficient than purpose-built reentrant locks for + this use case. + +* When using :c:macro:`Py_BEGIN_CRITICAL_SECTION2`, the order of the objects + doesn't affect correctness (the implementation handles deadlock avoidance), + but it's good practice to always lock objects in a consistent order. + +* Remember that the critical section macros are primarily for protecting access + to *Python objects* that might be involved in internal CPython operations + susceptible to the deadlock scenarios described above. For protecting purely + internal extension state, standard mutexes or other synchronization + primitives might be more appropriate. + +.. _per-object-locks: + +Per-Object Locks (``ob_mutex``) +............................... + +In the free-threaded build, each Python object contains a :c:member:`~PyObject.ob_mutex` +field of type :c:type:`PyMutex`. This mutex is **reserved for use by the +critical section API** (:c:macro:`Py_BEGIN_CRITICAL_SECTION` / +:c:macro:`Py_END_CRITICAL_SECTION`). + +.. warning:: + + Do **not** lock ``ob_mutex`` directly with ``PyMutex_Lock(&obj->ob_mutex)``. + Mixing direct ``PyMutex_Lock`` calls with the critical section API on the + same mutex can cause deadlocks. + +Even if your own code never uses critical sections on a particular object type, +**CPython internals may use the critical section API on any Python object**. + +If your extension type needs its own lock, add a separate :c:type:`PyMutex` +field (or another synchronization primitive) to your object struct. +:c:type:`PyMutex` is very lightweight, so there is negligible cost to having +an additional one. + +Limited C API and Stable ABI +............................ + +If you use +`setuptools `_ to build +your extension, a future version of `setuptools` will allow ``py_limited_api=True`` +to be set to allow targeting limited API when building with the free-threaded build. + + +Windows +....... + +Due to a limitation of the official Windows installer, you will need to +manually define ``Py_GIL_DISABLED=1`` when building extensions from source. + +.. seealso:: + + `Porting Extension Modules to Support Free-Threading + `_: + A community-maintained porting guide for extension authors. From e0f3bd25c53dd4ba20798d558a46bb055de49253 Mon Sep 17 00:00:00 2001 From: Charlie Lin Date: Sun, 12 Apr 2026 11:21:33 -0400 Subject: [PATCH 03/21] Update documentation on PyObject opaqueness and module init Clarified access restrictions on PyObject members and recommended functions for type and reference count manipulation. --- Doc/howto/freethreading-stable-abi.rst | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/Doc/howto/freethreading-stable-abi.rst b/Doc/howto/freethreading-stable-abi.rst index a569c49f6ea16f0..90842c84c78f0de 100644 --- a/Doc/howto/freethreading-stable-abi.rst +++ b/Doc/howto/freethreading-stable-abi.rst @@ -29,6 +29,12 @@ You can use it to enable code that only runs under the free-threaded build:: can be used to determine whether the current running interpreter had the macro defined. +`PyObject` opaqueness +===================== + +Accessing any member of `PyObject` is now prohibited, like the non-GIL +stable ABI. For instance, prefer `Py_Type()` and `Py_SET_TYPE()` over manipulating `ob_type` directly, +`Py_REFCNT`, `Py_INCREF()` and `Py_DecRef().` over `ob_refcnt`, etc. Module Initialization ===================== @@ -412,10 +418,9 @@ Limited C API and Stable ABI If you use `setuptools `_ to build -your extension, a future version of `setuptools` will allow ``py_limited_api=True`` +your extension, a future version of `setuptools` will allow ``py_limited_api=True`` to be set to allow targeting limited API when building with the free-threaded build. - Windows ....... From 3739aecfd570c2957c88f983962783972556ea63 Mon Sep 17 00:00:00 2001 From: Charlie Lin Date: Sun, 12 Apr 2026 11:23:35 -0400 Subject: [PATCH 04/21] Fix formatting for PyObject opaqueness section --- Doc/howto/freethreading-stable-abi.rst | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/Doc/howto/freethreading-stable-abi.rst b/Doc/howto/freethreading-stable-abi.rst index 90842c84c78f0de..9640a01acc847c8 100644 --- a/Doc/howto/freethreading-stable-abi.rst +++ b/Doc/howto/freethreading-stable-abi.rst @@ -29,12 +29,12 @@ You can use it to enable code that only runs under the free-threaded build:: can be used to determine whether the current running interpreter had the macro defined. -`PyObject` opaqueness +``PyObject`` opaqueness ===================== -Accessing any member of `PyObject` is now prohibited, like the non-GIL -stable ABI. For instance, prefer `Py_Type()` and `Py_SET_TYPE()` over manipulating `ob_type` directly, -`Py_REFCNT`, `Py_INCREF()` and `Py_DecRef().` over `ob_refcnt`, etc. +Accessing any member of ``PyObject`` directly is now prohibited, like the non-GIL +stable ABI. For instance, prefer ``Py_Type()`` and ``Py_SET_TYPE()`` over ``ob_type``, +``Py_REFCNT``, ``Py_INCREF()`` and ``Py_DecRef()`` over ``ob_refcnt``, etc. Module Initialization ===================== From c235a75b723cb14d3ecb70d9891fa11dad4825ed Mon Sep 17 00:00:00 2001 From: Charlie Lin Date: Sun, 12 Apr 2026 11:26:38 -0400 Subject: [PATCH 05/21] Fix formatting of code snippet in documentation --- Doc/howto/freethreading-stable-abi.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Doc/howto/freethreading-stable-abi.rst b/Doc/howto/freethreading-stable-abi.rst index 9640a01acc847c8..1de095259fe5509 100644 --- a/Doc/howto/freethreading-stable-abi.rst +++ b/Doc/howto/freethreading-stable-abi.rst @@ -418,7 +418,7 @@ Limited C API and Stable ABI If you use `setuptools `_ to build -your extension, a future version of `setuptools` will allow ``py_limited_api=True`` +your extension, a future version of ``setuptools`` will allow ``py_limited_api=True`` to be set to allow targeting limited API when building with the free-threaded build. Windows From 753156b31e3c2aff9aac9d50318a993ec1595aa9 Mon Sep 17 00:00:00 2001 From: Charlie Lin Date: Sun, 12 Apr 2026 11:27:23 -0400 Subject: [PATCH 06/21] Update documentation on PyObject opaqueness Clarified that direct access to PyObject members is prohibited. --- Doc/howto/freethreading-stable-abi.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Doc/howto/freethreading-stable-abi.rst b/Doc/howto/freethreading-stable-abi.rst index 1de095259fe5509..ebf77e6aeaee47b 100644 --- a/Doc/howto/freethreading-stable-abi.rst +++ b/Doc/howto/freethreading-stable-abi.rst @@ -30,7 +30,7 @@ You can use it to enable code that only runs under the free-threaded build:: macro defined. ``PyObject`` opaqueness -===================== +======================= Accessing any member of ``PyObject`` directly is now prohibited, like the non-GIL stable ABI. For instance, prefer ``Py_Type()`` and ``Py_SET_TYPE()`` over ``ob_type``, From 344105e9abd32ce0476101d3d909e31b73036687 Mon Sep 17 00:00:00 2001 From: Charlie Lin Date: Sun, 12 Apr 2026 12:14:00 -0400 Subject: [PATCH 07/21] Py_Type Co-authored-by: da-woods --- Doc/howto/freethreading-stable-abi.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Doc/howto/freethreading-stable-abi.rst b/Doc/howto/freethreading-stable-abi.rst index ebf77e6aeaee47b..6ad380866b0629b 100644 --- a/Doc/howto/freethreading-stable-abi.rst +++ b/Doc/howto/freethreading-stable-abi.rst @@ -33,7 +33,7 @@ You can use it to enable code that only runs under the free-threaded build:: ======================= Accessing any member of ``PyObject`` directly is now prohibited, like the non-GIL -stable ABI. For instance, prefer ``Py_Type()`` and ``Py_SET_TYPE()`` over ``ob_type``, +stable ABI. For instance, prefer ``Py_TYPE()`` and ``Py_SET_TYPE()`` over ``ob_type``, ``Py_REFCNT``, ``Py_INCREF()`` and ``Py_DecRef()`` over ``ob_refcnt``, etc. Module Initialization From 29b932d077fcea74fe09ac078c54ec09318a7641 Mon Sep 17 00:00:00 2001 From: Charlie Lin Date: Sun, 12 Apr 2026 13:11:49 -0400 Subject: [PATCH 08/21] Clarify free-threaded limited API build documentation Updated documentation to clarify the identification of free-threaded limited API builds in C, including changes to macros and initialization methods. --- Doc/howto/freethreading-stable-abi.rst | 133 +++++++++++-------------- 1 file changed, 57 insertions(+), 76 deletions(-) diff --git a/Doc/howto/freethreading-stable-abi.rst b/Doc/howto/freethreading-stable-abi.rst index 6ad380866b0629b..7ab0180f42b4ef5 100644 --- a/Doc/howto/freethreading-stable-abi.rst +++ b/Doc/howto/freethreading-stable-abi.rst @@ -11,30 +11,35 @@ a unique kind of Stable ABI with interpreters having the :term:`global interpret For 3.14 and 3.13, continue compiling with the version-specific ABI. This document describes how to adapt C API extensions to support free threading. -Identifying the Free-Threaded Build in C -======================================== +Identifying the Free-Threaded Limited API Build in C +==================================================== -The CPython C API exposes the ``Py_GIL_DISABLED`` macro: in the free-threaded +The CPython C API exposes the :c:macro:`!Py_LIMITED_API` macro: in the free-threaded stable ABI build it's defined to ``1``, and in the regular build it's not defined. You can use it to enable code that only runs under the free-threaded build:: - #ifdef Py_GIL_DISABLED - /* code that only runs in the free-threaded build */ + #ifdef Py_TARGET_ABI3T + /* code that only runs in the free-threaded stable ABI build */ #endif -.. note:: +If you wish to build youe extension with both ``abi3`` (Stable ABI with GIL) and ``abi3t`` (no-GIL stable ABI) tags, +do one of the following: + +- define both :c:macro:`!Py_LIMITED_API` and :c:macro:`!Py_TARGET_ABI3T`, or +- define only :c:macro:`!Py_LIMITED_API` and: - On Windows, this macro is not defined automatically, but must be specified - to the compiler when building. The :func:`sysconfig.get_config_var` function - can be used to determine whether the current running interpreter had the - macro defined. + - on Windows, define :c:macro:`!Py_GIL_DISABLED`; + - on other systems, use the headers of free-threaded build of Python. -``PyObject`` opaqueness +``PyObject`` and ``PyVarObject`` opaqueness ======================= Accessing any member of ``PyObject`` directly is now prohibited, like the non-GIL stable ABI. For instance, prefer ``Py_TYPE()`` and ``Py_SET_TYPE()`` over ``ob_type``, -``Py_REFCNT``, ``Py_INCREF()`` and ``Py_DecRef()`` over ``ob_refcnt``, etc. +``Py_REFCNT``, ``Py_IncRef()`` and ``Py_DecRef()`` over ``ob_refcnt``, etc. + +Similarly, members of ``PyVarObject`` are not visible. If you need any object of such type +to be passed as a ``PyObject`` parameter to any API function, cast it directly as ``PyObject``. Module Initialization ===================== @@ -43,9 +48,8 @@ Extension modules need to explicitly indicate that they support running with the GIL disabled; otherwise importing the extension will raise a warning and enable the GIL at runtime. -There are two ways to indicate that an extension module supports running with -the GIL disabled depending on whether the extension uses multi-phase or -single-phase initialization. +Multi-phase and single-phase initialization is supported to indicate that an extension module +targeting the stable ABI supports running with the GIL disabled, though the former is preferred. Multi-Phase Initialization .......................... @@ -56,48 +60,43 @@ Extensions that use :ref:`multi-phase initialization = 0x030D0000 + #ifdef Py_GIL_DISABLED {Py_mod_gil, Py_MOD_GIL_NOT_USED}, #endif {0, NULL} }; +Additionally, using :c:macro:`PyABIInfo_VAR and :c:data:`Py_mod_abi` is recommended so that an +extension module loaded for an incompatible interpreter will trigger an exception, rather than +fail with a crash. -Single-Phase Initialization -........................... - -Extensions that use legacy :ref:`single-phase initialization ` -(that is, :c:func:`PyModule_Create`) should call :c:func:`PyUnstable_Module_SetGIL` to -indicate that they support running with the GIL disabled. The function is -only defined in the free-threaded build, so you should guard the call with -``#ifdef Py_GIL_DISABLED`` to avoid compilation errors in the regular build. +.. code-block:: c -:: + #ifdef PY_VERSION_HEX >= 0x030F0000 + PyABIInfo_VAR(abi_info); + #endif Py_GIL_DISABLED - static struct PyModuleDef moduledef = { - PyModuleDef_HEAD_INIT, - ... - }; + static PyModuleDef_Slot mymodule_slots[] = { + ... + #ifdef PY_VERSION_HEX >= 0x030F0000 + {Py_mod_abi, &abi_info}, + #endif + {0, NULL} + }; - PyMODINIT_FUNC - PyInit_mymodule(void) - { - PyObject *m = PyModule_Create(&moduledef); - if (m == NULL) { - return NULL; - } - #ifdef Py_GIL_DISABLED - PyUnstable_Module_SetGIL(m, Py_MOD_GIL_NOT_USED); - #endif - return m; - } +Single-Phase Initialization +........................... +Although members of ``PyModuleDef`` is still available for no-GIL Stable ABI and can be used +for :ref:`single-phase initialization ` +(that is, :c:func:`PyModule_Create`), they are not exposed when targeting the regular Stable ABI. +Prefer multi-phased initializtion when possible. General API Guidelines ====================== @@ -106,12 +105,6 @@ Most of the C API is thread-safe, but there are some exceptions. * **Struct Fields**: Accessing fields in Python C API objects or structs directly is not thread-safe if the field may be concurrently modified. -* **Macros**: Accessor macros like :c:macro:`PyList_GET_ITEM`, - :c:macro:`PyList_SET_ITEM`, and macros like - :c:macro:`PySequence_Fast_GET_SIZE` that use the object returned by - :c:func:`PySequence_Fast` do not perform any error checking or locking. - These macros are not thread-safe if the container object may be modified - concurrently. * **Borrowed References**: C API functions that return :term:`borrowed references ` may not be thread-safe if the containing object is modified concurrently. See the section on @@ -389,43 +382,31 @@ Important Considerations internal extension state, standard mutexes or other synchronization primitives might be more appropriate. -.. _per-object-locks: +Platform-specific considerations +................................ -Per-Object Locks (``ob_mutex``) -............................... +On some platforms, Python will look for and load shared library files named +with the ``abi3`` or ``abi3t`` tag (for example, ``mymodule.abi3.so``). +:term:`Free-threaded ` interpreters only recognize the +``abi3t`` tag, while non-free-threaded ones will prefer ``abi3`` but fall back +to ``abi3t``. +Thus, extensions compatible with both ABIs should use the ``abi3t`` tag. -In the free-threaded build, each Python object contains a :c:member:`~PyObject.ob_mutex` -field of type :c:type:`PyMutex`. This mutex is **reserved for use by the -critical section API** (:c:macro:`Py_BEGIN_CRITICAL_SECTION` / -:c:macro:`Py_END_CRITICAL_SECTION`). +Python does not necessarily check that extensions it loads +have compatible ABI. +Extension authors are encouraged to add a check using the :c:macro:`Py_mod_abi` +slot or the :c:func:`PyABIInfo_Check` function. -.. warning:: - - Do **not** lock ``ob_mutex`` directly with ``PyMutex_Lock(&obj->ob_mutex)``. - Mixing direct ``PyMutex_Lock`` calls with the critical section API on the - same mutex can cause deadlocks. - -Even if your own code never uses critical sections on a particular object type, -**CPython internals may use the critical section API on any Python object**. - -If your extension type needs its own lock, add a separate :c:type:`PyMutex` -field (or another synchronization primitive) to your object struct. -:c:type:`PyMutex` is very lightweight, so there is negligible cost to having -an additional one. - -Limited C API and Stable ABI -............................ +Limited C API Build Tools +......................... If you use `setuptools `_ to build your extension, a future version of ``setuptools`` will allow ``py_limited_api=True`` to be set to allow targeting limited API when building with the free-threaded build. -Windows -....... - -Due to a limitation of the official Windows installer, you will need to -manually define ``Py_GIL_DISABLED=1`` when building extensions from source. +Other build tools will support this ABI as well: +`` .. seealso:: From 8545eeb138d830fa306f04ea96813ebbbc6ef28b Mon Sep 17 00:00:00 2001 From: Charlie Lin Date: Sun, 12 Apr 2026 13:27:27 -0400 Subject: [PATCH 09/21] Revise freethreading-stable-abi.rst for clarity Updated the documentation to clarify the use of the stable ABI and GIL management in Python extensions, including changes to member access and initialization methods. --- Doc/howto/freethreading-stable-abi.rst | 48 +++----------------------- 1 file changed, 4 insertions(+), 44 deletions(-) diff --git a/Doc/howto/freethreading-stable-abi.rst b/Doc/howto/freethreading-stable-abi.rst index 7ab0180f42b4ef5..d01ef90a7f7acd4 100644 --- a/Doc/howto/freethreading-stable-abi.rst +++ b/Doc/howto/freethreading-stable-abi.rst @@ -32,7 +32,7 @@ do one of the following: - on other systems, use the headers of free-threaded build of Python. ``PyObject`` and ``PyVarObject`` opaqueness -======================= +=========================================== Accessing any member of ``PyObject`` directly is now prohibited, like the non-GIL stable ABI. For instance, prefer ``Py_TYPE()`` and ``Py_SET_TYPE()`` over ``ob_type``, @@ -48,7 +48,7 @@ Extension modules need to explicitly indicate that they support running with the GIL disabled; otherwise importing the extension will raise a warning and enable the GIL at runtime. -Multi-phase and single-phase initialization is supported to indicate that an extension module +Multi-phase and single-phase initialization is supported to indicate that an extension module targeting the stable ABI supports running with the GIL disabled, though the former is preferred. Multi-Phase Initialization @@ -72,7 +72,7 @@ you should guard the slot with a :c:data:`Py_GIL_DISABLED` check. {0, NULL} }; -Additionally, using :c:macro:`PyABIInfo_VAR and :c:data:`Py_mod_abi` is recommended so that an +Additionally, using ``PyABIInfo_VAR`` and ``Py_mod_abi`` is recommended so that an extension module loaded for an incompatible interpreter will trigger an exception, rather than fail with a crash. @@ -119,25 +119,6 @@ Containers like :c:struct:`PyListObject`, in the free-threaded build. For example, the :c:func:`PyList_Append` will lock the list before appending an item. -.. _PyDict_Next: - -``PyDict_Next`` -''''''''''''''' - -A notable exception is :c:func:`PyDict_Next`, which does not lock the -dictionary. You should use :c:macro:`Py_BEGIN_CRITICAL_SECTION` to protect -the dictionary while iterating over it if the dictionary may be concurrently -modified:: - - Py_BEGIN_CRITICAL_SECTION(dict); - PyObject *key, *value; - Py_ssize_t pos = 0; - while (PyDict_Next(dict, &pos, &key, &value)) { - ... - } - Py_END_CRITICAL_SECTION(); - - Borrowed References =================== @@ -190,26 +171,6 @@ Some of these functions were added in Python 3.13. You can use the to provide implementations of these functions for older Python versions. -.. _free-threaded-memory-allocation: - -Memory Allocation APIs -====================== - -Python's memory management C API provides functions in three different -:ref:`allocation domains `: "raw", "mem", and "object". -For thread-safety, the free-threaded build requires that only Python objects -are allocated using the object domain, and that all Python objects are -allocated using that domain. This differs from the prior Python versions, -where this was only a best practice and not a hard requirement. - -.. note:: - - Search for uses of :c:func:`PyObject_Malloc` in your - extension and check that the allocated memory is used for Python objects. - Use :c:func:`PyMem_Malloc` to allocate buffers instead of - :c:func:`PyObject_Malloc`. - - Thread State and GIL APIs ========================= @@ -405,8 +366,7 @@ If you use your extension, a future version of ``setuptools`` will allow ``py_limited_api=True`` to be set to allow targeting limited API when building with the free-threaded build. -Other build tools will support this ABI as well: -`` +`Other build tools will support this ABI as well `_. .. seealso:: From 2a113685530cb39cbb97aa930aa19c9b507d4d18 Mon Sep 17 00:00:00 2001 From: Charlie Lin Date: Sun, 12 Apr 2026 13:36:21 -0400 Subject: [PATCH 10/21] Add freethreading-stable-abi.rst to documentation index --- Doc/howto/index.rst | 1 + 1 file changed, 1 insertion(+) diff --git a/Doc/howto/index.rst b/Doc/howto/index.rst index 8ae0b66b2750c10..906dba5140b0e53 100644 --- a/Doc/howto/index.rst +++ b/Doc/howto/index.rst @@ -37,6 +37,7 @@ Python Library Reference. mro.rst free-threading-python.rst free-threading-extensions.rst + freethreading-stable-abi.rst remote_debugging.rst General: From 2f2e7560a03f27e0553b35dd9fc64ffa49de8afb Mon Sep 17 00:00:00 2001 From: Charlie Lin Date: Fri, 1 May 2026 12:18:29 -0400 Subject: [PATCH 11/21] Simplify threading and API guidelines in documentation Removed sections on API guidelines, critical sections, and thread safety from the documentation. --- Doc/howto/freethreading-stable-abi.rst | 255 ++----------------------- 1 file changed, 17 insertions(+), 238 deletions(-) diff --git a/Doc/howto/freethreading-stable-abi.rst b/Doc/howto/freethreading-stable-abi.rst index d01ef90a7f7acd4..213ca44658cb11e 100644 --- a/Doc/howto/freethreading-stable-abi.rst +++ b/Doc/howto/freethreading-stable-abi.rst @@ -98,259 +98,37 @@ for :ref:`single-phase initialization ` (that is, :c:func:`PyModule_Create`), they are not exposed when targeting the regular Stable ABI. Prefer multi-phased initializtion when possible. -General API Guidelines -====================== -Most of the C API is thread-safe, but there are some exceptions. - -* **Struct Fields**: Accessing fields in Python C API objects or structs - directly is not thread-safe if the field may be concurrently modified. -* **Borrowed References**: C API functions that return - :term:`borrowed references ` may not be thread-safe if - the containing object is modified concurrently. See the section on - :ref:`borrowed references ` for more information. - - -Container Thread Safety -....................... - -Containers like :c:struct:`PyListObject`, -:c:struct:`PyDictObject`, and :c:struct:`PySetObject` perform internal locking -in the free-threaded build. For example, the :c:func:`PyList_Append` will -lock the list before appending an item. - -Borrowed References -=================== - -.. _borrowed-references: +Critical Sections +================= -Some C API functions return :term:`borrowed references `. -These APIs are not thread-safe if the containing object is modified -concurrently. For example, it's not safe to use :c:func:`PyList_GetItem` -if the list may be modified concurrently. +.. _critical-sections: -The following table lists some borrowed reference APIs and their replacements -that return :term:`strong references `. +Replacements: +-----------------------------------+-----------------------------------+ -| Borrowed reference API | Strong reference API | +| Macro functions | C API functions | +===================================+===================================+ -| :c:func:`PyList_GetItem` | :c:func:`PyList_GetItemRef` | +| :c:func:`Py_BEGIN_CRITICAL_SECTION` | :c:func:`PyCriticalSection_Begin` | +| :c:func:`Py_END_CRITICAL_SECTION` | |:c:func:`PyCriticalSection_End` | +-----------------------------------+-----------------------------------+ -| :c:func:`PyList_GET_ITEM` | :c:func:`PyList_GetItemRef` | +| :c:func:`Py_BEGIN_CRITICAL_SECTION2` | :c:func:`PyCriticalSection2_Begin` | +| :c:func:`Py_END_CRITICAL_SECTION2` | |:c:func:`PyCriticalSection2_End` | +-----------------------------------+-----------------------------------+ -| :c:func:`PyDict_GetItem` | :c:func:`PyDict_GetItemRef` | +| :c:func:`Py_BEGIN_CRITICAL_SECTION_MUTEX` | :c:func:`PyCriticalSection_BeginMutex` | +| :c:func:`Py_END_CRITICAL_SECTION` | |:c:func:`PyCriticalSection_End` | +-----------------------------------+-----------------------------------+ -| :c:func:`PyDict_GetItemWithError` | :c:func:`PyDict_GetItemRef` | +| :c:func:`Py_BEGIN_CRITICAL_SECTION2_MUTEX` | :c:func:`PyCriticalSection2_BeginMutex` | +| :c:func:`Py_END_CRITICAL_SECTION2` | |:c:func:`PyCriticalSection2_End` | +-----------------------------------+-----------------------------------+ -| :c:func:`PyDict_GetItemString` | :c:func:`PyDict_GetItemStringRef` | -+-----------------------------------+-----------------------------------+ -| :c:func:`PyDict_SetDefault` | :c:func:`PyDict_SetDefaultRef` | -+-----------------------------------+-----------------------------------+ -| :c:func:`PyDict_Next` | none (see :ref:`PyDict_Next`) | -+-----------------------------------+-----------------------------------+ -| :c:func:`!PyWeakref_GetObject` | :c:func:`PyWeakref_GetRef` | -+-----------------------------------+-----------------------------------+ -| :c:func:`!PyWeakref_GET_OBJECT` | :c:func:`PyWeakref_GetRef` | -+-----------------------------------+-----------------------------------+ -| :c:func:`PyImport_AddModule` | :c:func:`PyImport_AddModuleRef` | -+-----------------------------------+-----------------------------------+ -| :c:func:`PyCell_GET` | :c:func:`PyCell_Get` | -+-----------------------------------+-----------------------------------+ - -Not all APIs that return borrowed references are problematic. For -example, :c:func:`PyTuple_GetItem` is safe because tuples are immutable. -Similarly, not all uses of the above APIs are problematic. For example, -:c:func:`PyDict_GetItem` is often used for parsing keyword argument -dictionaries in function calls; those keyword argument dictionaries are -effectively private (not accessible by other threads), so using borrowed -references in that context is safe. - -Some of these functions were added in Python 3.13. You can use the -`pythoncapi-compat `_ package -to provide implementations of these functions for older Python versions. - - -Thread State and GIL APIs -========================= - -Python provides a set of functions and macros to manage thread state and the -GIL, such as: - -* :c:func:`PyGILState_Ensure` and :c:func:`PyGILState_Release` -* :c:func:`PyEval_SaveThread` and :c:func:`PyEval_RestoreThread` -* :c:macro:`Py_BEGIN_ALLOW_THREADS` and :c:macro:`Py_END_ALLOW_THREADS` - -These functions should still be used in the free-threaded build to manage -thread state even when the :term:`GIL` is disabled. For example, if you -create a thread outside of Python, you must call :c:func:`PyGILState_Ensure` -before calling into the Python API to ensure that the thread has a valid -Python thread state. - -You should continue to call :c:func:`PyEval_SaveThread` or -:c:macro:`Py_BEGIN_ALLOW_THREADS` around blocking operations, such as I/O or -lock acquisitions, to allow other threads to run the -:term:`cyclic garbage collector `. - - -Protecting Internal Extension State -=================================== - -Your extension may have internal state that was previously protected by the -GIL. You may need to add locking to protect this state. The approach will -depend on your extension, but some common patterns include: - -* **Caches**: global caches are a common source of shared state. Consider - using a lock to protect the cache or disabling it in the free-threaded build - if the cache is not critical for performance. -* **Global State**: global state may need to be protected by a lock or moved - to thread local storage. C11 and C++11 provide the ``thread_local`` or - ``_Thread_local`` for - `thread-local storage `_. - - -Critical Sections -================= - -.. _critical-sections: - -In the free-threaded build, CPython provides a mechanism called "critical -sections" to protect data that would otherwise be protected by the GIL. -While extension authors may not interact with the internal critical section -implementation directly, understanding their behavior is crucial when using -certain C API functions or managing shared state in the free-threaded build. - -What Are Critical Sections? -........................... - -Conceptually, critical sections act as a deadlock avoidance layer built on -top of simple mutexes. Each thread maintains a stack of active critical -sections. When a thread needs to acquire a lock associated with a critical -section (e.g., implicitly when calling a thread-safe C API function like -:c:func:`PyDict_SetItem`, or explicitly using macros), it attempts to acquire -the underlying mutex. - -Using Critical Sections -....................... - -The primary APIs for using critical sections are: - -* :c:macro:`Py_BEGIN_CRITICAL_SECTION` and :c:macro:`Py_END_CRITICAL_SECTION` - - For locking a single object - -* :c:macro:`Py_BEGIN_CRITICAL_SECTION2` and :c:macro:`Py_END_CRITICAL_SECTION2` - - For locking two objects simultaneously - -These macros must be used in matching pairs and must appear in the same C -scope, since they establish a new local scope. These macros are no-ops in -non-free-threaded builds, so they can be safely added to code that needs to -support both build types. - -A common use of a critical section would be to lock an object while accessing -an internal attribute of it. For example, if an extension type has an internal -count field, you could use a critical section while reading or writing that -field:: - - // read the count, returns new reference to internal count value - PyObject *result; - Py_BEGIN_CRITICAL_SECTION(obj); - result = Py_NewRef(obj->count); - Py_END_CRITICAL_SECTION(); - return result; - - // write the count, consumes reference from new_count - Py_BEGIN_CRITICAL_SECTION(obj); - obj->count = new_count; - Py_END_CRITICAL_SECTION(); - - -How Critical Sections Work -.......................... - -Unlike traditional locks, critical sections do not guarantee exclusive access -throughout their entire duration. If a thread would block while holding a -critical section (e.g., by acquiring another lock or performing I/O), the -critical section is temporarily suspended—all locks are released—and then -resumed when the blocking operation completes. - -This behavior is similar to what happens with the GIL when a thread makes a -blocking call. The key differences are: - -* Critical sections operate on a per-object basis rather than globally - -* Critical sections follow a stack discipline within each thread (the "begin" and - "end" macros enforce this since they must be paired and within the same scope) - -* Critical sections automatically release and reacquire locks around potential - blocking operations - -Deadlock Avoidance -.................. - -Critical sections help avoid deadlocks in two ways: - -1. If a thread tries to acquire a lock that's already held by another thread, - it first suspends all of its active critical sections, temporarily releasing - their locks - -2. When the blocking operation completes, only the top-most critical section is - reacquired first - -This means you cannot rely on nested critical sections to lock multiple objects -at once, as the inner critical section may suspend the outer ones. Instead, use -:c:macro:`Py_BEGIN_CRITICAL_SECTION2` to lock two objects simultaneously. - -Note that the locks described above are only :c:type:`PyMutex` based locks. -The critical section implementation does not know about or affect other locking -mechanisms that might be in use, like POSIX mutexes. Also note that while -blocking on any :c:type:`PyMutex` causes the critical sections to be -suspended, only the mutexes that are part of the critical sections are -released. If :c:type:`PyMutex` is used without a critical section, it will -not be released and therefore does not get the same deadlock avoidance. - -Important Considerations -........................ - -* Critical sections may temporarily release their locks, allowing other threads - to modify the protected data. Be careful about making assumptions about the - state of the data after operations that might block. - -* Because locks can be temporarily released (suspended), entering a critical - section does not guarantee exclusive access to the protected resource - throughout the section's duration. If code within a critical section calls - another function that blocks (e.g., acquires another lock, performs blocking - I/O), all locks held by the thread via critical sections will be released. - This is similar to how the GIL can be released during blocking calls. - -* Only the lock(s) associated with the most recently entered (top-most) - critical section are guaranteed to be held at any given time. Locks for - outer, nested critical sections might have been suspended. - -* You can lock at most two objects simultaneously with these APIs. If you need - to lock more objects, you'll need to restructure your code. - -* While critical sections will not deadlock if you attempt to lock the same - object twice, they are less efficient than purpose-built reentrant locks for - this use case. - -* When using :c:macro:`Py_BEGIN_CRITICAL_SECTION2`, the order of the objects - doesn't affect correctness (the implementation handles deadlock avoidance), - but it's good practice to always lock objects in a consistent order. - -* Remember that the critical section macros are primarily for protecting access - to *Python objects* that might be involved in internal CPython operations - susceptible to the deadlock scenarios described above. For protecting purely - internal extension state, standard mutexes or other synchronization - primitives might be more appropriate. Platform-specific considerations ................................ On some platforms, Python will look for and load shared library files named -with the ``abi3`` or ``abi3t`` tag (for example, ``mymodule.abi3.so``). -:term:`Free-threaded ` interpreters only recognize the -``abi3t`` tag, while non-free-threaded ones will prefer ``abi3`` but fall back -to ``abi3t``. +with the ``abi3`` or ``abi3t`` tag (for example, ``mymodule.abi3t.so``). +:term:`Free-threaded ` interpreters prefer ``abi3t``, +but can fall back to ``abi3``. Thus, extensions compatible with both ABIs should use the ``abi3t`` tag. Python does not necessarily check that extensions it loads @@ -365,6 +143,7 @@ If you use `setuptools `_ to build your extension, a future version of ``setuptools`` will allow ``py_limited_api=True`` to be set to allow targeting limited API when building with the free-threaded build. +``uv`` supports targeting PEP 803 as of 0.11.3: `https://github.com/astral-sh/uv/releases/tag/0.11.3`. `Other build tools will support this ABI as well `_. From 5d238106714cf57a0201530edeff4ba3aec47a3e Mon Sep 17 00:00:00 2001 From: Charlie Lin Date: Fri, 1 May 2026 12:22:34 -0400 Subject: [PATCH 12/21] Update freethreading-stable-abi.rst --- Doc/howto/freethreading-stable-abi.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/Doc/howto/freethreading-stable-abi.rst b/Doc/howto/freethreading-stable-abi.rst index 213ca44658cb11e..08033c529b841db 100644 --- a/Doc/howto/freethreading-stable-abi.rst +++ b/Doc/howto/freethreading-stable-abi.rst @@ -127,7 +127,7 @@ Platform-specific considerations On some platforms, Python will look for and load shared library files named with the ``abi3`` or ``abi3t`` tag (for example, ``mymodule.abi3t.so``). -:term:`Free-threaded ` interpreters prefer ``abi3t``, +:term:`Free-threaded ` interpreters prefer ``abi3t``, but can fall back to ``abi3``. Thus, extensions compatible with both ABIs should use the ``abi3t`` tag. @@ -143,7 +143,7 @@ If you use `setuptools `_ to build your extension, a future version of ``setuptools`` will allow ``py_limited_api=True`` to be set to allow targeting limited API when building with the free-threaded build. -``uv`` supports targeting PEP 803 as of 0.11.3: `https://github.com/astral-sh/uv/releases/tag/0.11.3`. +``uv`` supports targeting PEP 803 as of 0.11.3: ``https://github.com/astral-sh/uv/releases/tag/0.11.3``. `Other build tools will support this ABI as well `_. From a8294fa72faec3a8b86d4f1161c7f7a8a1d4b3b9 Mon Sep 17 00:00:00 2001 From: Charlie Lin Date: Fri, 1 May 2026 20:13:26 -0400 Subject: [PATCH 13/21] Update freethreading-stable-abi.rst --- Doc/howto/freethreading-stable-abi.rst | 30 +++++++++++++------------- 1 file changed, 15 insertions(+), 15 deletions(-) diff --git a/Doc/howto/freethreading-stable-abi.rst b/Doc/howto/freethreading-stable-abi.rst index 08033c529b841db..13b23e759874bcf 100644 --- a/Doc/howto/freethreading-stable-abi.rst +++ b/Doc/howto/freethreading-stable-abi.rst @@ -106,21 +106,21 @@ Critical Sections Replacements: -+-----------------------------------+-----------------------------------+ -| Macro functions | C API functions | -+===================================+===================================+ -| :c:func:`Py_BEGIN_CRITICAL_SECTION` | :c:func:`PyCriticalSection_Begin` | -| :c:func:`Py_END_CRITICAL_SECTION` | |:c:func:`PyCriticalSection_End` | -+-----------------------------------+-----------------------------------+ -| :c:func:`Py_BEGIN_CRITICAL_SECTION2` | :c:func:`PyCriticalSection2_Begin` | -| :c:func:`Py_END_CRITICAL_SECTION2` | |:c:func:`PyCriticalSection2_End` | -+-----------------------------------+-----------------------------------+ -| :c:func:`Py_BEGIN_CRITICAL_SECTION_MUTEX` | :c:func:`PyCriticalSection_BeginMutex` | -| :c:func:`Py_END_CRITICAL_SECTION` | |:c:func:`PyCriticalSection_End` | -+-----------------------------------+-----------------------------------+ -| :c:func:`Py_BEGIN_CRITICAL_SECTION2_MUTEX` | :c:func:`PyCriticalSection2_BeginMutex` | -| :c:func:`Py_END_CRITICAL_SECTION2` | |:c:func:`PyCriticalSection2_End` | -+-----------------------------------+-----------------------------------+ ++------------------------------------------+---------------------------------------+ +| Macro functions | C API functions | ++==========================================+=======================================+ +|:c:func:`Py_BEGIN_CRITICAL_SECTION` |:c:func:`PyCriticalSection_Begin` | +|:c:func:`Py_END_CRITICAL_SECTION` |:c:func:`PyCriticalSection_End` | ++------------------------------------------+---------------------------------------+ +|:c:func:`Py_BEGIN_CRITICAL_SECTION2` |:c:func:`PyCriticalSection2_Begin` | +|:c:func:`Py_END_CRITICAL_SECTION2` |:c:func:`PyCriticalSection2_End` | ++------------------------------------------+---------------------------------------+ +|:c:func:`Py_BEGIN_CRITICAL_SECTION_MUTEX` |:c:func:`PyCriticalSection_BeginMutex` | +|:c:func:`Py_END_CRITICAL_SECTION` |:c:func:`PyCriticalSection_End` | ++------------------------------------------+---------------------------------------+ +|:c:func:`Py_BEGIN_CRITICAL_SECTION2_MUTEX`|:c:func:`PyCriticalSection2_BeginMutex`| +|:c:func:`Py_END_CRITICAL_SECTION2` |:c:func:`PyCriticalSection2_End` | ++------------------------------------------+---------------------------------------+ Platform-specific considerations ................................ From 43538cf2d1d27d849d2c604c5e56b94e8104d249 Mon Sep 17 00:00:00 2001 From: Charlie Lin Date: Fri, 1 May 2026 20:58:52 -0400 Subject: [PATCH 14/21] Update freethreading-stable-abi.rst --- Doc/howto/freethreading-stable-abi.rst | 30 +++++++++++++------------- 1 file changed, 15 insertions(+), 15 deletions(-) diff --git a/Doc/howto/freethreading-stable-abi.rst b/Doc/howto/freethreading-stable-abi.rst index 13b23e759874bcf..1767bed4c59c24b 100644 --- a/Doc/howto/freethreading-stable-abi.rst +++ b/Doc/howto/freethreading-stable-abi.rst @@ -106,21 +106,21 @@ Critical Sections Replacements: -+------------------------------------------+---------------------------------------+ -| Macro functions | C API functions | -+==========================================+=======================================+ -|:c:func:`Py_BEGIN_CRITICAL_SECTION` |:c:func:`PyCriticalSection_Begin` | -|:c:func:`Py_END_CRITICAL_SECTION` |:c:func:`PyCriticalSection_End` | -+------------------------------------------+---------------------------------------+ -|:c:func:`Py_BEGIN_CRITICAL_SECTION2` |:c:func:`PyCriticalSection2_Begin` | -|:c:func:`Py_END_CRITICAL_SECTION2` |:c:func:`PyCriticalSection2_End` | -+------------------------------------------+---------------------------------------+ -|:c:func:`Py_BEGIN_CRITICAL_SECTION_MUTEX` |:c:func:`PyCriticalSection_BeginMutex` | -|:c:func:`Py_END_CRITICAL_SECTION` |:c:func:`PyCriticalSection_End` | -+------------------------------------------+---------------------------------------+ -|:c:func:`Py_BEGIN_CRITICAL_SECTION2_MUTEX`|:c:func:`PyCriticalSection2_BeginMutex`| -|:c:func:`Py_END_CRITICAL_SECTION2` |:c:func:`PyCriticalSection2_End` | -+------------------------------------------+---------------------------------------+ ++-------------------------------------------+---------------------------------------+ +| Macro functions | C API functions | ++===========================================+=======================================+ +|:c:macro:`Py_BEGIN_CRITICAL_SECTION` |``PyCriticalSection_Begin`` | +|:c:macro:`Py_END_CRITICAL_SECTION` |``PyCriticalSection_End`` | ++-------------------------------------------+---------------------------------------+ +|:c:macro:`Py_BEGIN_CRITICAL_SECTION2` |``PyCriticalSection2_Begin`` | +|:c:macro:`Py_END_CRITICAL_SECTION2` |``PyCriticalSection2_End`` | ++-------------------------------------------+---------------------------------------+ +|:c:macro:`Py_BEGIN_CRITICAL_SECTION_MUTEX` |``PyCriticalSection_BeginMutex`` | +|:c:macro:`Py_END_CRITICAL_SECTION` |``PyCriticalSection_End`` | ++-------------------------------------------+---------------------------------------+ +|:c:macro:`Py_BEGIN_CRITICAL_SECTION2_MUTEX`|``PyCriticalSection2_BeginMutex`` | +|:c:macro:`Py_END_CRITICAL_SECTION2` |``PyCriticalSection2_End`` | ++-------------------------------------------+---------------------------------------+ Platform-specific considerations ................................ From ed0e4fee01763360b0054734d394b55440d745e2 Mon Sep 17 00:00:00 2001 From: Charlie Lin Date: Sun, 17 May 2026 06:49:48 -0400 Subject: [PATCH 15/21] Update module slot definitions to use PySlot from PEP 820 --- Doc/howto/freethreading-stable-abi.rst | 15 ++++++++------- 1 file changed, 8 insertions(+), 7 deletions(-) diff --git a/Doc/howto/freethreading-stable-abi.rst b/Doc/howto/freethreading-stable-abi.rst index 1767bed4c59c24b..2368f85586fa3b0 100644 --- a/Doc/howto/freethreading-stable-abi.rst +++ b/Doc/howto/freethreading-stable-abi.rst @@ -61,18 +61,19 @@ Extensions that use :ref:`multi-phase initialization = 0x030F0000 - {Py_mod_abi, &abi_info}, + PySlot_STATIC_DATA(Py_mod_abi, &abi_info), #endif - {0, NULL} + PySlot_END }; Single-Phase Initialization From a7fcbcc9e7c988eb5dccc9235a435389707e4a82 Mon Sep 17 00:00:00 2001 From: Charlie Lin Date: Sun, 17 May 2026 11:22:15 -0400 Subject: [PATCH 16/21] Update freethreading-stable-abi.rst --- Doc/howto/freethreading-stable-abi.rst | 21 +++++++-------------- 1 file changed, 7 insertions(+), 14 deletions(-) diff --git a/Doc/howto/freethreading-stable-abi.rst b/Doc/howto/freethreading-stable-abi.rst index 2368f85586fa3b0..8f4c0bbc1ce6816 100644 --- a/Doc/howto/freethreading-stable-abi.rst +++ b/Doc/howto/freethreading-stable-abi.rst @@ -14,29 +14,22 @@ to adapt C API extensions to support free threading. Identifying the Free-Threaded Limited API Build in C ==================================================== -The CPython C API exposes the :c:macro:`!Py_LIMITED_API` macro: in the free-threaded stable ABI -build it's defined to ``1``, and in the regular build it's not defined. +Define :c:macro:`!Py_TARGET_ABI3T` to the lowest Python version your extension supports, +either in the form of `Py_PACK_VERSION(3.15)` or its direct hex value (such as ``0x30f0000`` for 3.15). You can use it to enable code that only runs under the free-threaded build:: #ifdef Py_TARGET_ABI3T /* code that only runs in the free-threaded stable ABI build */ #endif -If you wish to build youe extension with both ``abi3`` (Stable ABI with GIL) and ``abi3t`` (no-GIL stable ABI) tags, -do one of the following: - -- define both :c:macro:`!Py_LIMITED_API` and :c:macro:`!Py_TARGET_ABI3T`, or -- define only :c:macro:`!Py_LIMITED_API` and: - - - on Windows, define :c:macro:`!Py_GIL_DISABLED`; - - on other systems, use the headers of free-threaded build of Python. - ``PyObject`` and ``PyVarObject`` opaqueness =========================================== -Accessing any member of ``PyObject`` directly is now prohibited, like the non-GIL -stable ABI. For instance, prefer ``Py_TYPE()`` and ``Py_SET_TYPE()`` over ``ob_type``, +Accessing any member of ``PyObject`` directly is now prohibited, unlike the GIL +stable ABI, where accessing such members are merely discouraged. +For instance, prefer ``Py_TYPE()`` and ``Py_SET_TYPE()`` over ``ob_type``, ``Py_REFCNT``, ``Py_IncRef()`` and ``Py_DecRef()`` over ``ob_refcnt``, etc. +Also, embedding :c:macro:`PyObject_HEAD` within a struct is impossible. Similarly, members of ``PyVarObject`` are not visible. If you need any object of such type to be passed as a ``PyObject`` parameter to any API function, cast it directly as ``PyObject``. @@ -105,7 +98,7 @@ Critical Sections .. _critical-sections: -Replacements: +Equivalent functions: +-------------------------------------------+---------------------------------------+ | Macro functions | C API functions | From ccdedb08d40c1e300c27341a7440c97d800ecb63 Mon Sep 17 00:00:00 2001 From: Charlie Lin Date: Sun, 17 May 2026 11:28:02 -0400 Subject: [PATCH 17/21] Fix typo in freethreading-stable-abi.rst --- Doc/howto/freethreading-stable-abi.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Doc/howto/freethreading-stable-abi.rst b/Doc/howto/freethreading-stable-abi.rst index 8f4c0bbc1ce6816..2038309b394b127 100644 --- a/Doc/howto/freethreading-stable-abi.rst +++ b/Doc/howto/freethreading-stable-abi.rst @@ -15,7 +15,7 @@ Identifying the Free-Threaded Limited API Build in C ==================================================== Define :c:macro:`!Py_TARGET_ABI3T` to the lowest Python version your extension supports, -either in the form of `Py_PACK_VERSION(3.15)` or its direct hex value (such as ``0x30f0000`` for 3.15). +either in the form of :c:macro:`Py_PACK_VERSION(3.15)` or its direct hex value (such as ``0x30f0000`` for 3.15). You can use it to enable code that only runs under the free-threaded build:: #ifdef Py_TARGET_ABI3T From 3b401db2e0b01fa5671b188b388ae2d531594df7 Mon Sep 17 00:00:00 2001 From: Charlie Lin Date: Sun, 17 May 2026 11:32:05 -0400 Subject: [PATCH 18/21] Update freethreading-stable-abi.rst --- Doc/howto/freethreading-stable-abi.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Doc/howto/freethreading-stable-abi.rst b/Doc/howto/freethreading-stable-abi.rst index 2038309b394b127..928d6dd42fdce00 100644 --- a/Doc/howto/freethreading-stable-abi.rst +++ b/Doc/howto/freethreading-stable-abi.rst @@ -15,7 +15,7 @@ Identifying the Free-Threaded Limited API Build in C ==================================================== Define :c:macro:`!Py_TARGET_ABI3T` to the lowest Python version your extension supports, -either in the form of :c:macro:`Py_PACK_VERSION(3.15)` or its direct hex value (such as ``0x30f0000`` for 3.15). +either in the form of ``Py_PACK_VERSION(3.15)`` or its direct hex value (such as ``0x30f0000`` for 3.15). You can use it to enable code that only runs under the free-threaded build:: #ifdef Py_TARGET_ABI3T From cb9545a3f53c3e9c9f8c416ac20c92be7361b717 Mon Sep 17 00:00:00 2001 From: Petr Viktorin Date: Wed, 27 May 2026 15:53:00 +0200 Subject: [PATCH 19/21] Rewrite as a guide --- Doc/howto/abi3t-migration.rst | 613 +++++++++++++++++++++++++ Doc/howto/freethreading-stable-abi.rst | 148 ------ Doc/howto/index.rst | 4 +- Doc/tools/extensions/c_annotations.py | 2 +- 4 files changed, 616 insertions(+), 151 deletions(-) create mode 100644 Doc/howto/abi3t-migration.rst delete mode 100644 Doc/howto/freethreading-stable-abi.rst diff --git a/Doc/howto/abi3t-migration.rst b/Doc/howto/abi3t-migration.rst new file mode 100644 index 000000000000000..a5da062f3e36b3f --- /dev/null +++ b/Doc/howto/abi3t-migration.rst @@ -0,0 +1,613 @@ +.. highlight:: c + +.. _abi3t-migration-howto: + +****************************************************** +Migrating to Stable ABI for Free Threading (``abi3t``) +****************************************************** + +Starting with the 3.15 release, CPython supports a variant of the Stable ABI +that supports :term:`free-threaded ` Python: +Stable ABI for Free-Threaded Builds, or ``abi3t`` for short. +This document describes how to adapt C API extensions to support free threading. + +Why do this +=========== + +The typical reason to use Stable ABI is to reduce the number of artifacts that +you need to build and distribute for each version of your library. + +Without the Stable ABI, you must build a separate shared library, and typically +a *wheel* distribution, for each feature version of CPython you wish +to support. +For example, each tag in the following table represents a separate +library/wheel: + ++-----------------+-------------------+------------------+ +| CPython version | Non-free-threaded | Free-threaded | ++=================+===================+==================+ +| 3.12 | ``cpython-312`` | --- | ++-----------------+-------------------+------------------+ +| 3.13 | ``cpython-313`` | ``cpython-313t`` | ++-----------------+-------------------+------------------+ +| 3.14 | ``cpython-314`` | ``cpython-314t`` | ++-----------------+-------------------+------------------+ +| 3.15 | ``cpython-315`` | ``cpython-315t`` | ++-----------------+-------------------+------------------+ +| 3.16 | ``cpython-316`` | ``cpython-316t`` | ++-----------------+-------------------+------------------+ +| Future versions | (etc.) | (etc.) | ++-----------------+-------------------+------------------+ + +That's a lot of builds, especially when multiplied by the number +of supported platforms. + +With the Stable ABI (``abi3``, introduced in CPython 3.2), a single extension +(per platform) can cover all *non-free-threaded* builds of CPython: + ++-----------------+-------------------+------------------+ +| CPython version | Non-free-threaded | Free-threaded | ++=================+===================+==================+ +| 3.12 | ``abi3`` | --- | ++-----------------+ +------------------+ +| 3.13 | | ``cpython-313t`` | ++-----------------+ +------------------+ +| 3.14 | | ``cpython-314t`` | ++-----------------+ +------------------+ +| 3.15 | | ``cpython-315t`` | ++-----------------+ +------------------+ +| 3.16 | | ``cpython-316t`` | ++-----------------+ +------------------+ +| Future versions | | (etc.) | ++-----------------+-------------------+------------------+ + +The Stable ABI for free-threaded builds (``abi3t``), introduced in +CPython 3.15, does the same for free-threaded builds. +And it's compatible with non-free-threaded ones as well: + ++-----------------+-------------------+------------------+ +| CPython version | Non-free-threaded | Free-threaded | ++=================+===================+==================+ +| 3.12 | ``abi3`` * | --- | ++-----------------+ +------------------+ +| 3.13 | | ``cpython-313t`` | ++-----------------+ +------------------+ +| 3.14 | | ``cpython-314t`` | ++-----------------+-------------------+------------------+ +| 3.15 | ``abi3t`` | ++-----------------+ + +| 3.16 | | ++-----------------+ + +| Future versions | | ++-----------------+-------------------+------------------+ + +\* (As above, the ``abi3`` extension is compatible with all non-free-threaded +builds; even the 3.15+ ones that this table "attributes" to ``abi3t``.) + +Why *not* do this +----------------- + +There are two main downsides to Stable ABI. + +First, you extension may become slower, since Stable ABI prioritizes +compatibility over performance. +The difference is usually not noticeable, and often can be mitigated by +using the same source to build both a Stable ABI build and a few +version-specific ones for "tier 1" CPython versions. + +Second, not all of the C API is available. +Extensions need to be ported to build for Stable ABI, which may be difficult +or, in rare cases, impossible. + +Specifically, ``abi3t`` requires on API added in CPython 3.15. +If you want to build your extension for older versions of CPython from the +same source, you have two main options: + +- Use preprocessor conditionals. + + When following this guide, use ``#ifdef Py_TARGET_ABI3T`` blocks whenever + you are told to do a change that breaks the build on CPython versions you + care about. Keep the pre-existing code in ``#else`` blocks. + + For hand-written C extensions, this approach is reasonable down to + CPython 3.12, due to additions introduced in :pep:`697`. + Keeping compatibility with 3.11 and below may be worth it for code + generators (for example, Cython). + +- Do not port to ``abi3t``, and continue building separate extensions for + each version of CPython, until you can drop support for the older versions. + + This is a valid approach. Not all extensions need to switch to ``abi3t`` + right now. + + +Prerequisites +============= + +This guide assumes that you have an extension written directly in C (or C++), +which you want to port to ``abi3t``. + +If your extenstion uses a code generator (like Cython) or language binding +(like PyO3), it's best to wait until that tool has support for ``abi3t``. +If you maintain such a tool, you might be able to adapt the instructions +here for your tool. + +Non-free-threaded Stable ABI +---------------------------- + +Your extension should support the Stable ABI (``abi3t``). +If not, either port it first, or follow this guide but be prepared to fix +issues it does not mention. + +Free-threading support +---------------------- + +While it's technically not a hard prerequisite, you will most likely want to +prepare your extension for free threading before you port it to ``abi3t``. +See :ref:`freethreading-extensions-howto` for instructions. + +.. seealso:: + + `Porting Extension Modules to Support Free-Threading + `_: + A community-maintained porting guide for extension authors. + +Isolating Extension Modules +--------------------------- + +Your module should use :ref:`multi-phase initialization `, +and it should either be isolated or limit itself to be loaded at most once +per process. +If it is not your case, follow :ref:`isolating-extensions-howto` first. +(See the :ref:`opt-out section ` for a shortcut.) + +Avoiding variable-sized types +----------------------------- + +If your extension defines variable-sized types (using :c:macro:`Py_tp_itemsize` +or :c:member:`PyTypeObject.tp_itemsize`), it cannot be ported to +``abi3t`` 3.15. + + +Setting up the build +==================== + +If you use a build tool (such as setuptools, meson-python, scikit-build-core), +search its documentation for a way to select ``abi3t``. +At the time of writing, not all of them have this; but if your tool does, +use it. +You may want to verify that it set the right flag by temporarily adding the +following just after ``#include ``:: + + #if Py_TARGET_ABI3T+0 <= 0x30f0000 + #error "abi3t define is not set!" + #endif + +This should result in a different error than "``abt3t`` define is not set". + +.. note:: + + If your build tool doesn't support ``abi3t`` yet, set the following macro + before including ``Python.h``:: + + #define Py_TARGET_ABI3T 0x30f0000 + + or specify it as a compiler flag, for example:: + + -DPy_TARGET_ABI3T=0x30f0000 + + Once your extension builds with this setting, it will be compatible with + CPython 3.15 and above. + + If you set this macro manually, you will later need to name and tag the + resulting extension manually as well. + This is covered in :ref:`abi3t-migration-tagging` below. + +This guide will ask you to do a series of changes. +After each one, verify that your extension still builds in the original +(non-``abi3t``) configuration, and ideally run tests on all Python +versions you support. +This will ensure that nothing breaks as you are porting. + + +Module export hook +================== + +Unless you've done this step already, your extension module defines a +:ref:`module initialization function ` +named :samp:`PyInit_{}`. +You will need to port it to :ref:`module export hook `, +:samp:`PyModExport_{}`, a feature added in CPython 3.15 in +:pep:`793`. + +Your existing init function should look like this (with your own names +for ```` and ````): + +.. code-block:: + :class: bad + + PyMODINIT_FUNC + PyInit_(void) + { + return PyModuleDef_Init(&); + } + +If there is some code before the ``return``, move it to +a :c:macro:`Py_mod_create` or :c:macro:`Py_mod_exec` slot function. +See :ref:`PyInit documentation ` for related information. + +The function references a ``PyModuleDef`` object (```` in the code +above). +Its definition should be similar to the following, with different values +and perhaps some fields unnnamed or left out: + +.. code-block:: + :class: bad + + static PyModuleDef = { + PyModuleDef_HEAD_INIT, + .m_name = "my_module", + .m_doc = "my docstring", + .m_size = sizeof(my_state_struct), + .m_methods = my_methods, + .m_slots = my_slots, + .m_traverse = my_traverse, + .m_clear = my_clear, + .m_free = my_free, + }; + +Remove this definition and the ``PyInit`` function (or put them in +an ``#ifndef Py_TARGET_ABI3T`` block, to retain backwards compatibility), +and replace them with the following: + +.. code-block:: + :class: good + + PyABIInfo_VAR(abi_info); + + static PySlot my_slot_array[] = { + PySlot_STATIC_DATA(Py_mod_abi, &abi_info), + PySlot_STATIC_DATA(Py_mod_name, "my_module"), + PySlot_STATIC_DATA(Py_mod_doc, "my docstring"), + PySlot_SIZE(Py_mod_state_size, sizeof(my_state_struct)), + PySlot_STATIC_DATA(Py_mod_methods, my_methods), + PySlot_STATIC_DATA(Py_mod_slots, my_slots), + PySlot_FUNC(Py_mod_state_traverse, my_traverse), + PySlot_FUNC(Py_mod_state_clear, my_clear), + PySlot_FUNC(Py_mod_state_free, my_free), + PySlot_END + }; + + PyMODEXPORT_FUNC + PyModExport_(void) + { + return my_slot_array; + } + +Leave out any fields that were missing, and substitute your own values. + +See :c:type:`PySlot` and :c:ref:`export hook ` +documentation for details on this API. + +Associated ``PyModuleDef`` +-------------------------- + +Since the new API does not use a :c:type:`!PyModuleDef` structure, a definition +will not be associated with the resulting module. +This changes the behavior of the following functions: + +- :c:func:`PyModule_GetDef` +- :c:func:`PyType_GetModuleByDef` + +Check your code for these. +If you do not use them, you skip the rest of this section. + +These functions are typically used for two purposes: + +1. To get the definition the module was created with. + This is no longer possible using the new API. + Modules no longer keep a reference to the definition, so you will need to + figure out a different way to pass the relevant data around. + +.. _abi3t-migration-module-token: + +2. To check if a given module object is “yours”. + This use case is now served by :ref:`module tokens ` -- + opaque pointers that identify a module. + To use a token, declare (or reuse) a unique static variable, for example: + + .. code-block:: + :class: good + + static char my_token; + + and add a pointer to it in a new entry to your module's ``PySlot`` array: + + .. code-block:: + :class: good + :emphasize-lines: 3 + + static PySlot my_slot_array[] = { + ... + PySlot_STATIC_DATA(Py_mod_token, &my_token), + PySlot_END + } + + Then, switch from :c:func:`PyModule_GetDef` calls such as: + + .. code-block:: + :class: bad + + PyModuleDef *def = PyModule_GetDef(module); + + to :c:func:`PyModule_GetToken` (which uses an output argument and may fail + with an exception): + + .. code-block:: + :class: good + + void *token; + if (PyModule_GetToken(module, &token) < 0) { + /* handle error */ + } + + and from :c:func:`PyType_GetModuleByDef` calls such as: + + .. code-block:: + :class: bad + + PyObject *module = PyType_GetModuleByDef(type, my_def); + /* handle error; use module */ + + to :c:func:`PyType_GetModuleByToken` (which returns a strong reference): + + .. code-block:: + :class: good + + PyObject *module = PyType_GetModuleByToken(type, my_token); + /* handle error; use module */ + Py_XDECREF(module); + +``PyObject`` opaqueness +======================= + +The :c:type:`PyObject` and :c:type:`PyVarObject` structures are opaque +in ``abi3t``. + +Accessing their members is prohibited. +If you do this, switch to getter/setter functions mentioned in +their documentation: + +- :c:member:`PyObject.ob_type` +- :c:member:`PyObject.ob_refcnt` +- :c:member:`PyVarObject.ob_size` + +Also, the *size* of the :c:type:`PyObject` structures is +unknown to the compiler. +It can -- and *does* -- change between different CPython builds. + +.. note:: + + While the size is available at runtime (for example as + ``sys.getsizeof(object())`` in Python code), you should resist the + temptation to calculate pointer offsets from it. + The object memory layout is subject to change in future + ``abi3t`` implementations. + + +Custom type definitions +----------------------- + +Since :c:type:`!PyObject` is opaque, the traditional way of defining +custom types no longer works: + +.. code-block:: + :class: bad + + typedef struct { + PyObject_HEAD // expands to `PyObject ob_base;` which has unknown size + + int my_data; + } CustomObject; + + static PyTypeObject CustomType = { + ... + .tp_basicsize = sizeof(CustomObject), + ... + }; + +Most likely, all your class definitions, *and* all code that accesses +your classes' data, will need to be rewritten. +This will probably be the biggest change you need to support ``abi3t``. + +For each such type: + +Instead of defining a ``struct`` for the entire instance, define one with only +the “additional” fields -- ones specific to your class, not its superclasses: + +.. code-block:: + :class: good + + typedef struct { + int my_data; + } CustomObjectData; + +Change the name. +Almost all code that uses the struct will need to change +(notably, pointers to the new structure cannot be cast to/from ``PyObject*``), +and changing the name will highlight the usages as compiler errors. +(If you use ``typeof``, C++ ``auto``, or similar ways to avoid +typing the type name, this won't work. Be extra careful, and consider running +tools to detect undefined behavior.) + +Then, to create the class, use *negative* :c:macro:`tp_basicsize` to indicate +“extra” storage space rather than *total* instance size: + +.. code-block:: + :class: good + + static PyTypeObject CustomType = { + ... + .tp_basicsize = -sizeof(CustomObjectData), /* note the minus sign */ + ... + }; + +If you use :c:macro:`Py_tp_members`, set the :c:macro:`Py_RELATIVE_OFFSET` +flag on each member and specify the :c:member:`~PyMemberDef.offset` +relative to your new struct. + + +Custom type data access +----------------------- + +Then comes the hard part: in all code that needs to access this struct, +you will need an additional :c:func:`PyObject_GetTypeData` call to +retrieve a ``CustomObjectData *`` pointer from ``PyObject *``: + +.. code-block:: + :class: good + + PyObject *obj = ...; + CustomObjectData *data = PyObject_GetTypeData(obj, cls); + +Note that this call requires the *type object* for your class (``cls``). + +If your class is not subclassable (that is, it does not use the +:c:macro:`Py_TPFLAGS_BASETYPE` flag), ``cls`` will be ``Py_TYPE(obj)``. +Otherwise, **DO NOT USE** ``Py_TYPE`` with :c:func:`!PyObject_GetTypeData`: +it might return memory reserved to an unrelated subclass! +For example, if a user makes a subclass like this: + +.. code-block:: python + + class Sub(YourCustomClass): + __slots__ = ('a', 'b') + +then ``Py_TYPE(obj)`` is ``YourCustomClass``, and the underlying memory may +look like this: + +.. code-block:: text + + ╭─ PyObject *obj + │ ╭─ the pointer you want + │ │ ╭─ PyObject_GetTypeData(obj, Py_TYPE(obj)) + ▼ ▼ ▼ + ┌──────────┬───┬────────────────┬───┬─────────────┬───┬─────────────┐ + │ PyObject │...│ CustomTypeData │...│ PyObject *a │...│ PyObject *b │ + └──────────┴───┴────────────────┴───┴─────────────┴───┴─────────────┘ + +(Ellipses indicate possible padding. +Note that this memory layout is not guaranteed: future versions of Python may +add different padding or even switch the order of the structures.) + +There are two main ways to get the right class: + +- In instance methods, your implementation may use the :c:type:`PyCMethod` + signature (and the :c:macro:`METH_METHOD` bit in + :c:member:`PyMethodDef.ml_flags`), + and get the class as the ``defining_class`` argument. +- Otherwise, give your class a unique static token using the + :c:macro:`Py_tp_token` slot, and use: + + .. code-block:: + :class: good + + PyTypeObject cls; + if (PyType_GetBaseByToken(Py_TYPE(obj), my_tp_token, &cls) < 0) { + /* handle error */ + } + CustomObjectData *data = PyObject_GetTypeData(obj, cls); + + Type tokens work similarly to module tokens covered :ref:`earlier in this + guide `. + + + +Avoid build-time conditionals +============================= + +Check your code for API that identifies the version of Python used to +*build* your extension. +This no longer corresponds to the Python your extension runs on, so code +that uses this information often needs changing. +The macros to check for are: + +- :c:macro:`PY_VERSION_HEX`, :c:macro:`PY_MAJOR_VERSION`, + :c:macro:`PY_MINOR_VERSION`: + + - to get the run-time version, use :c:data:`Py_Version`; + - to determine what C API is available, use :c:macro:`Py_TARGET_ABI3T`. + This macro is set to the minimum supported version. + +- :c:macro:`Py_GIL_DISABLED`: under ``abi3t``, this macro is always defined. + Code that works with free-threaded Python *should* also work with + the GIL enabled (since the GIL can be enabled at run time), + and usually *does* (unless it, for some reason, requires more than one + :term:`attached thread state ` at one time). + + +Further code changes +==================== + +If you are still left with compiler errors or warnings, find a way to fix them. +Alas, this guide is limited, and cannot cover all possible code +changes extensions may need. + +If you find a problem that other extension authors might run into, +consider submitting an issue (or pull request) for this guide. + +It is possible your issue cannot be fixed for the current version of ``abi3t``. +In that case, reporting it may help it get prioritized for the next version +of CPython. + + +.. _abi3t-migration-tagging: + +Tagging and distribution +======================== + +If you are using a build tool with ``abi3t`` support, your extension is ready, +but you might want to check that it was built correctly. + +Extensions built with ``abi3t`` should have the following extension: + +- On Windows: ``.pyd`` (like any other extension); +- Linux, macOS, and other systems that use the ``.so`` suffix: ``.abi3t.so`` + (**not** ``.cpython-315t.so`` or ``.abi3.so``). + Note that both free-threaded and non-free-threaded builds will + load ``.abi3t.so`` extensions; +- Other systems: consult your distributor, and perhaps update this guide. + +If you distribute the extension as a *wheel*, use the following tags: + +* Python tag: :samp:`cp3{YY}`, where *YY* is the minimum Python version + the extension is built for. + (For example, ``cp315`` if you set ``Py_TARGET_ABI3T`` to ``0x30f0000``. + See :ref:`abi3-compiling` for more values.) +* ABI tag: ``abi3.abi3t``. This is a *compressed tag set* that indicates + support for both non-free-threaded and free-threaded builds. + +For example, the wheel filename may look like this: + +.. code-block:: text + + myproject-1.0-cp315-abi3.abi3t-macosx_11_0_arm64.whl + +.. seealso:: `Platform Compatibility Tags `__ in the PyPA package distribution metadata. + +If the filename or tags are incorrect, fix them. + + +Testing +======= + +Note that when you build an extension compatible with multiple versions of +CPython, you should always *test* it witch each version it supports (for +example, 3.15, 3.16, and so on). +Stable ABI only guarantees *ABI* compatibility; there may also be behavior +changes -- both intentional ones (covered by :pep:`387`) and bugs. + +Be sure to run tests on both free-threaded and non-free-threaded builds +of CPython. + +If they pass, congratulations! You have an ``abi3t`` extension. diff --git a/Doc/howto/freethreading-stable-abi.rst b/Doc/howto/freethreading-stable-abi.rst deleted file mode 100644 index 928d6dd42fdce00..000000000000000 --- a/Doc/howto/freethreading-stable-abi.rst +++ /dev/null @@ -1,148 +0,0 @@ -.. highlight:: c - -.. _freethreading-stable-abi-howto: - -****************************************** -How to Use Free Threading Stable ABI -****************************************** - -Starting with the 3.15 release, CPython has support for compiling extensions targeting -a unique kind of Stable ABI with interpreters having the :term:`global interpreter lock` (GIL) disabled. -For 3.14 and 3.13, continue compiling with the version-specific ABI. This document describes how -to adapt C API extensions to support free threading. - -Identifying the Free-Threaded Limited API Build in C -==================================================== - -Define :c:macro:`!Py_TARGET_ABI3T` to the lowest Python version your extension supports, -either in the form of ``Py_PACK_VERSION(3.15)`` or its direct hex value (such as ``0x30f0000`` for 3.15). -You can use it to enable code that only runs under the free-threaded build:: - - #ifdef Py_TARGET_ABI3T - /* code that only runs in the free-threaded stable ABI build */ - #endif - -``PyObject`` and ``PyVarObject`` opaqueness -=========================================== - -Accessing any member of ``PyObject`` directly is now prohibited, unlike the GIL -stable ABI, where accessing such members are merely discouraged. -For instance, prefer ``Py_TYPE()`` and ``Py_SET_TYPE()`` over ``ob_type``, -``Py_REFCNT``, ``Py_IncRef()`` and ``Py_DecRef()`` over ``ob_refcnt``, etc. -Also, embedding :c:macro:`PyObject_HEAD` within a struct is impossible. - -Similarly, members of ``PyVarObject`` are not visible. If you need any object of such type -to be passed as a ``PyObject`` parameter to any API function, cast it directly as ``PyObject``. - -Module Initialization -===================== - -Extension modules need to explicitly indicate that they support running with -the GIL disabled; otherwise importing the extension will raise a warning and -enable the GIL at runtime. - -Multi-phase and single-phase initialization is supported to indicate that an extension module -targeting the stable ABI supports running with the GIL disabled, though the former is preferred. - -Multi-Phase Initialization -.......................... - -Extensions that use :ref:`multi-phase initialization ` -(functions like :c:func:`PyModuleDef_Init`, -:c:func:`PyModExport_* ` export hook, -:c:func:`PyModule_FromSlotsAndSpec`) should add a -:c:data:`Py_mod_gil` slot in the module definition. -If your extension supports older versions of CPython, -you should guard the slot with a :c:data:`Py_GIL_DISABLED` check. -Additionally, prefer :c:type:`PySlot` over :c:type:`PyModuleDef_Slot`. - -:: - - static PySlot module_slots[] = { - ... - #ifdef Py_GIL_DISABLED - PySlot_STATIC_DATA(Py_mod_gil, Py_MOD_GIL_NOT_USED), - #endif - PySlot_END - }; - -Furthermore, using ``PyABIInfo_VAR`` and ``Py_mod_abi`` is recommended so that an -extension module loaded for an incompatible interpreter will trigger an exception, rather than -fail with a crash. - -.. code-block:: c - - #ifdef PY_VERSION_HEX >= 0x030F0000 - PyABIInfo_VAR(abi_info); - #endif Py_GIL_DISABLED - - static PySlot mymodule_slots[] = { - ... - #ifdef PY_VERSION_HEX >= 0x030F0000 - PySlot_STATIC_DATA(Py_mod_abi, &abi_info), - #endif - PySlot_END - }; - -Single-Phase Initialization -........................... - -Although members of ``PyModuleDef`` is still available for no-GIL Stable ABI and can be used -for :ref:`single-phase initialization ` -(that is, :c:func:`PyModule_Create`), they are not exposed when targeting the regular Stable ABI. -Prefer multi-phased initializtion when possible. - - -Critical Sections -================= - -.. _critical-sections: - -Equivalent functions: - -+-------------------------------------------+---------------------------------------+ -| Macro functions | C API functions | -+===========================================+=======================================+ -|:c:macro:`Py_BEGIN_CRITICAL_SECTION` |``PyCriticalSection_Begin`` | -|:c:macro:`Py_END_CRITICAL_SECTION` |``PyCriticalSection_End`` | -+-------------------------------------------+---------------------------------------+ -|:c:macro:`Py_BEGIN_CRITICAL_SECTION2` |``PyCriticalSection2_Begin`` | -|:c:macro:`Py_END_CRITICAL_SECTION2` |``PyCriticalSection2_End`` | -+-------------------------------------------+---------------------------------------+ -|:c:macro:`Py_BEGIN_CRITICAL_SECTION_MUTEX` |``PyCriticalSection_BeginMutex`` | -|:c:macro:`Py_END_CRITICAL_SECTION` |``PyCriticalSection_End`` | -+-------------------------------------------+---------------------------------------+ -|:c:macro:`Py_BEGIN_CRITICAL_SECTION2_MUTEX`|``PyCriticalSection2_BeginMutex`` | -|:c:macro:`Py_END_CRITICAL_SECTION2` |``PyCriticalSection2_End`` | -+-------------------------------------------+---------------------------------------+ - -Platform-specific considerations -................................ - -On some platforms, Python will look for and load shared library files named -with the ``abi3`` or ``abi3t`` tag (for example, ``mymodule.abi3t.so``). -:term:`Free-threaded ` interpreters prefer ``abi3t``, -but can fall back to ``abi3``. -Thus, extensions compatible with both ABIs should use the ``abi3t`` tag. - -Python does not necessarily check that extensions it loads -have compatible ABI. -Extension authors are encouraged to add a check using the :c:macro:`Py_mod_abi` -slot or the :c:func:`PyABIInfo_Check` function. - -Limited C API Build Tools -......................... - -If you use -`setuptools `_ to build -your extension, a future version of ``setuptools`` will allow ``py_limited_api=True`` -to be set to allow targeting limited API when building with the free-threaded build. -``uv`` supports targeting PEP 803 as of 0.11.3: ``https://github.com/astral-sh/uv/releases/tag/0.11.3``. - -`Other build tools will support this ABI as well `_. - -.. seealso:: - - `Porting Extension Modules to Support Free-Threading - `_: - A community-maintained porting guide for extension authors. diff --git a/Doc/howto/index.rst b/Doc/howto/index.rst index 906dba5140b0e53..57e2d6e0752447f 100644 --- a/Doc/howto/index.rst +++ b/Doc/howto/index.rst @@ -37,7 +37,7 @@ Python Library Reference. mro.rst free-threading-python.rst free-threading-extensions.rst - freethreading-stable-abi.rst + abi3t-migration.rst remote_debugging.rst General: @@ -61,8 +61,8 @@ Advanced development: * :ref:`curses-howto` * :ref:`freethreading-python-howto` * :ref:`freethreading-extensions-howto` -* :ref:`freethreading-stable-abi-howto` * :ref:`isolating-extensions-howto` +* :ref:`abi3t-migration-howto` * :ref:`python_2.3_mro` * :ref:`socket-howto` * :ref:`timerfd-howto` diff --git a/Doc/tools/extensions/c_annotations.py b/Doc/tools/extensions/c_annotations.py index 1409c77aed9c6bc..0e762042979c2b7 100644 --- a/Doc/tools/extensions/c_annotations.py +++ b/Doc/tools/extensions/c_annotations.py @@ -394,7 +394,7 @@ def run(self) -> list[nodes.Node]: current_minor = int(self.config.version.removeprefix('3.')) for minor in range(current_minor - 5, current_minor + 1): value = (3 << 24) | (minor << 16) - content.append(f' {value:#x} /* Py_PACK_VERSION(3.{minor}) */') + content.append(f' {value:#x} /* Py_PACK_VERSION(3,{minor}) */') node = nodes.paragraph() self.state.nested_parse(StringList(content), 0, node) return [node] From 22a9bfe1371bf19ba5a9b67ceb37d5aea83f29d0 Mon Sep 17 00:00:00 2001 From: Petr Viktorin Date: Fri, 29 May 2026 18:33:41 +0200 Subject: [PATCH 20/21] Clarification --- Doc/howto/abi3t-migration.rst | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/Doc/howto/abi3t-migration.rst b/Doc/howto/abi3t-migration.rst index a5da062f3e36b3f..c4fd36d29c71fe8 100644 --- a/Doc/howto/abi3t-migration.rst +++ b/Doc/howto/abi3t-migration.rst @@ -284,7 +284,8 @@ and replace them with the following: return my_slot_array; } -Leave out any fields that were missing, and substitute your own values. +Leave out any fields that were missing (excexpt the new :c:macro:`Py_mod_abi`), +and substitute your own values. See :c:type:`PySlot` and :c:ref:`export hook ` documentation for details on this API. From 1448abdc76181b127964f2d9fcc251a3848dc737 Mon Sep 17 00:00:00 2001 From: Petr Viktorin Date: Fri, 29 May 2026 18:35:17 +0200 Subject: [PATCH 21/21] No static types --- Doc/howto/abi3t-migration.rst | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/Doc/howto/abi3t-migration.rst b/Doc/howto/abi3t-migration.rst index c4fd36d29c71fe8..6905d96782f927d 100644 --- a/Doc/howto/abi3t-migration.rst +++ b/Doc/howto/abi3t-migration.rst @@ -411,9 +411,9 @@ custom types no longer works: int my_data; } CustomObject; - static PyTypeObject CustomType = { + static PyType_Spec CustomType_spec = { ... - .tp_basicsize = sizeof(CustomObject), + .basicsize = sizeof(CustomObject), ... }; @@ -441,15 +441,15 @@ and changing the name will highlight the usages as compiler errors. typing the type name, this won't work. Be extra careful, and consider running tools to detect undefined behavior.) -Then, to create the class, use *negative* :c:macro:`tp_basicsize` to indicate +Then, to create the class, use *negative* ``basicsize`` to indicate “extra” storage space rather than *total* instance size: .. code-block:: :class: good - static PyTypeObject CustomType = { + static PyType_Spec CustomType_spec = { ... - .tp_basicsize = -sizeof(CustomObjectData), /* note the minus sign */ + .basicsize = -sizeof(CustomObjectData), /* note the minus sign */ ... };