COMP: Move ITK Python wrapping to SWIG 4.4.1

[hjmjohnson]

Move ITK Python wrapping to SWIG 4.4.1. Fixes the import regression in #6479; supersedes the temporary #6480 (4.3.x baseline) and #6481 (WIP CI).

Three focused commits, no WIP — standard GHA + Azure CI run on this PR. The Azure/ARM Python pipelines exercise the vendored 4.4.1 binaries; local pixi run build-python uses conda-forge SWIG/CastXML.

What changed

• Multi-phase init fix. ITK packs several SWIG modules per shared library and injects C that calls PyInit__<X>Python() and stores the return in sys.modules. SWIG 4.4 (PEP 489 multi-phase init) makes PyInit_* return a raw PyModuleDef, so the proxy failed at import with 'moduledef' object has no attribute 'delete_SwigPyIterator'. The injection now detects a returned def at runtime and instantiates it via PyModule_FromDefAndSpec + PyModule_ExecDef; the single-phase (≤4.3) path is unchanged; all APIs are Limited-API-safe (≥3.5).
• abi3 floor. Pinned to a single cached source of truth _ITK_MINIMUM_SUPPORTED_LIMITED_API_VERSION (3.11), shared by the Limited-API auto-detect and USE_SABI. 3.11+ build abi3 and load on 3.11; 3.10 keeps the non-Limited-API path.
• Vendored SWIG 4.4.1 via FetchContent + version assertion. The prebuilt-binary download moves from ExternalProject_Add to FetchContent, so SWIG_EXECUTABLE exists at configure time and itk_assert_swig_version() confirms it reports ≥ swig_version_min (now 4.3.0; too-old system swig is a FATAL_ERROR). Binaries are repackaged from the official, broadly-portable SWIG PyPI wheels and hosted on data.kitware.com (resolved by SHA512). Source-build fallback stays on ExternalProject.
• Local dev. configure-python / configure-debug-python pass ITK_USE_SYSTEM_SWIG/ITK_USE_SYSTEM_CASTXML; swig (>=4.4.1) + castxml added to the python pixi feature.

How the vendored binaries were produced (reproducible)

The swig<os>-<arch>-4.4.1.zip archives are repackaged from the official swig PyPI package (https://pypi.org/project/swig/, maintained by the SWIG team, built with cibuildwheel). These wheels are the broadest-portability prebuilt SWIG available — Linux manylinux2010 (glibc 2.12) / manylinux2014, macOS 10.9 universal2 (x86_64+arm64 in one file), and Windows — far more portable than building on a dev box. Each wheel contains swig/data/bin/swig[.exe] + swig/data/share/swig/4.4.1/, i.e. ITK's exact layout, so repackaging is just extract-and-rezip (platform-independent — no cross-compiling):

process() {  # $1=pip-platform-tag  $2=itk-archive-name
  wd=$(mktemp -d)
  pip download --no-deps swig==4.4.1 --platform "$1" --only-binary=:all: -d "$wd"
  ( cd "$wd" && unzip -q swig-4.4.1*.whl )
  chmod +x "$wd/swig/data/bin/"*
  ( cd "$wd/swig/data" && zip -r -q "$OLDPWD/$2.zip" bin share )   # bin/ + share/ at archive root
}
process macosx_10_9_universal2   swigmacos-arm64-4.4.1   # universal2 also serves swigmacos-amd64
process manylinux2010_x86_64     swiglinux-amd64-4.4.1
process manylinux2014_aarch64    swiglinux-arm64-4.4.1
process win_amd64                swigwin-amd64-4.4.1

The resulting archives were uploaded to data.kitware.com and are fetched by SHA512:

archive	wheel source	SHA512
swigmacos-arm64 / -amd64	macosx_10_9_universal2	17c4637d…1ce3cef9
swiglinux-amd64	manylinux2010_x86_64	6447a7b9…d88ca11
swiglinux-arm64	manylinux2014_aarch64	44a6d174…c8902d24
swigwin-amd64	win_amd64	700bd4de…e338f3892

itk_assert_swig_version() runs swig -version on the downloaded binary at configure time and fails if it is not ≥ swig_version_min, so a wrong/corrupt upload is caught immediately. For a future SWIG bump, rerun the recipe with the new version, re-upload, and update ITK_SWIG_VERSION + the per-platform SHA512s.

Validation

Builds and imports on Python 3.11.15 and 3.13.9 (same abi3 build); the std::vector/SwigPyIterator path and 10/10 scoped Python ctests pass. The vendored FetchContent download from data.kitware.com was verified at configure time to fetch 4.4.1, pass the assertion, and build _ITKCommonPython.abi3.so cleanly.

Download mirrors

data.kitware.com (keyed by SHA512) is the primary; dweb.link, ipfs.io, and gateway.pinata.cloud serve the same bytes by content id as IPFS fallbacks, matching ITK's ExternalData gateway set. URL_HASH SHA512 validates whichever mirror responds. The content ids are the reproducible ipfs-car/IPIP-0499 CIDs produced by the ExternalData upload pipeline:

archive	CID
swigwin-amd64	bafybeifxjnbydus4eq46bocwb2hcj5ufpbj4etocia5ims426gfiw7krfi
swiglinux-amd64	bafybeiep2wbecrcii7xzfe2cekhfco6mlxqym4jhfdl6xi3p4aieblgz6m
swiglinux-arm64	bafybeifaqaus3yzp5oujxupxk422hfmcg2idf3oyvdtsppzq7ymn64nvze
swigmacos-arm64/amd64	bafybeidllsbllkqwoby6r5w4ueqzky2o7r6qkjbh45q57mczyh2fwfwd3q

The archives are pinned to IPFS and mirrored into ITKTestingData via the ExternalData upload pipeline.

[hjmjohnson]

Pushed a fix for the vendored-SWIG Python CI failure (ARMBUILD-Python / ITK.*.Python): the FetchContent refactor removed the swig ExternalProject target, but the wrapping build-ordered against it whenever ITK_USE_SYSTEM_SWIG=OFF. Now guarded with if(NOT ITK_USE_SYSTEM_SWIG AND TARGET swig) — prebuilt binaries are fetched at configure time and need no build-order dependency. Verified locally: the vendored path downloads SWIG 4.4.1 from data.kitware.com, passes the configure-time version assertion, and builds _ITKCommonPython.abi3.so cleanly.

[greptile-apps]

Greptile Summary

This PR upgrades ITK's Python wrapping toolchain to SWIG 4.4.1, fixing an import regression caused by SWIG 4.4's adoption of PEP 489 multi-phase module initialization. Three focused changes are included: a C-level runtime shim that detects and instantiates multi-phase PyModuleDef returns from PyInit_*, a promotion of the abi3 floor constant to a shared CMake cache variable so USE_SABI is always pinned to Python 3.11, and a migration of the prebuilt SWIG binary download from ExternalProject_Add (build-time) to FetchContent (configure-time) so SWIG_EXECUTABLE is available immediately and can be version-asserted before compilation begins.

• Multi-phase init shim (Wrapping/Generators/Python/CMakeLists.txt): injected C code now checks PyModule_Check on the PyInit_* return value and, when SWIG returns a PyModuleDef, creates a ModuleSpec via importlib.machinery and calls PyModule_FromDefAndSpec + PyModule_ExecDef; reference counting appears correct throughout.
• abi3 floor pin (CMake/ITKSetPython3Vars.cmake, itk_end_wrap_module.cmake): _ITK_MINIMUM_SUPPORTED_LIMITED_API_VERSION promoted to CACHE INTERNAL; USE_SABI is now always 3.11 rather than the current interpreter minor version, producing wheels loadable on any 3.11+ interpreter from a single abi3 build.
• FetchContent migration (Wrapping/Generators/SwigInterface/CMakeLists.txt): prebuilt archives for Windows/Linux/macOS are fetched and SHA512-verified at configure time; a new itk_assert_swig_version() helper confirms the binary reports ≥ 4.3.0 before proceeding; the source-build fallback on unsupported platforms stays on ExternalProject_Add.

Confidence Score: 3/5

Safe to merge on supported platforms (Win/Linux/macOS); the source-build fallback on unsupported platforms needs confirmation that the new hash is actually SWIG 4.4.1 source.

The multi-phase init shim and FetchContent migration are well-constructed, and the abi3 floor change is semantically correct. Two gaps stand out: errors during multi-phase module init are silently swallowed with PyErr_Clear, making failures very hard to diagnose; and the version assertion for prebuilt SWIG is bypassed if SWIG_EXECUTABLE is already cached. Most critically, the source-build fallback has contradictory signals — the hash and CID both changed in this PR, but the PR description explicitly calls the version bump a followup, leaving it unclear whether the new archive is SWIG 4.4.1; if not, the SWIG_DIR path (share/swig/4.4.1) would be wrong and unsupported-platform builds would fail with a confusing error.

Wrapping/Generators/SwigInterface/CMakeLists.txt (source-build fallback archive version and stale-cache bypass); Wrapping/Generators/Python/CMakeLists.txt (silent error swallowing in multi-phase init failure path)

Important Files Changed

Filename	Overview
Wrapping/Generators/SwigInterface/CMakeLists.txt	Migrates prebuilt SWIG download from ExternalProject to FetchContent for configure-time availability; adds version assertion, but version check is bypassed when SWIG_EXECUTABLE is already cached. Source-build fallback hash changed but PR description calls the version bump a followup, creating ambiguity.
Wrapping/Generators/Python/CMakeLists.txt	Adds PEP 489 multi-phase init handling for SWIG 4.4; reference counting looks correct but errors during module instantiation are silently swallowed with PyErr_Clear().
CMake/ITKSetPython3Vars.cmake	Promotes _ITK_MINIMUM_SUPPORTED_LIMITED_API_VERSION to CACHE INTERNAL so itk_end_wrap_module.cmake can share it; removes the local unset() that previously prevented cross-file use.
Wrapping/macro_files/itk_end_wrap_module.cmake	USE_SABI floor pinned from Python3_VERSION_MAJOR.MINOR to _ITK_MINIMUM_SUPPORTED_LIMITED_API_VERSION (3.11), producing a genuine abi3 wheel that loads on any 3.11+ interpreter.
pyproject.toml	Adds swig >=4.4.1 and castxml >=0.7.0 to the python pixi feature; configure-python and configure-debug-python tasks pass ITK_USE_SYSTEM_SWIG/ITK_USE_SYSTEM_CASTXML to use the conda-forge packages.
pixi.lock	Lock file updated to add swig-4.4.1 and castxml-0.7.0 plus their transitive dependencies (libllvm20, libclang-cpp20.1, icu, libxml2-16, pcre2) across all five platforms.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[CMake configure starts] --> B{ITK_USE_SYSTEM_SWIG?}
    B -- yes --> C[find_package SWIG - Version >= 4.3.0 check - FATAL_ERROR if old]
    B -- no --> D{swig_prebuilt? Win/Linux/macOS}
    D -- yes --> E{SWIG_EXECUTABLE cached and exists?}
    E -- no --> F[FetchContent_Declare itkswig SHA512 - FetchContent_MakeAvailable]
    F --> G[Set SWIG_EXECUTABLE - Set SWIG_DIR - Verify swig.swg]
    G --> H[itk_assert_swig_version >= 4.3.0]
    E -- yes --> I[Skip fetch and assertion - stale cache risk]
    D -- no --> J[ExternalProject_Add swig - source build at build-time - SWIG_DIR share/swig/4.4.1]
    C & H & I & J --> K[CMake wrapping macros run]
    K --> L{ITK_USE_PYTHON_LIMITED_API? Python >= 3.11}
    L -- yes --> M[Python3_add_library USE_SABI 3.11 - produces .abi3.so]
    L -- no --> N[Python3_add_library non-Limited-API]
    M & N --> O[Injected C init code runs at Python import time]
    O --> P{PyInit_* returns PyModule_Check?}
    P -- yes SWIG 4.3 single-phase --> Q[PyDict_SetItemString sys.modules]
    P -- no SWIG 4.4 multi-phase PyModuleDef --> R[importlib.machinery.ModuleSpec - PyModule_FromDefAndSpec - PyModule_ExecDef]
    R --> S{Success?}
    S -- yes --> Q
    S -- no --> T[PyErr_Clear - error silently dropped]

Loading

%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
flowchart TD
    A[CMake configure starts] --> B{ITK_USE_SYSTEM_SWIG?}
    B -- yes --> C[find_package SWIG - Version >= 4.3.0 check - FATAL_ERROR if old]
    B -- no --> D{swig_prebuilt? Win/Linux/macOS}
    D -- yes --> E{SWIG_EXECUTABLE cached and exists?}
    E -- no --> F[FetchContent_Declare itkswig SHA512 - FetchContent_MakeAvailable]
    F --> G[Set SWIG_EXECUTABLE - Set SWIG_DIR - Verify swig.swg]
    G --> H[itk_assert_swig_version >= 4.3.0]
    E -- yes --> I[Skip fetch and assertion - stale cache risk]
    D -- no --> J[ExternalProject_Add swig - source build at build-time - SWIG_DIR share/swig/4.4.1]
    C & H & I & J --> K[CMake wrapping macros run]
    K --> L{ITK_USE_PYTHON_LIMITED_API? Python >= 3.11}
    L -- yes --> M[Python3_add_library USE_SABI 3.11 - produces .abi3.so]
    L -- no --> N[Python3_add_library non-Limited-API]
    M & N --> O[Injected C init code runs at Python import time]
    O --> P{PyInit_* returns PyModule_Check?}
    P -- yes SWIG 4.3 single-phase --> Q[PyDict_SetItemString sys.modules]
    P -- no SWIG 4.4 multi-phase PyModuleDef --> R[importlib.machinery.ModuleSpec - PyModule_FromDefAndSpec - PyModule_ExecDef]
    R --> S{Success?}
    S -- yes --> Q
    S -- no --> T[PyErr_Clear - error silently dropped]

Loading

_{Reviews (1): Last reviewed commit: "COMP: Factor swig vendored-download mirr..." | Re-trigger Greptile}

[greptile-apps]

Silent failure swallows module-init errors

When PyModule_FromDefAndSpec or PyModule_ExecDef fails (e.g., a missing symbol, ABI mismatch, or SWIG runtime issue), ${base_name}Module is left NULL, PyErr_Clear() drops the active exception, and execution continues. The submodule is never inserted into sys.modules, so users see a confusing AttributeError: module 'itk' has no attribute '...' rather than the actual initialization error. At minimum, the pending exception should be printed with PyErr_WriteUnraisable or converted to a warning before being cleared, so the root cause is visible in the Python traceback.

[greptile-apps]

Version assertion skipped when SWIG_EXECUTABLE is already cached

The entire FetchContent_Declare / itk_assert_swig_version block is guarded by if(NOT SWIG_EXECUTABLE OR NOT EXISTS "${SWIG_EXECUTABLE}"). If a developer is upgrading from the previous 4.3.x prebuilt (binary still present in _deps/) without deleting CMakeCache.txt, SWIG_EXECUTABLE passes both conditions and the stale 4.3.x binary is silently used. The version assertion is never reached, so the incompatibility (4.3.x lacking PEP 489 multi-phase init) is not caught at configure time; the build proceeds and fails at import with the same obscure 'moduledef' object error this PR is fixing.

[greptile-apps]

Source-build archive version may not match SWIG_DIR path

SWIG_VERSION is set to ITK_SWIG_VERSION (4.4.1), so SWIG_DIR resolves to share/swig/4.4.1/. Both swig_hash and swig_cid changed from the prior values, yet the PR description lists "bump source-build fallback swig-4.0.2 to 4.4.1" as a followup, implying the archive behind the new hash might not be the SWIG 4.4.1 tarball. If the extracted archive installs its standard library to share/swig/4.0.2/, SWIG_DIR will point to a non-existent path and the build will fail on any unsupported platform (non-x86_64/aarch64 Linux, non-Darwin) with a cryptic error. Can you confirm whether the new swig_hash corresponds to a SWIG 4.4.1 source tarball? Does the new swig_hash/swig_cid in the source-build fallback correspond to a SWIG 4.4.1 source tarball? The PR description lists this bump as a followup, but both the hash and CID changed in this PR.