Python 3.13 / 3.14 cross-build support#63
Closed
FeodorFitsner wants to merge 141 commits into
Closed
Conversation
* Added Android support * Fixed pillow, pandas and other Meson-baked packages * Try building a simple android package * Check python location--upgrade setuptools * re-build index * display python path * ensurepip --upgrade * Fix python * which python$PYTHON_SHORT_VERSION * which python in different place * Try again * echo $PATH * configure rust before * don't use source command * configure forge first * echo $PATH * cat source * Remove deactivate * Remove junk * Test android * Fix python in setup.sh * Re-build all Android packages * Trying to fix pillow for android * Pillow - disable platform guessing * Re-run all Android jobs * Trying to fix contourpy * Fix host tool shims. Build contourpy, matplotlib and pandas on Android * Try iOS-like shim script template * Fix path to pybind config * Build pandas and contourpy on all android abis * Re-build all android packages
* time-machine package added * Always build as simple package if `build.sh` found * flet-libcpp-shared recipe * flet-libjpeg recipe * Strip libraries in wheels - draft * Strip .so in built wheels * Build flet-libjpeg * PYTHON_VERSION: 3.12.7 * flet-libjpeg 3.0.90 * NDK_VERSION: r27c * Re-build android packages * Fix yaml * Re-build android packages * Fix recipes * Re-build numpy * re-built wheel in a temp dir * ls -al dist * Re-build matplotlib * flet-libcpp-shared * Add build number to a wheel * Re-build android packages * Fix kiwisolver
ZLIB_INCLUDE adds /usr/include which leaks host glibc headers on CI. Keep SSL_INCLUDE (now via OPENSSL_ROOT_DIR) always, skip ZLIB_INCLUDE on Android only.
opencv-python uses scikit-build/CMake which doesn't inherit LDFLAGS. Pass -DCMAKE_SHARED_LINKER_FLAGS with max-page-size=16384 via CMAKE_ARGS.
cv2.abi3.so is a CMake MODULE, not SHARED library, so it needs CMAKE_MODULE_LINKER_FLAGS in addition to CMAKE_SHARED_LINKER_FLAGS.
Use exact version (==) by default, preserve explicit specifiers like >=, and handle dependencies without versions. Fixes double >= in output.
* Bump flet-libxml2 to 2.15.3
Drive the source URL and patch selection from a single Jinja version
variable so switching back to 2.9.8 still builds. Replace the old
mobile.patch (config.sub iOS triplets + libxml2.syms tweaks for the
2.9 layout) with two version-targeted patches: mobile-2.9.x.patch
keeps the original content, mobile-2.15.x.patch only teaches the
modernized config.sub about *-apple-ios-simulator.
build.sh: pass --without-iconv on Android (NDK bionic lacks iconv
until API 28; 2.15.x makes it mandatory by default while 2.9.x
silently soft-failed). Make the post-install cleanup tolerate the
2.15.x layout where share/ is no longer created and globs may not
match (nullglob + rm -rf).
* Bump flet-libxslt to 1.1.45; ship static archives in iOS wheels
flet-libxslt: drive version, libxml2 host requirement, and patch
selection from a single Jinja block (numpy-style). Replace the old
mobile.patch (config.sub iOS triplets for the 1.1.32 layout) with
two version-targeted patches: mobile-1.1.32.patch keeps the original
content, mobile-1.1.45.patch only teaches the modernized config.sub
about *-apple-ios-simulator. 1.1.45 pulls in flet-libxml2 2.15.3 to
satisfy its >= 2.15.1 requirement.
build.sh: only recurse into libxslt/libexslt subdirs. xsltproc links
-lxml2 and on iOS the SDK's libxml2.tbd predates 1.1.45's use of
xmlCtxtParseDocument / xmlXPathValuePush, so the CLI binary fails to
link; the wheel only needs the libraries. Same cleanup robustness
fix as libxml2 (nullglob + rm -rf).
flet-libxml2 + flet-libxslt build.sh: stop deleting *.a from the
wheel's lib/. iOS only builds static libs, so the previous cleanup
left lib/ empty and forced downstream packages onto dynamic_lookup
or the SDK stub. Android only builds shared libs (no *.a produced),
so this is a no-op there.
flet-libxml2/meta.yaml: switch to the comment-prefixed Jinja idiom
(# {% ... %}) so YAML linters parse it cleanly, matching numpy and
flet-libxslt.
* Bump lxml to 6.1.0
Drive version, libxml2/libxslt host pinning, and the iOS-only
LDFLAGS=-liconv from a single Jinja block. lxml 5.x doesn't compile
against libxml2 2.15 (per upstream's CHANGES) so the older lxml
version stays paired with libxml2 2.9.8 / libxslt 1.1.32.
iOS libxml2.a now references iconv (it's built --with-iconv against
the system libiconv), and lxml's libraries() list omits iconv on
non-Windows non-static builds. Add -liconv via script_env LDFLAGS
on iOS only — Android's libxml2 is built --without-iconv and doesn't
need it.
The existing mobile.patch (filter MacOSX SDK include path out of
xml2-config --cflags) still applies cleanly to 6.1.0.
* Add native-recipe-bumps skill
Captures the conventions used while bumping flet-libxml2 (2.9.8 →
2.15.3), flet-libxslt (1.1.32 → 1.1.45), and lxml (5.3.0 → 6.1.0):
the comment-prefixed Jinja idiom for version-conditional meta.yaml
(URL, host pins, patch filename in one block), the build.sh quirks
that bit us (bash 3.2 + set -u, nullglob cleanup, keeping *.a in
the wheel, SUBDIRS overrides, --without-iconv on Android), the
pitfall catalogue (iOS SDK libxml2.tbd, lxml's missing -liconv,
config.sub ios-simulator gap, libxml2.syms removed in 2.10), and
the forge CLI surface for re-running builds.
* Rewrite absolute-path DT_NEEDED entries in built .so files CMake-based recipes link against libpython by absolute path. Combined with libpython lacking a DT_SONAME, that absolute build-host path ends up recorded verbatim in the extension's DT_NEEDED list and won't resolve at load time on the target device. Shift each NEEDED d_val past the last '/' to point at the basename within the same string — no string rewriting required. Handles both ELF32 and ELF64. * Add coolprop recipe for Android and iOS
Include a conditional host requirement in recipes/rasterio/meta.yaml: add flet-libcpp-shared >=27.2.12479018 wrapped in a Jinja if-check (sdk == 'android'). This ensures the shared C++ runtime is only required for Android builds; no other changes were made.
* initial commit * migrate to using uv * add Rust setup and target configuration for mobile builds * enhance CI workflow to detect changed recipes and add publish option * update build-wheels workflow to remove additional support-tree dependency wheels
* recipe: selectolax 0.4.10 * Update build-wheels.yml to publish wheels on python3.12 branch push
* recipe: biopython 1.87 * Update build-wheels.yml to publish wheels on python3.12 branch push
fix_wheel was unconditionally rewriting the WHEEL Tag with `self.wheel_tag` (cp3X-cp3X-<platform>), discarding the tag maturin / the upstream backend had emitted. For abi3 crates like cryptography this turned `cp37-abi3-<platform>` into `cp312-cp312-<platform>` and the produced wheel name followed, which was wrong on three counts: - semantically incorrect: the inner _rust.abi3.so is stable-ABI; - unnecessarily restrictive: pip refuses the wheel on cp313+; - wasteful: forces per-Python-version rebuilds. Now `fix_wheel` keeps the existing pythontag-abitag portion and swaps only the platform component, falling back to self.wheel_tag only when the upstream wheel didn't carry a Tag header. Restores the cp3X-abi3-* wheels we shipped before commit 4cf1c1f.
* Remove build_number input and related references from build-wheels workflow. * set build numbers for all recipes based on their respective PyPi max values
* numpy: declare flet-libcpp-shared host dep on Android * bump numpy meta.yaml build number
…` recipe (#62) * pyyaml: bundle the _yaml C accelerator via new flet-libyaml recipe The pyyaml wheels currently published on pypi.flet.dev ship pure- Python only — every Android build of PyYAML 6.0.2 to date is missing the `_yaml` Cython accelerator, and iOS has never shipped a pyyaml wheel at all (`https://pypi.flet.dev/pyyaml/` lists four Android wheels and zero iOS wheels). PyYAML's setup.py wraps the C-extension build in `except DistutilsPlatformError: log.warn ("skipping build_ext")`, so without libyaml's headers + .so on the build root the wheel still builds, it just silently omits the .so. The pure-Python loader works for basic `yaml.safe_load`, so the gap is invisible until code reaches for `yaml.CSafeDumper` / `yaml.CSafeLoader` (absent from the namespace because cyaml.py only exposes them when `_yaml` imports), or until someone notices 500-doc round-trips take ~10× longer than they should. Three changes here: 1. `recipes/flet-libyaml/` — new recipe for libyaml 0.2.5 (https://github.com/yaml/libyaml). Builds the shared library on Android and both shared + the .a archive on iOS, mirroring how `flet-libsodium` is consumed by PyNaCl: Android pyyaml dynamically links libyaml.so, iOS pyyaml links it statically into `_yaml.cpython-*.so`. `libyaml_la_LDFLAGS` is rewritten to add `-avoid-version` (so the install name stays `libyaml.so` / `libyaml.dylib` instead of `libyaml-0.so`) and `-pthread` (filler that keeps libtool's bare `-F` on iOS from eating the next token, mirroring the libsodium fix). 2. `recipes/pyyaml/meta.yaml` — declare `flet-libyaml 0.2.5` as a host dep. Comment explains the silent skip behaviour so a future reader doesn't undo it without realising what they're removing. 3. `recipes/pyyaml/test_pyyaml.py` — three pytest functions that any pyyaml-using Flet app actually depends on: - `test_basic`: yaml.safe_dump → safe_load roundtrip. Passes even on a pure-Python wheel; proves the wheel ships. - `test_c_extension`: import _yaml + assert hasattr(_yaml, "CParser"). This is the canary — fires when the .so was never bundled (the current published wheel) or when libyaml fails to load at runtime on-device. - `test_csafedumper_binding`: `from yaml import CSafeDumper, CSafeLoader`. Functionally subsumed by test_c_extension but kept because it's the import shape real apps break on, and a clean ImportError here points a debugger at the _yaml/libyaml chain instead of an obscure attribute-missing surprise downstream. Note on the publish-first dance ------------------------------- This PR introduces a brand-new flet-libyaml package that pyyaml host-deps. CI's matrix is flat (no inter-job artifact sharing), so on the first run after merge the `pyyaml` job will fail to resolve `flet-libyaml==0.2.5` from pypi.flet.dev — it doesn't exist there yet, and the `flet-libyaml` job builds it in a different runner. The unblock: dispatch the workflow manually with `packages=flet-libyaml:1` + `publish=true` to push libyaml's wheels to pypi.flet.dev via Gemfury, then re-run the pyyaml job (or wait for the next push). Once libyaml is on pypi.flet.dev, pyyaml resolves cleanly and produces a wheel that actually includes `_yaml.cpython-*.so` on both Android and iOS. * pyyaml: bump build.number from 4 to 5 The previously-published pyyaml-6.0.2-4-*.whl set on pypi.flet.dev (Android: arm64-v8a, armeabi-v7a, x86, x86_64; iOS: none) was built before `flet-libyaml` existed, so PyYAML's setup.py silently skipped `_yaml` and shipped pure-Python only. Build 5 — produced from the same source tarball but with libyaml on the build root courtesy of the preceding commit — ships `_yaml.cpython-*.so` for the first time and supersedes the -4- wheels via PEP 427 build-tag tie-break. iOS gets a pyyaml wheel for the first time at the same build tag.
* recipe: pyxirr 0.10.8 * remove mobile patch * Add build number to pyxirr meta.yaml
Resolutions: - .appveyor.yml: accept python3.12 deletion (CI moved to GitHub Actions) - setup.sh: keep python3.12 uv-based bootstrap; replace hard-coded `[ "$PYTHON_VER" = "3.12" ]` with `python_version_minor -lt 13` so the armeabi-v7a/x86 Android validation matches the runtime gate in cross.py and make_dep_wheels.py - src/forge/build.py: take python3.12 (16KB page-alignment LDFLAGS, _rewrite_absolute_needed/_check_elf_alignment helpers, upstream Python/ABI tag preservation in fix_wheel); drop the duplicate NDK env block that python3.13 had appended - .github/workflows/build-wheels.yml: take python3.12's matrix-generating workflow, bump UV_PYTHON to 3.13.12, and trim Android rust_targets to arm64 + x86_64 (no armeabi-v7a/i686 on 3.13+) - .github/workflows/build-wheels-with-cibuildwheel.yml: take python3.12 (uvx cibuildwheel, workflow_dispatch trigger) - recipes/flet-libcpp-shared/meta.yaml: take python3.12 (adds explicit build.number, required after meta.yaml became the source of truth) - recipes/flet-libcurl/build.sh: take python3.12 (PKG_CONFIG=false fix for Android cross builds)
`source ./setup.sh 3.13.13` now picks up MOBILE_FORGE_IOS_SUPPORT_PATH_3_13 and MOBILE_FORGE_ANDROID_SUPPORT_PATH_3_13 if they're set, so a single .envrc can declare paths for 3.12 / 3.13 / 3.14 side-by-side instead of having to re-export per version. The unversioned MOBILE_FORGE_*_SUPPORT_PATH vars are still honored as a fallback.
The per-version support-path resolver was using bash's `${!varname}`
indirect expansion. zsh doesn't support that form (it uses `${(P)var}`),
so sourcing .envrc from a zsh shell - the default on recent macOS -
aborted with `./setup.sh:47: bad substitution` before any venv could
be created.
Replace with `eval "varname=\${$indirect:-}"`, which is portable across
both shells. Functionally identical to the prior bash-only path: empty
or unset overrides leave the unversioned MOBILE_FORGE_*_SUPPORT_PATH
fallback untouched.
Python 3.14 made several changes that the upstream-pinned `@flet` branch
can't handle yet:
- `sysconfig._init_config_vars` became lazy, so crossenv's PathFinder
patch recursed into half-initialised sysconfig and crashed with
`AttributeError: 'installed_base'`.
- The patched-modules sweep iterated `sys.modules.items()` directly,
and the lazy sysconfig init grew it mid-iteration on 3.14, raising
`RuntimeError: dictionary changed size during iteration`.
- `ctypes/__init__.py` started doing
`PyDLL(_sysconfig.get_config_var("LDLIBRARY"))` at module load, and
the crossenv-patched value (the host's `libpython3.X.so`) doesn't
exist on the build platform's dlopen path.
All three are addressed in the open PR (flet-dev/crossenv#1). Pin the
branch directly so a fresh `pip install -e .` in a 3.13/3.14 venv gets
the fix without waiting for the PR to land on `@flet`. Flip back to
`@flet` once the PR is merged.
Three changes that together unblock building Rust-bound extension wheels for Android on Python 3.14. src/forge/build.py: add the host LIBDIR to the cargo link-search path. PYO3_CROSS_LIB_DIR is required to point at the host stdlib directory so that maturin's auto-detector finds `_sysconfigdata__*.py` (and, on 3.14+, build-details.json) - but that directory does not contain the actual `libpython*.so` files. The linker then receives `-lpython3` / `-lpython3.X` and fails to resolve them. Pull LIBDIR out of the host sysconfig and add it as an extra `-L` so the canonical libpython location is on the search path while PYO3_CROSS_LIB_DIR keeps doing its sysconfig-discovery job. recipes/cryptography/meta.yaml: select 48.0.0 on Python 3.14, keep 43.0.1 on <=3.13. cryptography 43.0.1 pins pyo3 0.22.2, which hard-stops at Python 3.13 (`error: the configured Python interpreter version (3.14) is newer than PyO3's maximum supported version (3.13)`). 48.0.0 moves the workspace to pyo3 0.28, which knows 3.14 natively and is classified as 3.14-supported on PyPI. Selecting per-version via Jinja so 3.12/3.13 consumers keep the previously-validated 43.0.1 build. recipes/pydantic-core/meta.yaml: bump 2.33.2 -> 2.47.0. Same root cause - 2.33.2 bundles a pre-3.14 pyo3 and aborts the Rust build with the same "newer than PyO3's maximum supported version" check; 2.47.0 bumps the embedded pyo3 to a version that knows 3.14.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Brings the Python 3.13 / 3.14 cross-build path on
python3.13to the point where Rust-bound extension wheels (cryptography, pydantic-core, …) build end-to-end on both iOS and Android, and folds in allpython3.12improvements since the branch was opened.Highlights
setup.sh— per-versionMOBILE_FORGE_{IOS,ANDROID}_SUPPORT_PATH_<MAJOR>_<MINOR>overrides so a single.envrccan declare 3.12 / 3.13 / 3.14 paths side-by-side andsource ./setup.sh <ver>picks the right pair. Drops the hardcoded[ "$PYTHON_VER" = "3.12" ]gate in favor of a numericpython_version_minor -lt 13check matching src/forge/cross.py:25. Indirect-var resolution now usesevalso it works in both bash and zsh.python3.12— pulls in everything that landed on python3.12 since the branch diverged (new/updated recipes, build-system fixes, CI matrix workflow). Conflicts resolved in favor of the newer python3.12 logic forsetup.sh,src/forge/build.py, the GHA workflows, and several recipes.-L<LIBDIR>for the Android linker (src/forge/build.py) —PYO3_CROSS_LIB_DIRmust point at the host stdlib (so maturin's auto-detector finds_sysconfigdata__*.pyandbuild-details.json), butlibpython*.solives in<host>/lib/(LIBDIR). Adding LIBDIR as an extra cargo search path unblocks-lpython3/-lpython3.<X>resolution without breaking maturin's discovery.forgeCLI changes from existing python3.13 work —cross.py/build.pyalready key paths/wheel-tags offsys.version_info.minor, so the CLI is already multi-Python aware once the surrounding plumbing above lands.Depends on
darwin-std-pkgPR (Android dep-wheel layout for 3.13+,build-details.jsonpath rewrite) — required for 3.13/3.14 Android builds to find native libs / linklibpython. PR opened alongside this one.fix/python-3.14-sysconfig-lazy-init(Avoid sysconfig recursion on Python 3.14 in PathFinder patch crossenv#1) — required for 3.14 cross-env bootstrap. The crossenv pin here stays at@flet; the PR will be merged into@fletbefore this lands.Test plan
forge iphoneos:arm64 cryptographyend-to-end (build → wheel).forge android:arm64-v8a cryptographyend-to-end.🤖 Generated with Claude Code