Python 3.14 support and dependency pinning and deprecation warning policies

[gotmax23]

Description

Please leave a brief description of the bug or feature request:

Fedora is proud to be an early adopter of Python 3.14 as part of our goal to help the ecosystem by testing and integrating new Python versions, and I worked on getting all dependencies functional and addressing test failures in the scancode stack. I am creating this issue to track supporting Python 3.14 officially upstream. We only run a subset of upstream scancode tests, but this reflects new failures after the Python 3.14 update.

There are also some concerns I have with dependency pinning and deprecation warning practices in the aboutcode projects that I mentioned that created extra friction when updating to Python 3.14. See the Commentary below.

scancode issues

• packagedcode: don't use removed ast module attributes #4539 that I filed earlier today was needed due to ast module removals.

• We ignored the remaining test failures in our scancode build:

  # ERROR: 'sbin/fstrim' is a link to an absolute path
  # These two are due to the tarfile module defaulting to the data filter in py3.14.
  # Let upstream figure out what to do about this issue.
  test_can_get_installed_system_packages_with_license_from_alpine_container_layer
  test_can_scan_installed_system_package_in_alpine_container_layer
  # These test failures have to do with the handling of tempfiles in the CLI
  # and don't seem particularly important. Disable them for now
  test_scan_keep_temp_files_is_false_by_default
  test_scan_keep_temp_files_keeps_files and not

• test_license_reference_to_file_beside_package_manifest test is broken #4540 is not directly caused by Python 3.14 but also impacted us while trying to update scancode to the latest version.

Commentary: Both the ast and tarfile issues were due to known deprecation that upstream seemingly ignored, so perhaps aboutcode should consider adopting a more proactive approach to addressing deprecation warnings (such as running pytest with -W error) and avoid pinning old dependency versions ¹. I'm not sure what's going on with the temp_files tests. I didn't take the time to debug them, and I'm not sure if the issue is with the Fedora build environment.

dependency issues

• I replaced pkginfo2 with pkginfo in scancode with a downstream patch. pkginfo2 seems to be an under-maintained fork of pkginfo and has failures with new setuptools versions that prevented it from being rebuilt for Python 3.14. These were reported in Tests are failing with new setuptools due to new metadata version pkginfo2#5 but never addressed upstream.
• Test failure on Python 3.14 commoncode#88 requires disabling an additional test in our commoncode package. This was also caused by tarfile changes.
• Issues with beartype prevented spdx-tools from being rebuilt. These were fixed in [Feature Request] Official Python 3.14 support! It do be like: ☢ ☣ ⚠ 🦖  beartype/beartype#537, but a new upstream release has not yet been created.
• Once we updated beartype to the latest git snapshot, I backported spdx/tools-python@6817ca7 for beartype compatibility and submitted relationship_writer: properly access __annotations__ dict spdx/tools-python#858 to spdx-tools to fix other issues that came up in the spdx-tools package. The spdx-tools maintainers have not yet applied my patch.

Commentary: The beartype issues were understandable due to major typing changes in Python 3.14. I'm not sure about the state of the pkginfo2 project or spdx-tools.

How To Reproduce

Tell us how to reproduce the issue.

Run scancode tests in an environment with Python 3.14. Make sure to install the latest git snapshots of https://github.com/gotmax23/tools-python/tree/py314-annotations and https://github.com/beartype/beartype that have the patches mentioned.

System configuration

For bug reports, it really helps us to know:

• What OS are you running on? (Windows/MacOS/Linux) Fedora Linux Rawhide (44)
• What version of scancode-toolkit was used to generate the scan file? 32.4.1
• What installation method was used to install/run scancode? (pip/source download/other) source

Thank you for reading my wall of text and for all the work you do on scancode!

Footnotes

1. EDIT: To be clear, I was referring to pinning of test dependencies for CI in requirements.txt, not the dependencies in the packaging metadata (setup.cfg) which are specified properly without excessive pinning for the most part. After looking again, it seems like requirements.txt does get updated on a somewhat regular basis but importlib-metadata was just held back in https://github.com/aboutcode-org/scancode-toolkit/commit/97bc9b5a6e752d06124280c66d5965b6be723293. Using something like dependabot or a Github Actions scheduled pipeline to automatically submit PRs to bump requirements.txt is still a good idea. ↩

[reneleonhardt]

Great to see Python 3.14 support!

I can't even install it on Python 3.13 😅

uvx scancode-toolkit 
      Built ftfy==4.4.3                                                                                                                                                                × Failed to build `bitarray==0.9.3`
  ├─▶ The build backend returned an error
  ╰─▶ Call to `setuptools.build_meta:__legacy__.build_wheel` failed (exit status: 1)

      [stderr]
      /Users/rene/.cache/uv/builds-v0/.tmp5o48Gd/lib/python3.13/site-packages/setuptools/dist.py:759: SetuptoolsDeprecationWarning: License classifiers are deprecated.
      !!

              ********************************************************************************
              Please consider removing the following classifiers in favor of a SPDX license expression:

              License :: OSI Approved :: Python Software Foundation License

              See https://packaging.python.org/en/latest/guides/writing-pyproject-toml/#license for details.
              ********************************************************************************

      !!
        self._finalize_license_expression()
      bitarray/_bitarray.c:131:23: error: expression is not assignable
        131 |         Py_SIZE(self) = newsize;
            |         ~~~~~~~~~~~~~ ^
      bitarray/_bitarray.c:156:19: error: expression is not assignable
        156 |     Py_SIZE(self) = newsize;
            |     ~~~~~~~~~~~~~ ^
      bitarray/_bitarray.c:177:18: error: expression is not assignable
        177 |     Py_SIZE(obj) = nbytes;
            |     ~~~~~~~~~~~~ ^
      bitarray/_bitarray.c:1372:14: error: call to undeclared function 'PyEval_CallObject'; ISO C99 and later do not support implicit function declarations
      [-Wimplicit-function-declaration]
       1372 |     result = PyEval_CallObject(reader, rargs);
            |              ^
      bitarray/_bitarray.c:1372:12: error: incompatible integer to pointer conversion assigning to 'PyObject *' (aka 'struct _object *') from 'int' [-Wint-conversion]
       1372 |     result = PyEval_CallObject(reader, rargs);
            |            ^ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
      bitarray/_bitarray.c:1511:14: error: call to undeclared function 'PyEval_CallObject'; ISO C99 and later do not support implicit function declarations
      [-Wimplicit-function-declaration]
       1511 |     result = PyEval_CallObject(writer, args);
            |              ^
      bitarray/_bitarray.c:1511:12: error: incompatible integer to pointer conversion assigning to 'PyObject *' (aka 'struct _object *') from 'int' [-Wint-conversion]
       1511 |     result = PyEval_CallObject(writer, args);
            |            ^ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
      bitarray/_bitarray.c:3206:28: error: expression is not assignable
       3206 |     Py_TYPE(&Bitarraytype) = &PyType_Type;
            |     ~~~~~~~~~~~~~~~~~~~~~~ ^
      bitarray/_bitarray.c:3207:31: error: expression is not assignable
       3207 |     Py_TYPE(&SearchIter_Type) = &PyType_Type;
            |     ~~~~~~~~~~~~~~~~~~~~~~~~~ ^
      bitarray/_bitarray.c:3208:31: error: expression is not assignable
       3208 |     Py_TYPE(&DecodeIter_Type) = &PyType_Type;
            |     ~~~~~~~~~~~~~~~~~~~~~~~~~ ^
      bitarray/_bitarray.c:3209:33: error: expression is not assignable
       3209 |     Py_TYPE(&BitarrayIter_Type) = &PyType_Type;
            |     ~~~~~~~~~~~~~~~~~~~~~~~~~~~ ^
      11 errors generated.
      error: command '/usr/bin/cc' failed with exit code 1

      hint: This usually indicates a problem with the package or the build environment.
  help: `bitarray` (v0.9.3) was included because `scancode-toolkit` (v21.2.9) depends on `bitarray`

bitarray v0.9.3 is 6 years old: https://pypi.org/project/bitarray/#history

I can only find outdated test data:
https://github.com/aboutcode-org/scancode-toolkit/blob/develop/tests/packagedcode/data/pypi/develop/scancode_toolkit.egg-info/requires.txt#L3

bitarray<1.0.0,>=0.8.1

But of course nobody should use requirements pinning anymore, exactly why packages can't be installed on newer Python versions anymore when updates are forbidden.
https://iscinumpy.dev/post/bound-version-constraints/

[gotmax23]

help: bitarray (v0.9.3) was included because scancode-toolkit (v21.2.9) depends on bitarray

The latest version of scancode-toolkit is 32.4.1 which should support Python 3.13. You are trying to install 21.2.9 from over four years ago. It looks like you are having a separate issue and should create a new ticket for that.

But of course nobody should use requirements pinning anymore, exactly why packages can't be installed on newer Python versions anymore when updates are forbidden.

I was referring to the fact that tests are all run in upstream CI using library versions pinned in requirements.txt, which is generally a good practice, provided that they are updated on a regular basis and not pinned to versions that are significantly out of date.

Pinning bitarray<1.0.0,>=0.8.1 in the packaging metadata probably made sense at the time if it was expected that bitarray 1.0.0 would have breaking changes.

[reneleonhardt]

You can see my command line, that's what uvx resolves to when ommitting the version.
But

uvx scancode-toolkit==32.4.1

isn't much better for Python 3.13.

  × No solution found when resolving tool dependencies:
  ╰─▶ Because only the following versions of typecode-libmagic are available:
          typecode-libmagic<=5.39.210223
          typecode-libmagic==5.39.210531
      and typecode-libmagic>=5.39.210223 has no wheels with a matching platform tag (e.g., `macosx_15_0_arm64`), we can conclude that typecode-libmagic>=5.39.210223 cannot be
      used. (1)
      And because typecode[full]>=30.0.1 depends on typecode-libmagic>=5.39.210223 and only the following versions of typecode[full] are available:
          typecode[full]<=30.0.1
          typecode[full]==30.0.2
      we can conclude that typecode[full]>=30.0.1,<30.0.2 cannot be used. (2)

      Because we know from (1) that typecode-libmagic>=5.39.210223 cannot be used and typecode[full]==30.0.2 depends on typecode-libmagic>=5.39.210223, we can conclude that
      typecode[full]==30.0.2 cannot be used.
      And because we know from (2) that typecode[full]>=30.0.1,<30.0.2 cannot be used, we can conclude that typecode[full]>=30.0.1 cannot be used.
      And because scancode-toolkit==32.4.1 depends on typecode[full]>=30.0.1 and you require scancode-toolkit==32.4.1, we can conclude that your requirements are unsatisfiable.

      hint: Wheels are available for `typecode-libmagic` (v5.39.210531) on the following platforms: `manylinux1_x86_64`, `macosx_10_14_x86_64`, `win_amd64`

But I didn't want to steal your thread, didn't know that you don't support arm64 or Apple Silicon yet, typecode-libmagic doesn't seem to be maintained anymore, last release is 4 years old:
https://pypi.org/project/typecode-libmagic/#files.

I can only recommend using the standard cibuildwheel action if possible to build for all platforms and architectures.
https://github.com/pypa/cibuildwheel#example-setup

[stefan6419846]

But I didn't want to steal your thread, didn't know that you don't support arm64 or Apple Silicon yet, typecode-libmagic doesn't seem to be maintained anymore, last release is 4 years old

This is a separate issue and is tracked in #3958.

[pombredanne]

@gotmax23 Thanks for all this. This is awesome! As for pkginfo2, it needs to be updated, but the reason for the fork and patch is security: pkginfo does insecure evaluation of code that is possibly dangerous when processing random code at scale. It may be OK for pkginfor, but is not OK for ScanCode

[AyanSinhaMahapatra]

Thanks++ for all your help @gotmax23, much appreciated.
python 3.14 is now supported in the latest scancode-toolkit release at https://github.com/aboutcode-org/scancode-toolkit/releases/tag/v32.5.0
Closing finally!

[reneleonhardt]

Great to see Python 3.14 support 🥳
Did you already try 3.15, could something be improved to make the update process easier?

[AyanSinhaMahapatra]

@reneleonhardt not really, python3.15 should not be ready for another half an year atleast: https://peps.python.org/pep-0790/#release-schedule so usually we don't plan on testing before we have release candidates on python3.15, because:

• There will be new features/deprecations added which would need to be addressed, better to do at once
• if this is not a stable release, there could be a lot of bugs which we cannot do anything about mostly, and once there are stable releases these tend to get fixed more
• We also need some python releases so we can test these efficiently on the CI without changing things a lot

One thing which we could improve is:

• handle deprecations earlier by looking at all the deprecation warnings ahead of time and having this in one place
• check new python support for the non pure-python dependencies earlier as these take up some amount of time

Help is much welcome here, like @gotmax23 helped us a lot with the issues and PRs for the python3.14 support.
So if you want to play with python3.15 support even in the pre-releases, please do, and also let us know if you face any issues there

[stefan6419846]

In general, at least starting testing with the release candidate is recommended. From my experience, SCTK still lagged behind with support for new Python releases by some months in the past.

if this is not a stable release, there could be a lot of bugs which we cannot do anything about mostly, and once there are stable releases these tend to get fixed more

I tend to start testing my tools and libraries with the latest unreleased Python versions early and - apart from having to build some native libraries manually and some rather uncommon failures with less than a handful of packages - did not observe any of such hard issues. The most annoying packages tend to be Rust-based ones. At least my basic testing of the most relevant SCTK functionality for my use-case with Python 3.15.0-alpha.5 did not run into hard issues.

handle deprecations earlier by looking at all the deprecation warnings ahead of time and having this in one place

I guess this is one of the most important aspects to consider, as well as timely feedback from maintainers about reported issues and PRs. While I understand that this might be a lot of work to start with, it could lower the burden for new or interested contributors (although this seems to have changed lately, with multiple persons wanting to work on newly reported issues and pinging maintainers constantly). At the moment it sometimes feels like issues or PRs go unnoticed for multiple months, especially in dependencies maintained as part of the organization. Nevertheless, this is something which probably should not be discussed here, but in a dedicated issue or across the current maintainers (which I am not, just a user and library consumer occasionally opening issues or PRs).

[reneleonhardt]

Never heard of massive problems from betas or release candidates.
Just a food for thought, official testing starts with beta1 to reduce upgrade efforts and delays to a minimum.

https://pythoninsider.blogspot.com/2025/05/python-3140-beta-1-is-here.html (May 7, 2025)

We strongly encourage maintainers of third-party Python projects to test with 3.14 during the beta phase and report issues found to the Python bug tracker as soon as possible. While the release is planned to be feature-complete entering the beta phase, it is possible that features may be modified or, in rare cases, deleted up until the start of the release candidate phase (Tuesday 2025-07-22). Our goal is to have no ABI changes after beta 4 and as few code changes as possible after the first release candidate. To achieve that, it will be extremely important to get as much exposure for 3.14 as possible during the beta phase.