Skip to content

add hatchling support for manual build mode#35

Merged
dfangl merged 7 commits intomainfrom
hatchling-support
Mar 11, 2026
Merged

add hatchling support for manual build mode#35
dfangl merged 7 commits intomainfrom
hatchling-support

Conversation

@thrau
Copy link
Member

@thrau thrau commented Nov 28, 2025

Motivation

Plux has been deeply depending on setuptools, but with #34 we now have a way to easily integrate new build backends. Hatch and hatchling are the first candidates given their significance in the python ecosystem.

This PR adds a Project implementation for hatchling as build backend, but limited to manual build mode for now (no build hook integration). The main things I added are:

  • Finding packages with hatchling: The primary difference to manual build mode in setuptools is the way the config is scanned and packages are discovered. In setuptools, there is a package finder that already recursively lists all existing python packages in the source tree. Hatchling does not have that, so I needed to add our own implementation that resolves packages, but uses information from the hatchling config in the case of namespace packages.

  • Test isolation: Because having both hatchling and setuptools in the same process makes testing unreliable, I found a hacky way to isolate tests (see the docstrings in the module for more explanation). I guess it's ok for now, given that everything was previously set up to use setuptools, so the codebase is still biased towards it. In the future, it would be great if we find a better way to test and develop multiple build backends.

  • Build backend detection: I had to add explicit build backend detection to fix an issue where setuptools may be in the sys path, but actually hatchling is being used. I added a config option with which the build backend can be set explicitly with [tool.plux] build_backend = "hatchling" if needed, but will automatically pick the one that's set in the build-backend field of build-system . If none of that is set it falls back to auto detection using imports.

Changes

  • Hatchling support for manual entrypoint build mode
  • Build backend can now be set explicitly with [tool.plux] build_backend = "hatchling" if needed, but will automatically pick the one that's set in the build-backend field of build-system

TODO

  • package discovery for hatchling
  • isolated testing mode for hatchling and setuptools
  • explicit build backend selection via config (setuptools seems to be included somehow in 3.10 and 3.11, which always leads to setuptools being selected, see this run)
  • more tests for hatch config scenarios

@thrau thrau force-pushed the hatchling-support branch from 3c47c8d to a7bac7b Compare January 28, 2026 21:07
@thrau thrau marked this pull request as ready for review January 29, 2026 13:36
@thrau thrau requested a review from dfangl as a code owner January 29, 2026 13:36
Comment on lines +196 to +198
if plux_config.bild_backend != BuildBackend.AUTO:
# first, check if the user configured one
return plux_config.bild_backend
Copy link
Member

@dfangl dfangl Jan 29, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
if plux_config.bild_backend != BuildBackend.AUTO:
# first, check if the user configured one
return plux_config.bild_backend
if plux_config.build_backend != BuildBackend.AUTO:
# first, check if the user configured one
return plux_config.build_backend

I will review the rest a bit later, however this seems like an obvious error 😅 It is quite consistent, however it is not instantiated with that name on line 138.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks for spotting, i'll do one more pass and write some tests :)

@thrau thrau requested a review from dfangl January 29, 2026 17:00
@alexrashed alexrashed mentioned this pull request Feb 5, 2026
1 task
Copy link
Member

@dfangl dfangl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is one pretty critical bug (the pycache one), and then there is one discussion I'd like to have about potentially differing logic to setuptools, but other than that I think this should work nicely!

Comment on lines +208 to +221
# if that also fails, just try to import both build backends and return the first one that works
try:
import setuptools # noqa

return BuildBackend.SETUPTOOLS
except ImportError:
pass

try:
import hatchling # noqa

return BuildBackend.HATCHLING
except ImportError:
pass
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would it make sense to LOG a warning here, at least if both can be imported, explaining that we default to setuptools?

everything in the preceding path that's not a package.
"""

DEFAULT_EXCLUDES = "__pycache__"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
DEFAULT_EXCLUDES = "__pycache__"
DEFAULT_EXCLUDES = {"__pycache__"}

The if os.path.basename(rel_path).strip(os.pathsep) in self.DEFAULT_EXCLUDES would otherwise be substring matching, not testing only for pycache. This would mean a package called cache would not be descended into, similar for packages called py or just c.

In setuptools this is a tuple, which is why the check works better 😅

This should be fixed before merging!

Comment on lines +118 to +119
if not self._looks_like_package(root):
continue
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In contrast to setuptools, we do "descend" into directories which do not look like packages here.

Example:

  src/
    parent/                  ( __init__.py)
      gap/                     (no  __init__.py)
        real_pkg/           (__init__.py)
          __init__.py

Should parent.gap.real_pkg be a package here or not?

Of course changing this would require a change in us handling namespace packages (by calling the SimplePackageFinder on each child directory manually and combining the results perhaps, or we need to change the SimplePackageFinder here to only exclude this case for the parent package).

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Example using setuptools:

(.venv) ~/localstack/test-ground/package-hierarchy-test > python
Python 3.13.12 (main, Feb 12 2026, 00:45:41) [Clang 21.1.4 ] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import setuptools
>>> setuptools.find_packages()
['outer_pkg']
>>>
(.venv) ~/localstack/test-ground/package-hierarchy-test > touch outer_pkg/gap/__init__.py
(.venv) ~/localstack/test-ground/package-hierarchy-test > python
Python 3.13.12 (main, Feb 12 2026, 00:45:41) [Clang 21.1.4 ] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import setuptools
>>> setuptools.find_packages()
['outer_pkg', 'outer_pkg.gap', 'outer_pkg.gap.inner_pkg']

I don't think this is a big problem, but something to discuss perhaps? Or is the gap package here somehow an implicit namespace package? (You can import it in python after all?). I couldn't find a good documentation on nesting namespace packages inside regular ones (since it doesn't really make a lot of sense anyway)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

plux will find something like: ["outer_pkg", "outer_pkg.gap.inner_pkg"]

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

great question! i haven't really thought about this deeply. how would you suggest we handle it?

Copy link
Member

@dfangl dfangl Mar 11, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would for now suggest to keep it as is, in our own codebase it should not raise any issues - however we will likely not detect any packages from those nested namespace packages - I don't really see reason to support this for now, however we can in the future.

Copy link
Member

@dfangl dfangl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As mentioned in the PR comments, I think it is fine to leave the package finder as is for now. Should not create problems in the LS codebase (or any other usually).

@dfangl dfangl merged commit 80c79e1 into main Mar 11, 2026
5 checks passed
@dfangl dfangl deleted the hatchling-support branch March 11, 2026 14:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants