Skip to content

fix: Invert input/output context if a Python function is called from C++#6055

Open
ximion wants to merge 1 commit intopybind:masterfrom
ximion:wip/callable-typing
Open

fix: Invert input/output context if a Python function is called from C++#6055
ximion wants to merge 1 commit intopybind:masterfrom
ximion:wip/callable-typing

Conversation

@ximion
Copy link
Copy Markdown

@ximion ximion commented May 5, 2026

Hi!

Description

In my project, I have a type ByteVector with a type-caster defined like this:
PYBIND11_TYPE_CASTER(ByteVector, io_name("bytes | bytearray", "bytes"));
So, we accept any bytes or bytearray to turn into a ByteVector, but when passing it to Python, it's always bytes (because Python can't modify the vector, which is alright in this case).

API users pass in Python functions as callbacks that are called on specific events that the C++ interface processes. This currently leads to type annotations that look like this:

@property
def on_save_settings(self) -> collections.abc.Callable[[os.PathLike | str | bytes], bytes]:
    ...
@on_save_settings.setter
def on_save_settings(self, arg1: collections.abc.Callable[[os.PathLike | str | bytes], bytes]) -> None:
    ...

The function return type is only bytes even though it is input to C++ and therefore should accept both byte-like types. Similar things apply for the path.

Currently, pybind11 expects Python to call C++ functions when receiving a callback, not the opposite.
This patch fixes the type annotations by inverting them if Python passes in a function into C++, and leaving them the same in the other direction. This leads to type annotations that look like this:

@property
def on_save_settings(self) -> collections.abc.Callable[[os.PathLike | str | bytes], bytes]:
    ...
@on_save_settings.setter
def on_save_settings(self, arg1: collections.abc.Callable[[pathlib.Path], bytes | bytearray]) -> None:
    ...

Now, if you pass in a function Python -> C++, its i/o context is inverted, as it will be called from C++. However, in the other direction (receiving one from C++) everything stays the same as before.

I think this is probably the best we can do to address both cases automatically in pybind11, or do you have a better solution to this problem?

Thank you for making pybind11, it's an incredibly helpful project!

Suggested changelog entry:

  • Invert the input/output context if a Python function is called from C++

@ximion ximion changed the title Invert the input/output context if a Python function is called from C++ fix: Invert input/output context if a Python function is called from C++ May 5, 2026
@ximion ximion force-pushed the wip/callable-typing branch from e867f87 to dccf0e5 Compare May 5, 2026 22:21
@ximion ximion force-pushed the wip/callable-typing branch from dccf0e5 to 79afc8b Compare May 5, 2026 23:56
When the Callable itself is an input (parameter) to a C++ function, its
arguments are outputs (C++ passes them to the Python callback), and vice
versa.
Therefore, we must invert them if C++ calls a Python function, but keep
them the same in the other direction.
@ximion ximion force-pushed the wip/callable-typing branch from 6e2a7f8 to 3c2fd06 Compare May 5, 2026 23:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant