Skip to content

feat(python): expose Bash class with Monty Python execution and external function handler#760

Merged
chaliy merged 4 commits intoeverruns:mainfrom
shubhlohiya:feat/python-external-handler-bindings
Mar 19, 2026
Merged

feat(python): expose Bash class with Monty Python execution and external function handler#760
chaliy merged 4 commits intoeverruns:mainfrom
shubhlohiya:feat/python-external-handler-bindings

Conversation

@shubhlohiya
Copy link
Contributor

@shubhlohiya shubhlohiya commented Mar 18, 2026

Summary

The crates/bashkit-python PyO3 bindings expose BashTool and ScriptedTool but not the
Bash builder API. The Rust python feature (Monty embedded interpreter) and
python_with_external_handler are available at the Rust level but unreachable from Python.

This PR adds a Bash Python class that exposes both, making it possible to:

  1. Run Python code via python3/python builtins (Monty embedded interpreter)
  2. Register named external functions that Python code can call as ordinary functions, dispatched
    to an async Python handler

Usage

from bashkit import Bash

# Plain bash (unchanged behaviour)
bash = Bash()
result = await bash.execute("echo hello")

# Bash + embedded Python execution
bash = Bash(python=True)
result = await bash.execute("python3 -c 'x = 6 * 7; print(x)'")
# result.stdout == "42"

# Bash + Python + external function handler
async def handler(fn_name: str, args: list, kwargs: dict) -> Any:
    if fn_name == "fetch":
        return {"status": "ok", "value": 42}
    raise ValueError(f"Unknown function: {fn_name}")

bash = Bash(
    python=True,
    external_functions=["fetch"],
    external_handler=handler,
)
result = await bash.execute(
    "python3 -c \"d = fetch(); print(d['status'], d['value'])\""
)
# result.stdout == "ok 42"

Changes

  • Cargo.toml — enable python feature; add num-bigint dep for BigInt round-trip
  • src/lib.rs:
    • Extend PyBash constructor with three new optional params: python: bool,
      external_functions: list[str] | None, external_handler: AsyncCallable | None
    • Extract make_external_handler and apply_python_config helpers shared between
      new() and reset()
    • Fix reset() to preserve python mode and external handler config (previously dropped)
    • Raise ValueError when external_functions is supplied without external_handler
    • Add monty_to_py / py_to_monty conversion helpers covering all MontyObject variants,
      including correct BigInt ↔ Python int round-trip and PyTuple support
  • _bashkit.pyi — update stub with new constructor params, ExternalHandler Protocol
    for precise type checking, and cancel() method
  • tests/test_bashkit.py — 19 new tests; all 336 tests pass

All existing BashTool and ScriptedTool behaviour is unchanged.


🤖 Generated with Claude Code

@shubhlohiya
Copy link
Contributor Author

Hey @chaliy , I've been using bashkit in a Python agent project and needed to expose the Monty Python execution and external function handler from the Rust side to Python. I made the changes on my fork and they're working well for my use case. I wanted to check if you'd be open to merging this back upstream? Happy to iterate on the implementation if anything needs adjusting.

@shubhlohiya shubhlohiya force-pushed the feat/python-external-handler-bindings branch 2 times, most recently from f65ff8b to 82f10ce Compare March 18, 2026 19:45
…nal function handler

Extends the PyO3 bindings to support Monty Python execution and
programmatic tool calling from within Bash scripts:

- Add `python=True` param to `Bash.__init__` — enables `python`/`python3`
  builtins (Monty embedded interpreter)
- Add `external_functions` + `external_handler` params — wires
  `python_with_external_handler` so async Python callbacks are invoked
  when Monty code calls registered external functions
- Validate: `external_functions` requires handler; handler requires
  `python=True`; handler must be callable; handler must be async
  (checked via `inspect.iscoroutinefunction`)
- Raise `RuntimeError` in `execute_sync` when `external_handler` is set —
  async handlers require a running event loop; use `execute()` instead
- Fix `reset()` to preserve python mode and external handler config
- Extract `make_external_handler` + `apply_python_config` helpers shared
  between `new()` and `reset()`
- Fix `py_to_monty`: isinstance guards for float/bytes; PyTuple, PyInt+BigInt,
  PySet, PyFrozenSet branches; BigInt parse errors propagated
- Fix `monty_to_py`: BigInt → Python int; Tuple → PyTuple; NamedTuple →
  dict with field names; document Set/FrozenSet → list as known limitation;
  Path documented as known limitation (becomes str)
- Pin `num-bigint = "=0.4.6"` to match bashkit core (verified in Cargo.lock)
- Add `ExternalHandler` Protocol with `async def __call__` to `_bashkit.pyi`
- 19 tests added; all 336 tests pass

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
@shubhlohiya shubhlohiya force-pushed the feat/python-external-handler-bindings branch from 82f10ce to 50f2e68 Compare March 18, 2026 20:13
@chaliy chaliy requested a review from Copilot March 18, 2026 20:32
@chaliy
Copy link
Contributor

chaliy commented Mar 18, 2026

@shubhlohiya yep, sounds totally missed feature. I will look at it in next couple of hous and if all good, will merge it. Thank you!

Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR extends the crates/bashkit-python PyO3 bindings so Python users can configure the core Bash interpreter to enable Monty embedded Python execution (python=True) and optionally register external function names that dispatch to an async Python handler.

Changes:

  • Extend Bash (PyBash) constructor/reset to support python, external_functions, and external_handler.
  • Add MontyObject ⇄ Python conversion helpers, including BigInt round-tripping.
  • Update Python stubs and add tests covering Python mode + external handler behavior.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 5 comments.

File Description
crates/bashkit-python/src/lib.rs Adds Python/external-handler configuration and conversion glue for Monty interop.
crates/bashkit-python/tests/test_bashkit.py Adds tests for python execution, external handler dispatch, reset behavior, and BigInt conversion.
crates/bashkit-python/bashkit/_bashkit.pyi Updates type stubs for new Bash constructor options, ExternalHandler Protocol, and cancel().
crates/bashkit-python/Cargo.toml Enables bashkit’s python feature and adds num-bigint dependency.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

You can also share your feedback on Copilot code review. Take the survey.

Comment on lines 509 to 512
builder = builder.limits(limits);
builder = apply_python_config(builder, python, external_functions, handler_clone);
*bash = builder.build();
Ok(())
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed — reset() now clears the shared cancellation flag after rebuilding the interpreter so cancel() does not erroneously abort post-reset executions.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rechecked on the current head (36dd822) and this is still reproducible. Clearing the old flag avoids an immediate abort, but cancel() is still wired to the pre-reset token rather than the rebuilt Bash instance. Minimal repro on the updated extension:

bash = Bash(python=True)
bash.reset()
bash.cancel()
result = await bash.execute("echo hi")

After reset(), that returns exit_code == 0 with stdout == "hi\n"; without reset(), the same pre-cancelled execute returns execution cancelled. So the binding still needs to refresh self.cancelled (or otherwise route cancel() to the current inner interpreter token), and the new test only checks that cancel() does not raise.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed properly in 9617a09 — the cancellation token is now wrapped in Arc<RwLock<Arc<AtomicBool>>> so reset() swaps it to the new interpreter's token and cancel() always reads the current one. Applies to both Bash and BashTool.

Verified with your repro:

bash = Bash(python=True)
bash.reset()
bash.cancel()
result = await bash.execute("echo hi")
# exit_code=1, stderr="execution cancelled"  ✓

Also added tests for both directions (cancel-then-reset clears, reset-then-cancel targets new interpreter). 342 tests pass.

serde_json = { workspace = true }

# Big-integer support for py_to_monty BigInt extraction
num-bigint = "=0.4.6" # must match the version pulled in by bashkit core (see Cargo.lock)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fair point — relaxed to ^0.4.6 to allow compatible patch bumps.

Comment on lines 48 to 51
async def execute(self, commands: str) -> ExecResult: ...
def execute_sync(self, commands: str) -> ExecResult: ...
def cancel(self) -> None: ...
def reset(self) -> None: ...
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added — cancel() was already implemented on BashTool in Rust, just missing from the stub. Drive-by fix.

[dependencies]
# Bashkit core
bashkit = { path = "../bashkit", features = ["scripted_tool"] }
bashkit = { path = "../bashkit", features = ["scripted_tool", "python"] }
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The built wheel is ~3.5 MB with the python feature enabled. I think the size/build-time impact is small enough that it is not worth the added complexity of a conditional Cargo feature, but happy to revisit if the maintainer feels differently.

.iter()
.map(|v| monty_to_py(py, v))
.collect::<PyResult<Vec<_>>>()?;
Ok(PyList::new(py, &py_items)?.into_any().unbind())
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This bridge rewrites both MontyObject::Set and MontyObject::FrozenSet to PyList, so a Python external_handler sees a different type than the Monty caller passed. I reproduced this on the built extension with echo_type({1, 2}): the handler received list, not set. That silently changes semantics for supported Monty container types instead of preserving them or failing explicitly when an element is unhashable.

Copy link
Contributor Author

@shubhlohiya shubhlohiya Mar 18, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed — monty_to_py now produces PySet/PyFrozenSet directly and propagates the error if items are unhashable, rather than silently falling back to a list.

- Broaden iscoroutinefunction check to also inspect __call__ so objects
  with async def __call__ (matching ExternalHandler Protocol) are accepted
- Reset cancelled flag after reset() so cancel() doesn't erroneously
  abort post-reset executions
- Relax num-bigint pin from =0.4.6 to ^0.4.6 to avoid resolution failures
- Add cancel() stub to BashTool in _bashkit.pyi (drive-by fix)
- Convert Set/FrozenSet to PySet/PyFrozenSet in monty_to_py with fallback
  to PyList when items are unhashable
- Convert NamedTuple to dict (mapping field_names to values) instead of list
- Document Path -> str as known limitation in monty_to_py
- Add tests: async callable object validation, set type preservation,
  cancel after reset, sync handler rejection, BigInt value equality
- All 339 tests pass

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
@shubhlohiya shubhlohiya force-pushed the feat/python-external-handler-bindings branch from 9cd6538 to 7ee7412 Compare March 18, 2026 21:01
@shubhlohiya shubhlohiya requested a review from chaliy March 18, 2026 21:10
The set test was incorrectly accepting list as a valid fallback. With
the Set/FrozenSet -> PySet/PyFrozenSet fix in monty_to_py, Monty set
literals now correctly arrive as Python set in the handler. Updated
the test to assert isinstance(set) strictly and verify exact value.

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
@chaliy
Copy link
Contributor

chaliy commented Mar 18, 2026

One more issue on the current head (36dd822): BashTool still has the same reset/cancel token bug as Bash, but that code path is outside the changed hunks so I could not attach it inline.

Repro against the updated extension:

tool = BashTool()
tool.reset()
tool.cancel()
result = await tool.execute("echo hi")

That still succeeds with exit_code == 0 / stdout == "hi\n", so BashTool.reset() is rebuilding the inner Bash without refreshing self.cancelled to the new interpreter token. Since this PR also adds cancel() to the BashTool stub, it would be good to fix that reset/cancel path too and add a regression test for pre-cancel after reset.

…rpreter

Both Bash and BashTool stored the cancellation token as a plain
Arc<AtomicBool> obtained at construction time. After reset() rebuilt
the inner interpreter, cancel() still wrote to the stale token —
the new interpreter never saw it.

Fix: wrap the token in Arc<RwLock<Arc<AtomicBool>>> so reset() can
swap it to the new interpreter's token and cancel() always reads the
current one. Applies to both Bash and BashTool.

Tests cover both directions:
- cancel() then reset() clears cancellation (execution succeeds)
- reset() then cancel() targets the new interpreter (execution cancelled)

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
@shubhlohiya shubhlohiya force-pushed the feat/python-external-handler-bindings branch from a7a55a8 to 9617a09 Compare March 18, 2026 23:33
@shubhlohiya
Copy link
Contributor Author

@chaliy , addressed in 9617a09

@chaliy chaliy merged commit 8bb0c37 into everruns:main Mar 19, 2026
13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants