feat: allow extending hyperlight-js-runtime with custom native modules#49
Open
simongdavies wants to merge 3 commits intohyperlight-dev:mainfrom
Open
feat: allow extending hyperlight-js-runtime with custom native modules#49simongdavies wants to merge 3 commits intohyperlight-dev:mainfrom
simongdavies wants to merge 3 commits intohyperlight-dev:mainfrom
Conversation
acaa72b to
d721317
Compare
Resolves hyperlight-dev#48 Add a registration-based system for extending the JS runtime with custom native (Rust-implemented) modules that run inside the Hyperlight guest VM. Key changes: - hyperlight-js-runtime/Cargo.toml: Add [lib] target so the runtime can be used as a library dependency by extender crates. - modules/mod.rs: Add global CUSTOM_MODULES registry with register_native_module() and builtin_module_names(). NativeModuleLoader checks custom registry first, falls back to built-in modules. Panics if custom module name conflicts with a built-in. - native_modules! macro: Generates init_native_modules() (#[no_mangle]) that registers custom modules. Called automatically via spin::Once on first NativeModuleLoader access — no explicit init needed. - guest/mod.rs: Move hyperlight guest entry point (hyperlight_main, guest_dispatch_function, Host impl, stubs) from main/hyperlight.rs into the lib behind cfg(hyperlight). Extender binaries get all guest infrastructure for free by depending on the lib. - hyperlight-js/build.rs: Add HYPERLIGHT_JS_RUNTIME_PATH env var override to embed custom runtime binaries instead of the default. - host.rs: Restore original Host trait only (no FsHost extraction). Testing: - 13 unit/integration tests in hyperlight-js-runtime (loader, registry, macro, override prevention, E2E with native CLI fixture binary) - 3 Hyperlight VM integration tests in hyperlight-js (#[ignore], run via just test-native-modules) - Extended runtime fixture crate with shared native_math module - just test-native-modules recipe for full hyperlight pipeline Docs: - docs/extending-runtime.md with quick start, host-side usage, native testing, API reference, and architecture diagram
d721317 to
3e43831
Compare
Add custom_globals! macro alongside native_modules! to allow extender
crates to register global JS objects (TextEncoder, TextDecoder,
polyfills, constants) without modifying hyperlight-js-runtime.
- New custom_globals! macro in modules/mod.rs (same #[no_mangle] + extern
- setup_custom_globals() bridge called in JsRuntime::new() after
built-in globals
- Default empty custom_globals! {} in base runtime main.rs
- Test fixture with globalThis.CUSTOM_GLOBAL_TEST = 42
- 6 new tests: unit e2e + full pipeline (standalone, coexist with
builtins, combined with native modules)
- Documentation in docs/extending-runtime.md
- Allow overriding built-in modules (io, crypto, console) via native_modules! — custom modules take priority over built-ins. The require module is protected and cannot be overridden. - Make console and print globals writable during init so custom_globals! can extend them (e.g. add console.warn/error). - Freeze console (Object.freeze) and print (non-writable) after custom_globals! runs — handler code cannot tamper with them. - 3-step init in JsRuntime::new: setup → custom_globals → freeze. - New globals/freeze.rs module for post-init lockdown. - Tests: require override rejection, console extension via custom_globals, freeze verification, console.log after freeze. - Updated docs/extending-runtime.md with override rules. Signed-off-by: Simon Davies <simongdavies@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Resolves #48
Overview
Adds an extension system that lets downstream crates add custom native modules, custom globals, and override built-in modules — without forking the runtime.
Architecture
Three macros, one init sequence:
native_modules!— Custom native modulesRegister Rust-implemented modules that guest JS can
import:io,crypto,consolecan be overriddenrequirecannot be overridden (panics — it's core infrastructure)#[no_mangle]+extern "Rust"linkagecustom_globals!— Custom global objectsRegister global objects (constructors, polyfills, constants) without import:
console(addwarn,error,info,debug)#[rquickjs::class]) and JS polyfills (ctx.eval()) supportedGlobals freeze
After
custom_globals!runs, built-in globals are locked down:console→Object.freeze()(no new properties, no modifications)print→ non-writable, non-configurablerequire→ already non-configurable from setup viaProperty::from()Handler code cannot tamper with any of these.
Runtime as a library
hyperlight-js-runtimenow exposes a[lib]target. Extender crates depend on it and provide a binary that links everything together. TheHYPERLIGHT_JS_RUNTIME_PATHbuild-time env var tellshyperlight-jsto embed the custom binary.Guest infrastructure (entry point, host function dispatch, libc stubs) moved from
src/main/tosrc/guest/and is provided by the lib — no copying needed.Key changes
src/hyperlight-js-runtime/src/modules/mod.rsNativeModuleLoader,register_native_module,native_modules!,custom_globals!,setup_custom_globals()src/hyperlight-js-runtime/src/lib.rssetup→custom_globals→freezesrc/hyperlight-js-runtime/src/globals/console.rssrc/hyperlight-js-runtime/src/globals/print.rssrc/hyperlight-js-runtime/src/globals/freeze.rssrc/hyperlight-js-runtime/src/globals/mod.rsfreeze()public functionsrc/hyperlight-js-runtime/src/main.rsnative_modules! {}+custom_globals! {}src/hyperlight-js-runtime/src/guest/src/main/hyperlight.rs— lib-provided guest infrasrc/hyperlight-js/build.rsHYPERLIGHT_JS_RUNTIME_PATHsupportdocs/extending-runtime.mdTests
tests/native_modules.rs: loader resolution, custom modules, built-in override,requireprotection,custom_globals!macro, console extensions, freeze behavioursrc/hyperlight-js/tests/native_modules.rs:HYPERLIGHT_JS_RUNTIME_PATHembeddingnative_math(shared lib) +extended_runtime(binary with custom modules + globals)