A multi-runtime, multi-transport pipeline framework for real-time media.

Build streaming audio/video/text pipelines that run locally with native Rust performance — or offload to remote servers and Docker containers seamlessly. Same manifest, four transports (gRPC, HTTP/SSE, WebRTC, FFI), three runtimes (native Rust, multiprocess Python, WASM).

End-to-end avatar pipeline running entirely on a workstation:

mic → VAD → Whisper STT → Qwen3.6-27B → Kokoro TTS → Character-Creator avatar with lipsync, streamed over WebRTC. All body movements are diffusion-generated on the fly by KimodoMotionNode — the LLM emits a text-prompt tool call ("a person waves the right hand") and Kimodo's motion-diffusion model synthesizes per-frame SMPL skeletal poses live.

Nothing pre-recorded, scripted, or keyframed. See scripts/run-avatar-s2s.sh in the main repo.

• Native Rust audio nodes — VAD, resampling, Whisper STT run 2–16× faster than Python equivalents.
• Zero-copy IPC — process-isolated Python nodes communicate via iceoryx2 shared memory; no serialization, no socket overhead.
• Transport-agnostic manifests — the same pipeline JSON runs over gRPC, HTTP/SSE, WebRTC, or Python FFI without changes.
• Capability resolution — automatic format negotiation (sample rate, channels, dtype) at pipeline construction time, with actionable error messages and auto-inserted converters where possible.
• Loadable plugins — ship a node as a single .so/.dylib/.dll with abi_stable FFI; works across host-vs-plugin dep-version conflicts (e.g. ort=rc.10 plugin against ort=rc.12 host).
• Per-node managed Python venvs — declare deps via PEP 723 + @python_requires; the runtime provisions a content-addressed uv venv per node so transformers<5.0 and transformers>=5.0 nodes coexist in the same pipeline.

# CLI — fastest path
cargo install --git https://github.com/RemoteMedia-SDK/remotemedia-sdk \
              --path examples/cli/remotemedia-cli
remotemedia nodes list                     # 41+ built-in nodes
remotemedia run pipeline.yaml -i audio.wav

# Python
pip install remotemedia
python -c "import remotemedia; print(remotemedia.is_rust_runtime_available())"

# Node.js
npm install @matbee/remotemedia-native

Full reference in remotemedia-sdk's main README.

Start here if you're new:

• SDK developer guides — tutorials for the four common integration tasks:
  • Running a pipeline from Python — 5 entry points + decision tree.
  • Ship a HuggingFace model as a Python node.
  • Ship a HuggingFace ONNX export as a Rust node.
  • Benchmark a node (TTS / STT / S2S / video / data).
  • Smoke-test a node — probes, pytest, manifest validators.

Reference docs (in the main repo):

• Node reference — every built-in node with parameters.
• Custom node registration — all four shipping paths.
• Managed Python environments — PEP 723, @python_requires, content-addressed venv cache.
• Session control & typed RPC.
• Benchmarking.
• Performance tuning.
• Architecture deep-dive.

Apache 2.0 for the open-source runtime + plugins. Commercial license available — see COMMERCIAL-LICENSE.md in the main repo.

Issues, discussions, and pull requests are welcome on each repo. For substantial changes, please open an issue first to discuss approach — most non-trivial work goes through an OpenSpec change proposal.