feat: add RDMA support to MLX backend via mlx-jaccl-cluster integration

[localai-bot]

This PR integrates RDMA support with the MLX backend in LocalAI to enable high-performance distributed inference on Apple Silicon machines.

Summary of Changes

1. Backend Changes (backend/python/mlx/backend.py)

• Added parse_rdma_options() function to extract RDMA configuration from ModelOptions.Options
• Added conditional mx.distributed.init(backend="jaccl") call if RDMA is enabled
• Added model sharding logic using model.shard(mx.distributed.world_size())
• RDMA configuration is passed via options (e.g., mlx_rdma.enabled:true, mlx_rdma.ctrl_host:0.0.0.0, etc.)

2. Core CLI Changes (core/cli/run.go)

• Added MLX_GRPC_SERVERS environment variable support in the TunnelCallback
• When MLX_RDMA_ENABLED=true, the P2P worker IPs/ports are exposed to the backend via MLX_GRPC_SERVERS

3. Integration Pattern (Aligned with llama.cpp)

• Workers are started via local-ai worker mlx_rdma (similar to llama.cpp)
• Workers expose their services via P2P
• Main instance collects worker IPs and sets MLX_GRPC_SERVERS env var for the backend
• Backend reads MLX_GRPC_SERVERS and initializes RDMA if enabled

4. RDMA Options Format

Options are passed as repeated string list in ModelOptions.Options:

[
  "mlx_rdma.enabled:true",
  "mlx_rdma.ctrl_host:192.168.1.100",
  "mlx_rdma.ctrl_port:18080",
  "mlx_rdma.hostfile:/path/to/hosts.json"
]

Usage Example

1. Start workers on each node: local-ai worker mlx_rdma --host <ip>
2. Start main instance with RDMA enabled: local-ai run --mlx_rdma.enabled:true ...
3. Backend automatically initializes RDMA and shards model across workers

[netlify]

✅ Deploy Preview for localai ready!

Name	Link
🔨 Latest commit	06c17eb
🔍 Latest deploy log	https://app.netlify.com/projects/localai/deploys/699b645df080e000089ce1e0
😎 Deploy Preview	https://deploy-preview-8623--localai.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

---

To edit notification comments on pull requests, go to your Netlify project configuration.