Support Multi-Back for Demo Scripts (Part 2) and Unify Code Structure by YizeWang · Pull Request #6140 · isaac-sim/IsaacLab

YizeWang · 2026-06-11T09:01:46Z

Description

This PR updates several demo scripts to support multi-backends and unify the code structure of all demo scripts.

No new required dependencies.

Type of change

New feature (non-breaking change which adds functionality)
Bug fix (non-breaking change which fixes an issue)
Documentation update

Screenshots

Checklist

I have read and understood the contribution guidelines
I have run the pre-commit checks with ./isaaclab.sh -f
I have made corresponding changes to the documentation
My changes generate no new warnings
I have added tests that prove my fix is effective or that my feature works
I have updated the changelog
I have added my name to CONTRIBUTORS.md or my name already exists there

Signed-off-by: Yize Wang <yizew@nvidia.com>

greptile-apps · 2026-06-11T09:07:22Z

Greptile Summary

This PR unifies the structure of demo scripts to use launch_simulation / add_launcher_args instead of the older AppLauncher pattern, and extends multi_asset.py and haply_teleoperation.py with Newton (MJWarp) physics backend support.

Demo scripts refactored: All 14 scripts are unified to use launch_simulation, TYPE_CHECKING guards for heavy imports, and while sim.is_headless_or_exist_active_visualizer() loop pattern; stale inline comments are removed.
multi_asset.py extended: Adds --physics newton_mjwarp support, guards against None color spec, updates object collection prim paths under an ObjectCollection/ sub-path, and replaces ConeCfg with CylinderCfg.
Library changes: VisualizationMarkersCfg gains a class_type field; terrains/utils.py changes the default color map from \"turbo\" to \"inferno\".

Confidence Score: 4/5

The PR is mostly safe to merge; the structural cleanup across all demo scripts is straightforward, with one robot reset block in multi_asset.py that dropped .clone() inconsistently and may silently corrupt default tensor buffers on repeated resets.

The structural unification is clean and consistent across all 14 scripts. The .clone() removal for robot joint/velocity tensors in multi_asset.py is a concrete defect that could produce incorrect simulation state on reset without any error message. Everything else is cosmetic.

scripts/demos/multi_asset.py — specifically the robot reset block where .clone() was dropped for default_root_vel and default_joint_pos/vel tensors.

Important Files Changed

Filename	Overview
scripts/demos/multi_asset.py	Adds newton_mjwarp support, null-checks color_spec, fixes prim paths; contains unusual double-parenthesis syntax and inconsistent .clone() usage compared to rigid_object reset
source/isaaclab/isaaclab/terrains/utils.py	Default colormap changed from turbo to inferno but the module docstring still refers to the Turbo colormap
source/isaaclab/isaaclab/markers/visualization_markers_cfg.py	Adds class_type field and TYPE_CHECKING guard; enables the cfg.class_type(cfg) factory pattern
scripts/demos/procedural_terrain.py	Migrated to launch_simulation; --physics arg added with only physx as valid choice (effectively dead argument)
scripts/demos/haply_teleoperation.py	Migrated to launch_simulation; type annotation on robot fixed from Articulation to ArticulationCfg
scripts/demos/bin_packing.py	isaaclab_newton import repositioned with isort:skip; return type annotation removed from main()
scripts/demos/arms.py	Minor docstring formatting and typo fixes only
scripts/demos/bipeds.py	Docstring formatting unified; no functional change
scripts/demos/hands.py	Docstring and typo fix; no functional change

Comments Outside Diff (2)

scripts/demos/multi_asset.py, line 118-122 (link)

Redundant parentheses around sim_utils.CylinderCfg — (ClassName)(args) is syntactically valid Python but calls the class in an unusual two-step way. It's equivalent to plain sim_utils.CylinderCfg(...) and is likely a typo left over from removing ConeCfg.

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!
source/isaaclab/isaaclab/terrains/utils.py, line 22-24 (link)

The default colormap was changed from "turbo" to "inferno", but the function's docstring still says "using the Turbo colormap", which is now incorrect.

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

_{Reviews (1): Last reviewed commit: "Clean Up multi_asset.py" | Re-trigger Greptile}

greptile-apps · 2026-06-11T09:07:27Z

+            root_vel = robot.data.default_root_vel.torch
            robot.write_root_velocity_to_sim_index(root_velocity=root_vel)
            # -- joint state
-            joint_pos, joint_vel = (
-                robot.data.default_joint_pos.torch.clone(),
-                robot.data.default_joint_vel.torch.clone(),
-            )
+            joint_pos = robot.data.default_joint_pos.torch
+            joint_vel = robot.data.default_joint_vel.torch


.clone() is omitted for root_vel, joint_pos, and joint_vel when resetting the robot, but the immediately preceding root_pose reset still uses .clone(). The same applies to rigid_object reset at line 251 (.clone() is preserved there). If write_*_to_sim_index modifies the tensor in-place, skipping .clone() would corrupt the default_root_vel / default_joint_* buffers and produce wrong values on subsequent resets.

Suggested change

root_vel = robot.data.default_root_vel.torch

robot.write_root_velocity_to_sim_index(root_velocity=root_vel)

# -- joint state

joint_pos, joint_vel = (

robot.data.default_joint_pos.torch.clone(),

robot.data.default_joint_vel.torch.clone(),

)

joint_pos = robot.data.default_joint_pos.torch

joint_vel = robot.data.default_joint_vel.torch

root_vel = robot.data.default_root_vel.torch.clone()

robot.write_root_velocity_to_sim_index(root_velocity=root_vel)

# -- joint state

joint_pos = robot.data.default_joint_pos.torch.clone()

joint_vel = robot.data.default_joint_vel.torch.clone()

root_vel joint_pos joint_vel are not modified. No need to clone these variables. root_pose is cloned because of root_pose[:, :3] += scene.env_origins.

YizeWang · 2026-06-11T09:13:57Z

+    description="View ARL Robot 1 with Lee Position Controller.",
+    conflict_handler="resolve",
+)
+parser.add_argument("--physics", default="physx", choices=["physx"], help="Physics backend.")


The Newton backend is not supported because the Multirotor class is directly tied to the physx backend. Filed Proposal#6141: Add Newton Backend Support for Multirotor Asset.

I think this is Ok for now. We can look into drones later.

YizeWang · 2026-06-11T09:19:44Z

+)
+parser.add_argument("--physics", default="physx", choices=["physx", "newton_mjwarp"], help="Physics backend.")
+# Newton visualizer not supported for deformables
+parser.add_argument("--visualizer", default="kit", choices=["kit"], help="Visualizer backend.")


The Newton visualizer is not supported because the Newton shadow-model fails to import PhysX-authored deformable USD. Filed Proposal#6142: Support Kit-less Deformable USD.

@mmichelis an you take a look? :)

YizeWang · 2026-06-11T09:22:29Z

+            sim_device=args_cli.device,
+            limit_force=2.0,
+        )
+        haply_device = haply_cfg.class_type(cfg=haply_cfg)


This script has NOT been validated, because HaplyDevice requires peripheral devices of Inverse3 and VerseGrip, which are not available on my device. If needed, I can coordinate with Nic Ma to perform a validation.

YizeWang · 2026-06-11T09:24:10Z

+    description="This script demonstrates different types of markers.",
+    conflict_handler="resolve",
+)
+parser.add_argument("--physics", default="physx", choices=["physx"], help="Physics backend.")


The Newton backend is not supported. Filed Proposal#6143: Support General USD Models in Netwon. We will leave this script kit-and-physx-only as discussed in https://nvidia.slack.com/archives/C0AV4B102HG/p1781038183961149?thread_ts=1781013526.172589&cid=C0AV4B102HG.

YizeWang · 2026-06-11T09:25:04Z


 def randomize_shape_color(prim_path_expr: str):
    """Randomize the color of the geometry."""
+    from pxr import Gf, Sdf


Discussion with @ooctipus in progress. We want to avoid all local imports.

Signed-off-by: Yize Wang <yizew@nvidia.com>

YizeWang · 2026-06-11T12:26:54Z

 import carb
 import omni
 from omni.kit.viewport.utility import get_viewport_from_window_name
 from omni.kit.viewport.utility.camera_state import ViewportCameraState
 from pxr import Gf, Sdf


In discussion with @ooctipus on how to avoid local imports for carb omni pxr.

Signed-off-by: Yize Wang <yizew@nvidia.com>

YizeWang · 2026-06-11T12:56:01Z

-# append AppLauncher cli args
-AppLauncher.add_app_launcher_args(parser)
-# demos should open Kit visualizer by default
+parser.add_argument("--physics", default="physx", choices=["physx"], help="Physics backend.")


The Newton physics backend is not support because MJWarp requires at least 1 joint while this scene only contains a ground. Filed Proposal#6144: Support Newton MJWarp for No-joint Scenes.

YizeWang · 2026-06-11T13:08:53Z

+        if args_cli.physics == "newton_mjwarp":
+            # Newton/MJWarp requires a uniform layout, not supporting different setups across environments.
+            scene_cfg.object.spawn.assets_cfg = scene_cfg.object.spawn.assets_cfg[1:2]
+            scene_cfg.robot.spawn.usd_path = scene_cfg.robot.spawn.usd_path[0]


The newton backend does not support heterogeneous spawning. Filed Proposal#6162: Support Mixed Spawns with Newton-MJWarp.

YizeWang · 2026-06-11T13:47:23Z

        # Get the color for each vertex based on the height
-        color_map = kwargs.pop("color_map", "turbo")
+        # Valid options are ["viridis", "inferno", "plasma", "magma"]
+        color_map = kwargs.pop("color_map", "inferno")


Rationale: "turbo" is not available anymore.

YizeWang · 2026-06-11T13:48:12Z

        spawn=sim_utils.MultiAssetSpawnerCfg(
            assets_cfg=[
-                sim_utils.ConeCfg(
+                (sim_utils.CylinderCfg)(


Rational: I replaced ConeCfg with CylinderCfg because the former is not supported in Newton.

Maybe it should be. Can you open a discussion with Newton?

Signed-off-by: Yize Wang <yizew@nvidia.com>

YizeWang · 2026-06-11T14:21:26Z

@@ -111,7 +115,7 @@ class MultiObjectSceneCfg(InteractiveSceneCfg):
        prim_path="/World/envs/env_.*/Object",


@AntoineRichard The newton backend requires this to be "/World/envs/env_.*/ObjectCollection/Object_A".

[Update] @AntoineRichard is working on cleaning up this. Filed Proposal#6147: Fix Newton Object Collection Path Matching.

AntoineRichard

A high level comment, I'm wondering if we could add tests to validate that most the examples are working as expected. This would be great to maintain things in the future.

AntoineRichard · 2026-06-11T14:25:58Z

        spawn=sim_utils.MultiAssetSpawnerCfg(
            assets_cfg=[
-                sim_utils.ConeCfg(
+                (sim_utils.CylinderCfg)(


Maybe it should be. Can you open a discussion with Newton?

AntoineRichard · 2026-06-12T09:51:12Z

+    description="View ARL Robot 1 with Lee Position Controller.",
+    conflict_handler="resolve",
+)
+parser.add_argument("--physics", default="physx", choices=["physx"], help="Physics backend.")


I think this is Ok for now. We can look into drones later.

AntoineRichard · 2026-06-12T09:52:39Z

+)
+parser.add_argument("--physics", default="physx", choices=["physx", "newton_mjwarp"], help="Physics backend.")
+# Newton visualizer not supported for deformables
+parser.add_argument("--visualizer", default="kit", choices=["kit"], help="Visualizer backend.")


@mmichelis an you take a look? :)

AntoineRichard · 2026-06-12T09:53:26Z

It also looks like the deformable look a lot better with newton? At least in the video @YizeWang shared

Yize Wang added 16 commits June 11, 2026 16:55

Adapt deformables.py

0dda75a

Signed-off-by: Yize Wang <yizew@nvidia.com>

Adapt markers.py

925b00e

Adapt procedural_terrain.py

41131e6

Adapt arl_robot_1.py

293a188

Signed-off-by: Yize Wang <yizew@nvidia.com>

Remove Options For deformables.py

e7821c1

Signed-off-by: Yize Wang <yizew@nvidia.com>

Remove Launch Options for arl_robot_1.py

b206dc4

Signed-off-by: Yize Wang <yizew@nvidia.com>

Remove Invalid Options

af8170b

Signed-off-by: Yize Wang <yizew@nvidia.com>

Limit Launch Option for markers.py

d0607d8

Signed-off-by: Yize Wang <yizew@nvidia.com>

Adapt multi_asset.py

9efc566

Signed-off-by: Yize Wang <yizew@nvidia.com>

Adapt haply_teleoperation.py

06630da

Signed-off-by: Yize Wang <yizew@nvidia.com>

Limit Visualizer Options

81a5445

Signed-off-by: Yize Wang <yizew@nvidia.com>

Minor Changes

3e1dd04

Signed-off-by: Yize Wang <yizew@nvidia.com>

Remove Cone for multi_asset.py

e80df72

Signed-off-by: Yize Wang <yizew@nvidia.com>

Unify Demo Scripts

e0a0bca

Signed-off-by: Yize Wang <yizew@nvidia.com>

Minor Change

1f89b38

Signed-off-by: Yize Wang <yizew@nvidia.com>

Clean Up multi_asset.py

00e6751

Signed-off-by: Yize Wang <yizew@nvidia.com>

YizeWang requested review from Mayankm96 and ooctipus as code owners June 11, 2026 09:01

github-actions Bot added the isaac-lab Related to Isaac Lab team label Jun 11, 2026

greptile-apps Bot reviewed Jun 11, 2026

View reviewed changes

YizeWang commented Jun 11, 2026

View reviewed changes

Comments

681d84f

Signed-off-by: Yize Wang <yizew@nvidia.com>

YizeWang mentioned this pull request Jun 11, 2026

[Proposal] Add Newton Backend Support for Multirotor Asset #6141

Open

2 tasks

YizeWang commented Jun 11, 2026

View reviewed changes

YizeWang mentioned this pull request Jun 11, 2026

[Proposal] Support Kit-less Deformable USD #6142

Open

4 tasks

Robot Fix in haply_teleoperation.py

9c97693

Signed-off-by: Yize Wang <yizew@nvidia.com>

YizeWang mentioned this pull request Jun 11, 2026

[Proposal] Support Newton MJWarp for No-joint Scenes #6144

Open

3 tasks

YizeWang commented Jun 11, 2026

View reviewed changes

Modify multi_asset.py

cba2084

Signed-off-by: Yize Wang <yizew@nvidia.com>

YizeWang commented Jun 11, 2026

View reviewed changes

This was referenced Jun 11, 2026

[Proposal] General USD Model to Newton Mesh API #6143

Open

[Proposal] Fix Newton Object Collection Path Matching #6147

Open

[Proposal] Support Mixed Spawns with Newton-MJWarp #6162

Open

AntoineRichard reviewed Jun 12, 2026

View reviewed changes

		@@ -111,7 +115,7 @@ class MultiObjectSceneCfg(InteractiveSceneCfg):
		prim_path="/World/envs/env_.*/Object",

Conversation

YizeWang commented Jun 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Type of change

Screenshots

Checklist

Uh oh!

greptile-apps Bot commented Jun 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Greptile Summary

Confidence Score: 4/5

Important Files Changed

Comments Outside Diff (2)

Uh oh!

greptile-apps Bot Jun 11, 2026

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

YizeWang Jun 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

YizeWang Jun 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

YizeWang Jun 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

YizeWang Jun 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

YizeWang Jun 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

YizeWang Jun 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

YizeWang Jun 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

YizeWang Jun 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

YizeWang Jun 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

YizeWang Jun 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

AntoineRichard left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

YizeWang commented Jun 11, 2026 •

edited

Loading

greptile-apps Bot commented Jun 11, 2026 •

edited

Loading

YizeWang Jun 11, 2026 •

edited

Loading

YizeWang Jun 11, 2026 •

edited

Loading

YizeWang Jun 11, 2026 •

edited

Loading

YizeWang Jun 11, 2026 •

edited

Loading

YizeWang Jun 11, 2026 •

edited

Loading

YizeWang Jun 11, 2026 •

edited

Loading

YizeWang Jun 11, 2026 •

edited

Loading

YizeWang Jun 11, 2026 •

edited

Loading

YizeWang Jun 11, 2026 •

edited

Loading

YizeWang Jun 11, 2026 •

edited

Loading