[BugFix] PolicyVersion: int64 dtype + preserve tensordict device

[vmoens]

Stack from ghstack (oldest at bottom):

• [Doc] Migrate shifted=True callers to legacy/compact + docstring polish #3776
• [Example] Expose compact GAE cat dimension #3775
• [Example] Add Isaac RNN PPO rollout mode flags #3769
• [Example] Isaac Lab RNN PPO with compact memory + knowledge-base notes #3763
• [BugFix] Fix GAE compact path bias on recurrent value nets at internal truncations #3771
• [Refactor] Simplify LSTM/GRUModule recurrent-mode shape normalization #3768
• [Refactor] Keep [B, T] dim in value estimators #3767
• [Feature] Collector.fake_tensordict() / MultiCollector.fake_tensordict() #3764
• [Feature] MultiCollector: track policy version per fresh continue command #3759
• [BugFix] GAE shifted=True: tolerate missing next obs, non-canonical strides, docs #3757
• [Feature] timeit.mark_start/mark_end for non-context-manager timing #3762
• -> [BugFix] PolicyVersion: int64 dtype + preserve tensordict device #3755
• [Performance] LSTM/GRU scan: canonical strides + cuDNN flat-storage clones + thread-local recurrent mode #3754
• [BugFix] Recurrent policy auto-register with policy_factory #3753

The integer branch was casting the version through float() and
calling torch.full(shape, value) without a device, so the produced
tensor was a CPU float tensor regardless of the surrounding tensordict.
This broke any path that compared the version against an integer dtype
or that lived on CUDA.

Read the tensordict's device and pass dtype=torch.int64 (with
device only when the tensordict has one, since CPU tensordicts can
have device=None).

Add a regression test pinning the dtype, shape, and device of the
emitted policy_version tensor.

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/3755

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❗ 1 Active SEVs

There are 1 currently active SEVs. If your PR is affected, please view them below:

• Run pull request jobs on OSDC runners in shadow mode

❌ 3 New Failures, 3 Unrelated Failures

As of commit 938b896 with merge base 0a01ee8 ():

NEW FAILURES - The following jobs have failed:

• Build Windows Wheels / pytorch/rl / build-wheel-py3_10-cpu (gh)
  Process completed with exit code 1.
• Build Windows Wheels / pytorch/rl / upload / upload-wheel-py3_10-cpu (gh)
  Unable to download artifact(s): Artifact not found for name: pytorch_rl__3.10_cpu_x64
• Libs Tests on Linux / unittests-gym (3.10, 12.8) / linux-job (gh)
  test/libs/test_gym.py::TestGym::test_set_seed_and_reset_works

FLAKY - The following jobs failed but were likely due to flakiness present on trunk:

• LLM Tests on Linux / unittests-vllm (3.12, 12.9) / linux-job (gh) (detected as infra flaky with no log or failing log classifier)
• Unit-tests on Linux / tests-cpu (3.12) / linux-job (gh) (detected as infra flaky with no log or failing log classifier)
• Unit-tests on Linux / tests-cpu (3.13) / linux-job (gh) (detected as infra flaky with no log or failing log classifier)

This comment was automatically generated by Dr. CI and updates every 15 minutes.