feat: Add YaRN rope scaling support on Magatron-Bridge

[RayenTian]

What does this PR do ?

SFT

GRPO

Issues

close #1678

Usage

• You can potentially add a usage example below

SFT

NRL_FORCE_REBUILD_VENVS=true \
uv run examples/run_sft.py \
        --config examples/configs/sft_openmathinstruct2_megatron.yaml \
        policy.model_name=${MODEL} \
        sft.max_num_steps=100 \
        policy.max_total_sequence_length=8192 \
        checkpointing.enabled=false \
        cluster.num_nodes=1 \
        ++policy.hf_config_overrides.rope_scaling.rope_type="yarn" \
        ++policy.hf_config_overrides.rope_scaling.factor=6.0 \
        ++policy.hf_config_overrides.rope_scaling.original_max_position_embeddings=32768

GRPO

uv run python examples/run_grpo.py \
    --config=examples/configs/grpo_math_1B_megatron.yaml \
    policy.model_name=${MODEL} \
    grpo.max_num_steps=100\
    ++policy.hf_config_overrides.rope_scaling.rope_type=yarn \
    ++policy.hf_config_overrides.rope_scaling.factor=6.0 \
    ++policy.hf_config_overrides.rope_scaling.original_max_position_embeddings=32768 \
    ++policy.generation.vllm_kwargs.hf_overrides.rope_scaling.rope_type=yarn \
    ++policy.generation.vllm_kwargs.hf_overrides.rope_scaling.factor=6.0 \
    ++policy.generation.vllm_kwargs.hf_overrides.rope_scaling.original_max_position_embeddings=32768 \
    logger.wandb_enabled=false \
    checkpointing.enabled=false \
    cluster.num_nodes=1

Before your PR is "Ready for review"

Pre checks:

• Make sure you read and followed Contributor guidelines
• Did you write any new necessary tests?
• Did you run the unit tests and functional tests locally? Visit our Testing Guide for how to run tests
• Did you add or update any necessary documentation? Visit our Document Development Guide for how to write, build and test the docs.

Additional Information

• ...

[github-actions]

✅ Submodule Fast-Forward Check Results

Check based on commit: 3f16d49 (PR #2052 from ruit/yarn)

✅ Submodules that are properly updated:

Megatron-Bridge: ✅ PR branch is ahead of main branch (fast-forward)

All submodule changes look good! ✨

[github-actions]

✅ Submodule Fast-Forward Check Results

Check based on commit: dea9529 (PR #2052 from ruit/yarn)

✅ Submodules that are properly updated:

Megatron-Bridge: ✅ PR branch is ahead of main branch (fast-forward)

All submodule changes look good! ✨

[github-actions]

✅ Submodule Fast-Forward Check Results

Check based on commit: e4dc70a (PR #2052 from ruit/yarn)

✅ Submodules that are properly updated:

Megatron-Bridge: ✅ PR branch is ahead of main branch (fast-forward)

All submodule changes look good! ✨

[github-actions]

✅ Submodule Fast-Forward Check Results

Check based on commit: 6e11596 (PR #2052 from ruit/yarn)

✅ Submodules that are properly updated:

Megatron-Bridge: ✅ PR branch is ahead of main branch (fast-forward)

All submodule changes look good! ✨

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[github-actions]

✅ Submodule Fast-Forward Check Results

Check based on commit: 1a1d65b (PR #2052 from ruit/yarn)

✅ Submodules that are properly updated:

Megatron-Bridge: ✅ PR branch is ahead of main branch (fast-forward)

All submodule changes look good! ✨

[github-actions]

✅ Submodule Fast-Forward Check Results

Check based on commit: 62789ac (PR #2052 from ruit/yarn)

✅ Submodules that are properly updated:

Megatron-Bridge: ✅ PR branch is ahead of main branch (fast-forward)

All submodule changes look good! ✨

[github-actions]

✅ Submodule Fast-Forward Check Results

Check based on commit: ef29608 (PR #2052 from ruit/yarn)

✅ Submodules that are properly updated:

Megatron-Bridge: ✅ PR branch is ahead of main branch (fast-forward)

All submodule changes look good! ✨

[github-actions]

✅ Submodule Fast-Forward Check Results

Check based on commit: d386e40 (PR #2052 from ruit/yarn)

✅ Submodules that are properly updated:

Megatron-Bridge: ✅ PR branch is ahead of main branch (fast-forward)

All submodule changes look good! ✨

[RayenTian]

/ok to test d9cdeb4

[github-actions]

✅ Submodule Fast-Forward Check Results

Check based on commit: d9cdeb4 (PR #2052 from ruit/yarn)

✅ Submodules that are properly updated:

Megatron-Bridge: ✅ PR branch is ahead of main branch (fast-forward)

All submodule changes look good! ✨