Refine layerwise non-mutating calibration#1592
Conversation
Signed-off-by: realAsma <akuriparambi@nvidia.com>
Signed-off-by: realAsma <akuriparambi@nvidia.com>
|
Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually. Contributors can view more details about this message here. |
|
Important Review skippedDraft detected. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Enterprise Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
| if not writeback: | ||
| with _fsdp2_unshard_context(fsdp_module): | ||
| yield | ||
| return | ||
|
|
There was a problem hiding this comment.
@sugunav14 here is an easy perf improvement for layerwise FSDP2
Stacked on #1571.
Summary:
Testing: