Skip to content

Full MXFP4 Training Recipe #506

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
sarthak-amd wants to merge 10 commits into
base: dev
Choose a base branch
from
Open

Full MXFP4 Training Recipe #506

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
10 commits
Select commit Hold shift + click to select a range
bd648c6
Add MXFP4 HIP kernel infrastructure and build system integration
sarthak-amd Mar 26, 2026
57fcc78
Add MXFP4 quantizer, recipe, GEMM dispatch, and fp4_handler
sarthak-amd Mar 26, 2026
d7a19a5
Integrate MXFP4 into Linear and LayerNormLinear modules
sarthak-amd Mar 26, 2026
8c6af7f
Merge branch 'dev' into feature/mxfp4-recipe-210
sudhu2k Apr 8, 2026
1621983
MXFP4: Add GEMM kernel tuning and MXFP4Quantizer.copy()
sarthak-amd Apr 9, 2026
43958a4
Fix _get_quantizers indentation after dev merge
sarthak-amd Apr 9, 2026
0a91c99
Move MXFP4 kernel from PyTorch extensions to common/
sarthak-amd Apr 10, 2026
aa2f387
Add C++ gtest for MXFP4 fused cast-transpose kernel
sarthak-amd Apr 10, 2026
dd99e0c
Add MXFP4 pytest: A4W4 GEMM, Linear, LayerNormLinear
sarthak-amd Apr 10, 2026
ba77f88
guard _is_mxfp4_enabled()
suachong Apr 10, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitignore
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,7 @@ cmake_install.cmake
*.hip
*_hip.h
*_hip.cpp
!transformer_engine/pytorch/csrc/extensions/mxfp4_hip.cpp
*_hip.cuh
hip_driver*
hip_runtime*
Expand Down
3 changes: 2 additions & 1 deletion tests/cpp/operator/CMakeLists.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,8 @@ if(USE_CUDA)
test_swizzle.cu)
else()
list(APPEND test_cuda_sources
test_cublaslt_gemm.cu)
test_cublaslt_gemm.cu
test_cast_transpose_mxfp4_shuffled.cu)
endif()

if(USE_CUDA)
Expand Down
Loading
Loading