-
Notifications
You must be signed in to change notification settings - Fork 6.8k
Open
Labels
Description
The set checks when using hub kernels with GGUF are not traceable with torch.compile.
diffusers/src/diffusers/quantizers/gguf/utils.py
Lines 74 to 76 in b0dc51d
| DEQUANT_TYPES = STANDARD_QUANT_TYPES | KQUANT_TYPES | IMATRIX_QUANT_TYPES | |
| MMVQ_QUANT_TYPES = STANDARD_QUANT_TYPES | KQUANT_TYPES | IMATRIX_QUANT_TYPES | |
| MMQ_QUANT_TYPES = STANDARD_QUANT_TYPES | KQUANT_TYPES |
The GGUF Kernels also need to be updated to support the latest version of torch.
Initial Solution has been proposed: #12556
Reactions are currently unavailable