Skip to content

ggml-cpu: extend RVV quantization vec dot to higher VLENs#10

Open
taimur-10x wants to merge 4 commits intomasterfrom
10x/riscv-quant-vec-dot-vlens
Open

ggml-cpu: extend RVV quantization vec dot to higher VLENs#10
taimur-10x wants to merge 4 commits intomasterfrom
10x/riscv-quant-vec-dot-vlens

Conversation

@taimur-10x
Copy link
Copy Markdown
Collaborator

@taimur-10x taimur-10x commented Feb 13, 2026

Summary

This PR adds RVV implementations for quantized vector dot kernels (for VLENs 512-bit and 1024-bit).

Key Changes

  • Added the following RVV kernels:
Kernel VLEN
ggml_vec_dot_q3_K_q8_K 512, 1024
ggml_vec_dot_q6_K_q8_K 512, 1024
ggml_vec_dot_iq1_s_q8_K 512, 1024
ggml_vec_dot_iq1_m_q8_K 512, 1024
ggml_vec_dot_iq2_s_q8_K 512, 1024 and above
ggml_vec_dot_iq2_xs_q8_K 512 and above
ggml_vec_dot_iq2_xxs_q8_K 512 and above
ggml_vec_dot_iq3_s_q8_K 512 and above
ggml_vec_dot_iq3_xxs_q8_K 512, 1024 and above
ggml_vec_dot_iq4_xs_q8_K 512, 1024
ggml_vec_dot_tq1_0_q8_K 512 and above
  • Each kernel that is VLEN-dependent has its own separate function call now, for example ggml_vec_dot_tq1_0_q8_K_vl512.

Testing

Kernels were functionally tested through test-quantize-fns for VLENs 512-bit and 1024-bit with QEMU, using test-quantize-fns.

@taimur-10x taimur-10x marked this pull request as draft February 13, 2026 15:52
@github-actions github-actions Bot added the ggml label Feb 13, 2026
@taimur-10x taimur-10x changed the base branch from master to 10x/riscv-quant-vec-dot-128b March 4, 2026 11:48
@taimur-10x taimur-10x force-pushed the 10x/riscv-quant-vec-dot-vlens branch from 86ffc7e to bd69a20 Compare March 4, 2026 17:10
@taimur-10x taimur-10x marked this pull request as ready for review March 4, 2026 17:11
@rehan-10xengineer rehan-10xengineer force-pushed the 10x/riscv-quant-vec-dot-128b branch 2 times, most recently from f83ddf7 to c7c6abc Compare March 16, 2026 10:55
@taimur-10x taimur-10x force-pushed the 10x/riscv-quant-vec-dot-128b branch 3 times, most recently from cf95828 to 05a5425 Compare March 18, 2026 12:47
@rehan-10xengineer rehan-10xengineer force-pushed the 10x/riscv-quant-vec-dot-128b branch from 05a5425 to 80c0ac3 Compare April 14, 2026 11:37
@taimur-10x taimur-10x changed the base branch from 10x/riscv-quant-vec-dot-128b to master April 24, 2026 12:14
@taimur-10x taimur-10x force-pushed the 10x/riscv-quant-vec-dot-vlens branch from bd69a20 to aa171a2 Compare April 24, 2026 14:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants