-
Notifications
You must be signed in to change notification settings - Fork 12.2k
Update ggml_sycl_op_mul_mat_vec_q #5502
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Update ggml_sycl_op_mul_mat_vec_q #5502
Conversation
@NeoZhangJianyu, @abhilash1910, @Alcpz, feedback would be appreciated |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Minor comment on the refactor. Looks great.
Thanks @AidanBeltonS , could you please rebase to latest master for CI build? |
b7cc0f2
to
4098314
Compare
@ggerganov @0cc4m I think the vulkan build CI is exiting abruptly - maybe issue is common for other requests. Could you help take a look ? Thanks |
It's because we enabled the We can easily disable the |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
@AidanBeltonS could you please rebase to latest master - should solve some build issues with vulkan ci. |
|
Co-authored-by: Abhilash Majumder <[email protected]>
b1edf62
to
5d7d353
Compare
|
* Update ggml_sycl_op_mul_mat_vec_q * Apply suggestions from code review Co-authored-by: Abhilash Majumder <[email protected]> * revert suggestion on macro * fix bug * Add quant type GGML_TYPE_IQ1_S to unsupported * fix format --------- Co-authored-by: Abhilash Majumder <[email protected]>
* Update ggml_sycl_op_mul_mat_vec_q * Apply suggestions from code review Co-authored-by: Abhilash Majumder <[email protected]> * revert suggestion on macro * fix bug * Add quant type GGML_TYPE_IQ1_S to unsupported * fix format --------- Co-authored-by: Abhilash Majumder <[email protected]>
This PR updates the unsupported quantized data types and refactors the code for
ggml_sycl_op_mul_mat_vec_q
.SYCL does not currently have the intrinsics to support some quantized data types, this adds one missing quantized data type to the unsupported check, so tests won't be run.
This also refactors the code so there is a single templated
mul_mat_vec_q_sycl_submitter
rather than multiple duplicate functions which submit a different instantiated kernel. This makes the code less verbose and much smaller.