ggml-quants: Provide ggml_vqtbl1q_u8 for 64bit compatibility #5711
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
rgryta
changed the title
rgryta
changed the title
|
android-build workflow will fail as it's fetching the current llama.cpp dependency from master (which of course doesn't have the neon v7a fix for now) |
|
@ggerganov, let me know if you'd rather I split this PR into two separate ones instead (one that adds the ARM-V check and then another one that updates the build workflow). |
ggerganov
reviewed
Feb 25, 2024
ggml-quants.c
Outdated
| @@ -9452,7 +9452,7 @@ void ggml_vec_dot_iq3_s_q8_K (int n, float * GGML_RESTRICT s, size_t bs, const v | |||
|
|
|||
| const int nb = n / QK_K; | |||
|
|
|||
| #if defined(__ARM_NEON) | |||
| #if defined(__ARM_NEON) && (__ARM_ARCH >= 8) | |||
There was a problem hiding this comment.
Instead of this, we should add ggml_vqtbl1q_u8 similar to how we have ggml_vqtbl1q_s8 and replace all usages of vqtbl1q_u8 with ggml_vqtbl1q_u8
There was a problem hiding this comment.
Amended and compiled locally in and Android project
vqtbl1q_u8 is not part of arm v7 neon library
rgryta
changed the title
ggerganov
approved these changes
Feb 25, 2024
jordankanter
pushed a commit
to jordankanter/llama.cpp
that referenced
this pull request
Mar 13, 2024
…rg#5711) * [ggml-quants] Provide ggml_vqtbl1q_u8 for 64bit compatibility vqtbl1q_u8 is not part of arm v7 neon library * [android-example] Remove abi filter after arm v7a fix * [github-workflows] Do not skip Android armeabi-v7a build
hodlen
pushed a commit
to hodlen/llama.cpp
that referenced
this pull request
Apr 1, 2024
…rg#5711) * [ggml-quants] Provide ggml_vqtbl1q_u8 for 64bit compatibility vqtbl1q_u8 is not part of arm v7 neon library * [android-example] Remove abi filter after arm v7a fix * [github-workflows] Do not skip Android armeabi-v7a build
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Additionally unblocks Android example and workflow build for Android build with armeabi-v7a target.
Function
vqtbl1q_u8is not available under neon ARM-V7.