ggml : reading the runtime sve config of the cpu

[jdomke]

• I have read the contributing guidelines
• Self-reported review complexity:
  • Low
  • Medium
  • High

To go from SVE512 to SVE256 (or other configurations), one can use the following:

$ cat wrapper.c
#include <sys/prctl.h>
#include <unistd.h>
int main(int argc, char *argv[], char *envp[]){
  prctl(PR_SVE_SET_VL,32|PR_SVE_VL_INHERIT);
  execve(argv[1], &argv[1], envp); return 0;
}
$ gcc wrapper.c -o wrapper
$ ./wrapper ./bin/llama-cli ...

However, when ggml reads the "current" sve width via svcntb() it still gets the wrong value, and only prctl(PR_SVE_GET_VL) returns the correct value.

[jdomke]

The issue might actually be the result of a bug in our clang runtime causing svcntb to return the wrong value; we are confirming right now.

[jdomke]

The issue might actually be the result of a bug in our clang runtime causing svcntb to return the wrong value; we are confirming right now.

According to Fujitsu's compiler developers, the compiler assumes that the vector length does not change during the program execution, and hence svcntb() is assumed to return the same vector length and the compiler may optimize it away. Therefor, the only workaround to allow users (withour root rights) to lower the vector length with the wrapper is to fall back to PR_SVE_VL_LEN_MASK & prctl(PR_SVE_GET_VL) inside of llama.cpp.

[ggerganov]

Also need to update the svcntb() calls in ggml-quants.c