Skip to content

Commit 78fceb3

Browse files
netrunnerevearthw
authored andcommitted
llamafile : disable sgemm for batch-size 1 (ggml-org#9330)
1 parent d99241e commit 78fceb3

File tree

1 file changed

+4
-0
lines changed

1 file changed

+4
-0
lines changed

ggml/src/llamafile/sgemm.cpp

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1006,6 +1006,10 @@ bool llamafile_sgemm(int64_t m, int64_t n, int64_t k, const void *A, int64_t lda
10061006
assert(nth > 0);
10071007
assert(ith < nth);
10081008

1009+
// only enable sgemm for prompt processing
1010+
if (n < 2)
1011+
return false;
1012+
10091013
if (Ctype != GGML_TYPE_F32)
10101014
return false;
10111015

0 commit comments

Comments
 (0)