Skip to content

Commit 106964e

Browse files
authored
metal : enable mat-vec kernels for bs <= 4 (ggml-org#10491)
1 parent 80acb7b commit 106964e

File tree

1 file changed

+1
-1
lines changed

1 file changed

+1
-1
lines changed

ggml/src/ggml-metal/ggml-metal.m

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1927,7 +1927,7 @@ static void ggml_metal_encode_node(
19271927

19281928
// find the break-even point where the matrix-matrix kernel becomes more efficient compared
19291929
// to the matrix-vector kernel
1930-
int ne11_mm_min = 1;
1930+
int ne11_mm_min = 4;
19311931

19321932
#if 0
19331933
// the numbers below are measured on M2 Ultra for 7B and 13B models

0 commit comments

Comments
 (0)