Skip to content

Commit 5220a99

Browse files
authored
Increase 3B scratch buffers. (#1698)
The 128 MB was too optimistic. Too bad it is not dynamically computed.
1 parent d1f563a commit 5220a99

File tree

1 file changed

+2
-2
lines changed

1 file changed

+2
-2
lines changed

llama.cpp

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -63,7 +63,7 @@ static const size_t MB = 1024*1024;
6363
static const std::map<e_model, size_t> & MEM_REQ_SCRATCH0()
6464
{
6565
static std::map<e_model, size_t> k_sizes = {
66-
{ MODEL_3B, 128ull * MB },
66+
{ MODEL_3B, 256ull * MB },
6767
{ MODEL_7B, 512ull * MB },
6868
{ MODEL_13B, 512ull * MB },
6969
{ MODEL_30B, 512ull * MB },
@@ -75,7 +75,7 @@ static const std::map<e_model, size_t> & MEM_REQ_SCRATCH0()
7575
static const std::map<e_model, size_t> & MEM_REQ_SCRATCH1()
7676
{
7777
static std::map<e_model, size_t> k_sizes = {
78-
{ MODEL_3B, 128ull * MB },
78+
{ MODEL_3B, 256ull * MB },
7979
{ MODEL_7B, 512ull * MB },
8080
{ MODEL_13B, 512ull * MB },
8181
{ MODEL_30B, 512ull * MB },

0 commit comments

Comments
 (0)