Skip to content

Commit 21706db

Browse files
Mishiggary149
andauthored
[llama.cpp snippet] Present both options (#737)
follow up to #720 just like we show two snippets for transformers (pipeline & automodel) (example [here](https://huggingface.co/mistralai/Codestral-22B-v0.1?library=transformers)) ![Screenshot 2024-06-05 at 14 51 22](https://github.com/huggingface/huggingface.js/assets/11827707/04cc26ce-18c4-497d-8a2e-c0983bd07085) we should show the two options for llama.cpp clearly as well. The two options are: installing from brew and installing from source ![Screenshot 2024-06-05 at 15 37 22](https://github.com/huggingface/huggingface.js/assets/11827707/4cfd87f0-62d7-471c-bbb0-05d8b8c82ef7) --------- Co-authored-by: Victor Muštar <[email protected]>
1 parent 4118cb9 commit 21706db

File tree

1 file changed

+13
-8
lines changed

1 file changed

+13
-8
lines changed

packages/tasks/src/local-apps.ts

Lines changed: 13 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -48,20 +48,25 @@ function isGgufModel(model: ModelData) {
4848

4949
const snippetLlamacpp = (model: ModelData): string[] => {
5050
return [
51-
`## Install llama.cpp via brew
51+
`# Option 1: use llama.cpp with brew
5252
brew install llama.cpp
5353
54-
## or from source with curl support
55-
## see llama.cpp README for compilation flags to optimize for your hardware
56-
git clone https://github.com/ggerganov/llama.cpp
57-
cd llama.cpp
58-
LLAMA_CURL=1 make
59-
`,
60-
`## Load and run the model
54+
# Load and run the model
6155
llama \\
6256
--hf-repo "${model.id}" \\
6357
--hf-file file.gguf \\
6458
-p "I believe the meaning of life is" \\
59+
-n 128`,
60+
`# Option 2: build llama.cpp from source with curl support
61+
git clone https://github.com/ggerganov/llama.cpp.git
62+
cd llama.cpp
63+
LLAMA_CURL=1 make
64+
65+
# Load and run the model
66+
./main \\
67+
--hf-repo "${model.id}" \\
68+
-m file.gguf \\
69+
-p "I believe the meaning of life is" \\
6570
-n 128`,
6671
];
6772
};

0 commit comments

Comments
 (0)