Skip to content

Commit 299e70d

Browse files
NeoZhangJianyuarthw
andcommitted
enhance run script to be easy to change the parameters (ggml-org#9448)
Co-authored-by: arthw <[email protected]>
1 parent ba7432c commit 299e70d

File tree

1 file changed

+9
-19
lines changed

1 file changed

+9
-19
lines changed

examples/sycl/run-llama2.sh

Lines changed: 9 additions & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -4,33 +4,23 @@
44
# Copyright (C) 2024 Intel Corporation
55
# SPDX-License-Identifier: MIT
66

7-
INPUT2="Building a website can be done in 10 simple steps:\nStep 1:"
87
source /opt/intel/oneapi/setvars.sh
98

10-
if [ $# -gt 0 ]; then
11-
GGML_SYCL_DEVICE=$1
12-
GGML_SYCL_SINGLE_GPU=1
13-
else
14-
GGML_SYCL_DEVICE=0
15-
GGML_SYCL_SINGLE_GPU=0
16-
fi
17-
189
#export GGML_SYCL_DEBUG=1
1910

20-
2111
#ZES_ENABLE_SYSMAN=1, Support to get free memory of GPU by sycl::aspect::ext_intel_free_memory. Recommended to use when --split-mode = layer.
2212

23-
if [ $GGML_SYCL_SINGLE_GPU -eq 1 ]; then
13+
INPUT_PROMPT="Building a website can be done in 10 simple steps:\nStep 1:"
14+
MODEL_FILE=llama-2-7b.Q4_0.gguf
15+
NGL=33
16+
17+
if [ $# -gt 0 ]; then
18+
GGML_SYCL_DEVICE=$1
2419
echo "use $GGML_SYCL_DEVICE as main GPU"
2520
#use signle GPU only
26-
ZES_ENABLE_SYSMAN=1 ./build/bin/llama-cli -m models/llama-2-7b.Q4_0.gguf -p "${INPUT2}" -n 400 -e -ngl 33 -s 0 -mg $GGML_SYCL_DEVICE -sm none
21+
ZES_ENABLE_SYSMAN=1 ./build/bin/llama-cli -m models/${MODEL_FILE} -p "${INPUT_PROMPT}" -n 400 -e -ngl ${NGL} -s 0 -mg $GGML_SYCL_DEVICE -sm none
22+
2723
else
2824
#use multiple GPUs with same max compute units
29-
ZES_ENABLE_SYSMAN=1 ./build/bin/llama-cli -m models/llama-2-7b.Q4_0.gguf -p "${INPUT2}" -n 400 -e -ngl 33 -s 0
25+
ZES_ENABLE_SYSMAN=1 ./build/bin/llama-cli -m models/${MODEL_FILE} -p "${INPUT_PROMPT}" -n 400 -e -ngl ${NGL} -s 0
3026
fi
31-
32-
#use main GPU only
33-
#ZES_ENABLE_SYSMAN=1 ./build/bin/llama-cli -m models/llama-2-7b.Q4_0.gguf -p "${INPUT2}" -n 400 -e -ngl 33 -s 0 -mg $GGML_SYCL_DEVICE -sm none
34-
35-
#use multiple GPUs with same max compute units
36-
#ZES_ENABLE_SYSMAN=1 ./build/bin/llama-cli -m models/llama-2-7b.Q4_0.gguf -p "${INPUT2}" -n 400 -e -ngl 33 -s 0

0 commit comments

Comments
 (0)