GPU usage not change while using CLBlast using AMD GPU

[mikeyang01]

CPU: AMD Ryzen™ 7 7735HS
GPU: AMD 680M

Expectation:
High GPU usage, not CPU

Current:
I downloaded the zip file directly from the link
https://github.com/ggerganov/llama.cpp/releases/download/master-5656d10/llama-master-5656d10-bin-win-clblast-x64.zip
I found High CPU usage, GPU not using,
When I use other app using Directml , GPU usage is correct.

Software version:
https://github.com/ggerganov/llama.cpp/releases/tag/master-5656d10

Platform:
windows11

Imgur

PS C:\Users\Downloads\llama-master-5656d10-bin-win-clblast-x64> ./main -t 10 -ngl 32 -m C:\Github\models\wizard-vicuna-13B.ggmlv3.q4_0.bin --color -c 2048 --temp 0.7 --repeat_penalty 1.1 -n -1 -p "USER: how to travel from NY to Boston\nASSISTANT:"
main: build = 813 (5656d10)
main: seed  = 1689041934
ggml_opencl: selecting platform: 'AMD Accelerated Parallel Processing'
ggml_opencl: selecting device: 'gfx1035'
ggml_opencl: device FP16 support: true
llama.cpp: loading model from C:\Github\models\wizard-vicuna-13B.ggmlv3.q4_0.bin
llama_model_load_internal: format     = ggjt v3 (latest)
llama_model_load_internal: n_vocab    = 32000
llama_model_load_internal: n_ctx      = 2048
llama_model_load_internal: n_embd     = 5120
llama_model_load_internal: n_mult     = 256
llama_model_load_internal: n_head     = 40
llama_model_load_internal: n_layer    = 40
llama_model_load_internal: n_rot      = 128
llama_model_load_internal: ftype      = 2 (mostly Q4_0)
llama_model_load_internal: n_ff       = 13824
llama_model_load_internal: model size = 13B
llama_model_load_internal: ggml ctx size =    0.09 MB
llama_model_load_internal: using OpenCL for GPU acceleration
llama_model_load_internal: mem required  = 3585.46 MB (+ 1608.00 MB per state)
llama_model_load_internal: offloading 32 repeating layers to GPU
llama_model_load_internal: offloaded 32/41 layers to GPU
llama_model_load_internal: total VRAM used: 5447 MB
llama_new_context_with_model: kv self size  = 1600.00 MB

system_info: n_threads = 10 / 16 | AVX = 1 | AVX2 = 1 | AVX512 = 0 | AVX512_VBMI = 0 | AVX512_VNNI = 0 | FMA = 1 | NEON = 0 | ARM_FMA = 0 | F16C = 1 | FP16_VA = 0 | WASM_SIMD = 0 | BLAS = 1 | SSE3 = 1 | VSX = 0 |
sampling: repeat_last_n = 64, repeat_penalty = 1.100000, presence_penalty = 0.000000, frequency_penalty = 0.000000, top_k = 40, tfs_z = 1.000000, top_p = 0.950000, typical_p = 1.000000, temp = 0.700000, mirostat = 0, mirostat_lr = 0.100000, mirostat_ent = 5.000000
generate: n_ctx = 2048, n_batch = 512, n_predict = -1, n_keep = 0


 USER: how to travel from NY to Boston\nASSISTANT: There are several ways to travel from New York to Boston:
1. By car: Take I-95 Northbound for about 3 hours and 20 minutes until you reach Boston.
2. By train: Take the Amtrak Acela or Northeast Regional trains from Penn Station in Manhattan to South Station in Boston, with a travel time of around 3 hours and 15 minutes.
3. By plane: Fly from JFK or LGA Airport in New York to Boston Logan International Airport, with a flight time of around
 1 hour and 15 minutes.
4. By bus: Take the BoltBus, Megabus, or Peter Pan Bus Lines from various locations in Manhattan to Boston, with a travel time of around 3 hours and 30 minutes. [end of text]

llama_print_timings:        load time =  3541.00 ms
llama_print_timings:      sample time =    33.54 ms /   174 runs   (    0.19 ms per token,  5188.14 tokens per second)
llama_print_timings: prompt eval time =  6080.93 ms /    17 tokens (  357.70 ms per token,     2.80 tokens per second)
llama_print_timings:        eval time = 53173.72 ms /   173 runs   (  307.36 ms per token,     3.25 tokens per second)
llama_print_timings:       total time = 59305.99 ms

[ianscrivener]

Perhaps try the OpenBlas version: https://github.com/ggerganov/llama.cpp/releases/download/master-5656d10/llama-master-5656d10-bin-win-openblas-x64.zip

If not then you may need to build llama.cpp for your specific GPU - https://github.com/ggerganov/llama.cpp#build

[mikeyang01]

tried on openblas, speed not change.
Is there any difference between the release version which i downloaded and the version compiled by myself?

[ianscrivener]

Hmm.. I'm just guessing TBH

There's a Discussion about AMD 680M here #1185 that may be helpful

Or search llama.cpp Discussions for AMD discussions