Skip to content

Commit 8d5aa32

Browse files
wanix1988hodlen
authored andcommitted
readme : update details about running llama in Termux on Android (ggml-org#6039)
1 parent cbcd235 commit 8d5aa32

File tree

1 file changed

+25
-1
lines changed

1 file changed

+25
-1
lines changed

README.md

Lines changed: 25 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -904,6 +904,9 @@ First, install the essential packages for termux:
904904
pkg install clang wget git cmake
905905
```
906906
Second, obtain the [Android NDK](https://developer.android.com/ndk) and then build with CMake:
907+
908+
You can execute the following commands on your computer to avoid downloading the NDK to your mobile. Of course, you can also do this in Termux.
909+
907910
```
908911
$ mkdir build-android
909912
$ cd build-android
@@ -912,7 +915,28 @@ $ cmake -DCMAKE_TOOLCHAIN_FILE=$NDK/build/cmake/android.toolchain.cmake -DANDROI
912915
$ make
913916
```
914917
Install [termux](https://termux.dev/) on your device and run `termux-setup-storage` to get access to your SD card.
915-
Finally, copy the `llama` binary and the model files to your device storage. Here is a demo of an interactive session running on Pixel 5 phone:
918+
Finally, copy these built `llama` binaries and the model file to your device storage. Because the file permissions in the Android sdcard cannot be changed, you can copy the executable files to the `/data/data/com.termux/files/home/bin` path, and then execute the following commands in Termux to add executable permission:
919+
920+
(Assumed that you have pushed the built executable files to the /sdcard/llama.cpp/bin path using `adb push`)
921+
```
922+
$cp -r /sdcard/llama.cpp/bin /data/data/com.termux/files/home/
923+
$cd /data/data/com.termux/files/home/bin
924+
$chmod +x ./*
925+
```
926+
927+
Download model [llama-2-7b-chat.Q4_K_M.gguf](https://huggingface.co/TheBloke/Llama-2-7B-Chat-GGUF/blob/main/llama-2-7b-chat.Q4_K_M.gguf), and push it to `/sdcard/llama.cpp/`, then move it to `/data/data/com.termux/files/home/model/`
928+
929+
```
930+
$mv /sdcard/llama.cpp/llama-2-7b-chat.Q4_K_M.gguf /data/data/com.termux/files/home/model/
931+
```
932+
933+
Now, you can start chatting:
934+
```
935+
$cd /data/data/com.termux/files/home/bin
936+
$./main -m ../model/llama-2-7b-chat.Q4_K_M.gguf -n 128 -cml
937+
```
938+
939+
Here is a demo of an interactive session running on Pixel 5 phone:
916940
917941
https://user-images.githubusercontent.com/271616/225014776-1d567049-ad71-4ef2-b050-55b0b3b9274c.mp4
918942

0 commit comments

Comments
 (0)