|
2 | 2 | # Android
|
3 | 3 |
|
4 | 4 | ## Build on Android using Termux
|
5 |
| -[Termux](https://termux.dev/en/) is an Android terminal emulator and Linux environment app (no root required). With Termux, you can install and run `llama.cpp` as if the environment were Linux. Once in the Termux shell: |
6 | 5 |
|
7 |
| -``` |
8 |
| -apt update && apt upgrade -y |
9 |
| -apt install git make cmake |
10 |
| -``` |
11 |
| - |
12 |
| -It's recommended to move your model inside the `~/` directory for best performance: |
13 |
| - |
14 |
| -``` |
15 |
| -cd storage/downloads |
16 |
| -mv {model}.gguf ~/ |
17 |
| -``` |
18 |
| - |
19 |
| -[Get the code](https://github.com/ggerganov/llama.cpp#get-the-code) & [follow the Linux build instructions](https://github.com/ggerganov/llama.cpp#build) to build `llama.cpp`. |
| 6 | +[Termux](https://termux.dev/en/) is an Android terminal emulator and Linux environment app (no root required). As of writing, Termux is available experimentally in the Google Play Store; otherwise, it may be obtained directly from the project repo or on F-Droid. |
20 | 7 |
|
21 |
| -## Building the Project using Android NDK |
22 |
| -Obtain the [Android NDK](https://developer.android.com/ndk) and then build with CMake. |
| 8 | +With Termux, you can install and run `llama.cpp` as if the environment were Linux. Once in the Termux shell: |
23 | 9 |
|
24 |
| -Execute the following commands on your computer to avoid downloading the NDK to your mobile. Alternatively, you can also do this in Termux: |
25 | 10 | ```
|
26 |
| -$ mkdir build-android |
27 |
| -$ cd build-android |
28 |
| -$ export NDK=<your_ndk_directory> |
29 |
| -$ cmake -DCMAKE_TOOLCHAIN_FILE=$NDK/build/cmake/android.toolchain.cmake -DANDROID_ABI=arm64-v8a -DANDROID_PLATFORM=android-23 -DCMAKE_C_FLAGS=-march=armv8.4a+dotprod .. |
30 |
| -$ make |
| 11 | +$ apt update && apt upgrade -y |
| 12 | +$ apt install git cmake |
31 | 13 | ```
|
32 | 14 |
|
33 |
| -Install [termux](https://github.com/termux/termux-app#installation) on your device and run `termux-setup-storage` to get access to your SD card (if Android 11+ then run the command twice). |
| 15 | +Then, follow the [build instructions](https://github.com/ggerganov/llama.cpp/blob/master/docs/build.md), specifically for CMake. |
34 | 16 |
|
35 |
| -Finally, copy these built `llama` binaries and the model file to your device storage. Because the file permissions in the Android sdcard cannot be changed, you can copy the executable files to the `/data/data/com.termux/files/home/bin` path, and then execute the following commands in Termux to add executable permission: |
| 17 | +Once the binaries are built, download your model of choice (e.g., from Hugging Face). It's recommended to place it in the `~/` directory for best performance: |
36 | 18 |
|
37 |
| -(Assumed that you have pushed the built executable files to the /sdcard/llama.cpp/bin path using `adb push`) |
38 | 19 | ```
|
39 |
| -$cp -r /sdcard/llama.cpp/bin /data/data/com.termux/files/home/ |
40 |
| -$cd /data/data/com.termux/files/home/bin |
41 |
| -$chmod +x ./* |
| 20 | +$ curl -L {model-url} -o ~/{model}.gguf |
42 | 21 | ```
|
43 | 22 |
|
44 |
| -Download model [llama-2-7b-chat.Q4_K_M.gguf](https://huggingface.co/TheBloke/Llama-2-7B-Chat-GGUF/blob/main/llama-2-7b-chat.Q4_K_M.gguf), and push it to `/sdcard/llama.cpp/`, then move it to `/data/data/com.termux/files/home/model/` |
| 23 | +Then, if you are not already in the repo directory, `cd` into `llama.cpp` and: |
45 | 24 |
|
46 | 25 | ```
|
47 |
| -$mv /sdcard/llama.cpp/llama-2-7b-chat.Q4_K_M.gguf /data/data/com.termux/files/home/model/ |
| 26 | +$ ./build/bin/llama-simple -m ~/{model}.gguf -c {context-size} -p "{your-prompt}" |
48 | 27 | ```
|
49 | 28 |
|
50 |
| -Now, you can start chatting: |
51 |
| -``` |
52 |
| -$cd /data/data/com.termux/files/home/bin |
53 |
| -$./llama-cli -m ../model/llama-2-7b-chat.Q4_K_M.gguf -n 128 -cml |
54 |
| -``` |
| 29 | +Here, we show `llama-simple`, but any of the executables under `examples` should work, in theory. Be sure to set `context-size` to a reasonable number (say, 4096) to start with; otherwise, memory could spike and kill your terminal. |
55 | 30 |
|
56 |
| -Here's a demo of an interactive session running on Pixel 5 phone: |
| 31 | +To see what it might look like visually, here's an old demo of an interactive session running on a Pixel 5 phone: |
57 | 32 |
|
58 | 33 | https://user-images.githubusercontent.com/271616/225014776-1d567049-ad71-4ef2-b050-55b0b3b9274c.mp4
|
| 34 | + |
0 commit comments