Skip to content

Commit fc2b435

Browse files
committed
docs : add cross-compiling for Android
1 parent f5aaf75 commit fc2b435

File tree

1 file changed

+46
-0
lines changed

1 file changed

+46
-0
lines changed

docs/android.md

Lines changed: 46 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -32,3 +32,49 @@ To see what it might look like visually, here's an old demo of an interactive se
3232

3333
https://user-images.githubusercontent.com/271616/225014776-1d567049-ad71-4ef2-b050-55b0b3b9274c.mp4
3434

35+
## Cross-compile using Android NDK
36+
It's possible to build `llama.cpp` for Android on your host system via CMake and the Android NDK. If you are interested in this path, ensure you already have an environment prepared to cross-compile programs for Android (i.e., install the Android SDK). Note that, unlike desktop environments, the Android environment ships with a limited set of native libraries, and so only those libraries are available to CMake when building with the Android NDK (see: https://developer.android.com/ndk/guides/stable_apis.)
37+
38+
Once you're ready and have cloned `llama.cpp`, invoke the following in the project directory:
39+
40+
```
41+
$ cmake \
42+
-DCMAKE_TOOLCHAIN_FILE=$ANDROID_NDK/build/cmake/android.toolchain.cmake \
43+
-DANDROID_ABI=arm64-v8a \
44+
-DANDROID_PLATFORM=android-23 \
45+
-DGGML_OPENMP=OFF \
46+
-DGGML_LLAMAFILE=OFF \
47+
-B build-android
48+
```
49+
50+
Notes:
51+
- Android API level 23 is required (see: https://github.com/abetlen/llama-cpp-python/issues/1284)
52+
- Android does not ship with OpenMP
53+
- `llamafile` does not support Android (see: https://github.com/Mozilla-Ocho/llamafile/issues/325)
54+
55+
Feel free to adjust the Android ABI according to your needs. Once the project is configured:
56+
57+
```
58+
$ cmake --build build-android --config Release -j{n}
59+
$ cmake --install build-android --prefix {install-dir} --config Release
60+
```
61+
62+
After installing, go ahead and download the model of your choice to your host system. Then:
63+
64+
```
65+
$ adb shell "mkdir /data/local/tmp/llama.cpp"
66+
$ adb push {install-dir} /data/local/tmp/llama.cpp/
67+
$ adb push {model}.gguf /data/local/tmp/llama.cpp/
68+
$ adb shell
69+
```
70+
71+
In the `adb shell`:
72+
73+
```
74+
$ cd /data/local/tmp/llama.cpp
75+
$ LD_LIBRARY_PATH=lib ./bin/llama-simple -m {model}.gguf -c {context-size} -p "{your-prompt}"
76+
```
77+
78+
That's it!
79+
80+
Be aware that Android will not find the library path `lib` on its own and does not support `RPATH`, so we must specify `LD_LIBRARY_PATH` in order to run the installed executables. Refer to the previous section for information about `context-size` (very important!) and running other `examples`.

0 commit comments

Comments
 (0)