Skip to content

Commit fa049cd

Browse files
committed
docs : add cross-compiling for Android
1 parent 49a2fd0 commit fa049cd

File tree

1 file changed

+49
-0
lines changed

1 file changed

+49
-0
lines changed

docs/android.md

Lines changed: 49 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -32,3 +32,52 @@ To see what it might look like visually, here's an old demo of an interactive se
3232

3333
https://user-images.githubusercontent.com/271616/225014776-1d567049-ad71-4ef2-b050-55b0b3b9274c.mp4
3434

35+
## Cross-compile using Android NDK
36+
It's possible to build `llama.cpp` for Android on your host system via CMake and the Android NDK. If you are interested in this path, ensure you already have an environment prepared to cross-compile programs for Android (i.e., install the Android SDK). Note that, unlike desktop environments, the Android environment ships with a limited set of native libraries, and so only those libraries are available to CMake when building with the Android NDK (see: https://developer.android.com/ndk/guides/stable_apis.)
37+
38+
Once you're ready and have cloned `llama.cpp`, invoke the following in the project directory:
39+
40+
```
41+
$ cmake \
42+
-DCMAKE_TOOLCHAIN_FILE=$ANDROID_NDK/build/cmake/android.toolchain.cmake \
43+
-DANDROID_ABI=arm64-v8a \
44+
-DANDROID_PLATFORM=android-28 \
45+
-DCMAKE_C_FLAGS="-march=armv8.7a" \
46+
-DCMAKE_CXX_FLAGS="-march=armv8.7a" \
47+
-DGGML_OPENMP=OFF \
48+
-DGGML_LLAMAFILE=OFF \
49+
-B build-android
50+
```
51+
52+
Notes:
53+
- While later versions of Android NDK ship with OpenMP, it must still be installed by CMake as a dependency, which is not supported at this time
54+
- `llamafile` does not appear to support Android devices (see: https://github.com/Mozilla-Ocho/llamafile/issues/325)
55+
56+
The above command should configure `llama.cpp` with the most performant options for modern devices. Even if your device is not running `armv8.7a`, `llama.cpp` includes runtime checks for available CPU features it can use.
57+
58+
Feel free to adjust the Android ABI for your target. Once the project is configured:
59+
60+
```
61+
$ cmake --build build-android --config Release -j{n}
62+
$ cmake --install build-android --prefix {install-dir} --config Release
63+
```
64+
65+
After installing, go ahead and download the model of your choice to your host system. Then:
66+
67+
```
68+
$ adb shell "mkdir /data/local/tmp/llama.cpp"
69+
$ adb push {install-dir} /data/local/tmp/llama.cpp/
70+
$ adb push {model}.gguf /data/local/tmp/llama.cpp/
71+
$ adb shell
72+
```
73+
74+
In the `adb shell`:
75+
76+
```
77+
$ cd /data/local/tmp/llama.cpp
78+
$ LD_LIBRARY_PATH=lib ./bin/llama-simple -m {model}.gguf -c {context-size} -p "{your-prompt}"
79+
```
80+
81+
That's it!
82+
83+
Be aware that Android will not find the library path `lib` on its own, so we must specify `LD_LIBRARY_PATH` in order to run the installed executables. Android does support `RPATH` in later API levels, so this could change in the future. Refer to the previous section for information about `context-size` (very important!) and running other `examples`.

0 commit comments

Comments
 (0)