Skip to content

Commit 95b0daa

Browse files
kirklandsignfacebook-github-bot
authored andcommitted
Fix Android docs and add gradlew helper (#2751)
Summary: Pull Request resolved: #2751 Test Plan: https://docs-preview.pytorch.org/pytorch/executorch/2751/llm/llama-demo-android.html Reviewed By: shoumikhin Differential Revision: D55492241 Pulled By: kirklandsign fbshipit-source-id: 84a2aad3474ba0c8dcfbae515221cef8506f1648
1 parent 45c2557 commit 95b0daa

File tree

6 files changed

+238
-29
lines changed

6 files changed

+238
-29
lines changed

docs/source/llm/getting-started.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -32,3 +32,5 @@ In some cases, it is necessary to write custom kernels or import them from anoth
3232
## How to build Mobile Apps
3333

3434
Here's how to finally build a mobile app on Android and iOS. TODO
35+
36+
For Android demo app build, please see [this tutorial](./llama-demo-android.md).

docs/source/llm/llama-demo-android.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
```{include} ../../../examples/demo-apps/android/LlamaDemo/README.md
2+
```
Lines changed: 48 additions & 24 deletions
Original file line numberDiff line numberDiff line change
@@ -1,68 +1,83 @@
11
# Welcome to the ExecuTorch LLaMA Android Demo App
22

3-
This app demonstrates the use of the LLaMA local inference use case with ExecuTorch.
3+
This app demonstrates the use of the LLaMA chat app demonstrating local inference use case with ExecuTorch.
44

5+
::::{grid} 2
6+
:::{grid-item-card} What you will learn
7+
:class-card: card-prerequisites
8+
* How to set up a build target for Android arm64-v8a
9+
* How to build the required ExecuTorch runtime, LLaMA runner, and JNI wrapper for Android
10+
* How to build the app with required JNI library
11+
:::
12+
:::{grid-item-card} Prerequisites
13+
:class-card: card-prerequisites
14+
* Refer to [Setting up ExecuTorch](https://pytorch.org/executorch/stable/getting-started-setup) to set up the repo and dev environment.
15+
* Download and install [Android Studio and SDK](https://developer.android.com/studio).
16+
* Supported Host OS: CentOS, macOS Sonoma on Apple Silicon.
17+
:::
18+
::::
519

6-
## Getting models
20+
```{note}
21+
This demo app and tutorial has only been validated with arm64-v8a [ABI](https://developer.android.com/ndk/guides/abis), with NDK 25.
22+
```
723

8-
Please refer to the LLaMA tutorial to generate the model.
24+
## Getting models
25+
Please refer to the LLaMA tutorial to export the model.
926

10-
Once you have the model, you can push it to your device:
11-
```
27+
Once you have the model, you need to push it to your device:
28+
```bash
1229
adb shell mkdir -p /data/local/tmp/llama
1330
adb push model.pte /data/local/tmp/llama
1431
adb push tokenizer.bin /data/local/tmp/llama
1532
```
1633

34+
```{note}
1735
The demo app searches in `/data/local/tmp/llama` for .pte and .bin files as LLAMA model and tokenizer.
36+
```
1837

1938
## Build JNI library
2039
1. Open a terminal window and navigate to the root directory of the `executorch`.
2140
2. Set the following environment variables:
2241
```
23-
export BUCK=<path_to_buck>
2442
export ANDROID_NDK=<path_to_android_ndk>
25-
export PYTHON_EXECUTABLE=<path_to_python_executable>
26-
export ANDROID_ABI=<abi_type> # ex: arm64-v8a
27-
export FLATC=<path_to_flatc_executable>
43+
export ANDROID_ABI=arm64-v8a
2844
```
2945
3. Create a new directory for the CMake build output:
3046
```
3147
mkdir cmake-out
32-
pushd cmake-out
3348
```
3449
4. Run the following command to configure the CMake build:
3550
```
36-
cmake .. -DCMAKE_INSTALL_PREFIX=cmake-out \
51+
# Build the core ExecuTorch runtime library
52+
cmake . -DCMAKE_INSTALL_PREFIX=cmake-out \
3753
-DCMAKE_TOOLCHAIN_FILE="${ANDROID_NDK}/build/cmake/android.toolchain.cmake" \
3854
-DANDROID_ABI="${ANDROID_ABI}" \
39-
-DBUCK2="${BUCK2}" \
4055
-DEXECUTORCH_BUILD_XNNPACK=ON \
41-
-DEXECUTORCH_BUILD_FLATC=OFF \
4256
-DEXECUTORCH_BUILD_EXTENSION_DATA_LOADER=ON \
43-
-DFLATC_EXECUTABLE="${FLATC}" \
44-
-DEXECUTORCH_BUILD_EXTENSION_MODULE=ON
57+
-DEXECUTORCH_BUILD_EXTENSION_MODULE=ON \
58+
-DEXECUTORCH_BUILD_OPTIMIZED=ON \
59+
-Bcmake-out
4560
46-
cmake --build . -j16 --target install
61+
cmake --build cmake-out -j16 --target install
4762
48-
cmake ../examples/models/llama2 -DBUCK2="$BUCK" \
63+
# Build the llama2 runner library and custom ops
64+
cmake examples/models/llama2 \
4965
-DCMAKE_TOOLCHAIN_FILE=$ANDROID_NDK/build/cmake/android.toolchain.cmake \
5066
-DANDROID_ABI="$ANDROID_ABI" \
5167
-DCMAKE_INSTALL_PREFIX=cmake-out \
52-
-Bexamples/models/llama2
68+
-Bcmake-out/examples/models/llama2
5369
54-
cmake --build examples/models/llama2 -j16
70+
cmake --build cmake-out/examples/models/llama2 -j16
5571
56-
cmake ../extension/android -DBUCK2="${BUCK2}" \
72+
# Build the Android JNI library
73+
cmake extension/android \
5774
-DCMAKE_TOOLCHAIN_FILE=${ANDROID_NDK}/build/cmake/android.toolchain.cmake \
5875
-DANDROID_ABI="${ANDROID_ABI}" \
5976
-DCMAKE_INSTALL_PREFIX=cmake-out \
6077
-DEXECUTORCH_BUILD_LLAMA_JNI=ON \
61-
-Bextension/android
78+
-Bcmake-out/extension/android
6279
63-
cmake --build extension/android -j16
64-
65-
popd
80+
cmake --build cmake-out/extension/android -j16
6681
```
6782

6883
5. Copy the built library to your app:
@@ -75,3 +90,12 @@ cp cmake-out/extension/android/libexecutorch_llama_jni.so "${JNI_LIBS_PATH}/${AN
7590
## Build Java app
7691
1. Open Android Studio and select "Open an existing Android Studio project" to open examples/demo-apps/android/LlamaDemo.
7792
2. Run the app (^R).
93+
94+
On the phone or emulator, you can try running the model:
95+
<img src="../_static/img/android_llama_app.png" alt="Android LLaMA App" /><br>
96+
97+
## Takeaways
98+
Through this tutorial we've learnt how to build the ExecuTorch LLAMA library with XNNPACK backend, and expose it to JNI layer to build the Android app.
99+
100+
## Reporting Issues
101+
If you encountered any bugs or issues following this tutorial please file a bug/issue here on [Github](https://github.com/pytorch/executorch/issues/new).
Lines changed: 185 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,185 @@
1+
#!/usr/bin/env sh
2+
3+
#
4+
# Copyright 2015 the original author or authors.
5+
#
6+
# Licensed under the Apache License, Version 2.0 (the "License");
7+
# you may not use this file except in compliance with the License.
8+
# You may obtain a copy of the License at
9+
#
10+
# https://www.apache.org/licenses/LICENSE-2.0
11+
#
12+
# Unless required by applicable law or agreed to in writing, software
13+
# distributed under the License is distributed on an "AS IS" BASIS,
14+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
15+
# See the License for the specific language governing permissions and
16+
# limitations under the License.
17+
#
18+
19+
##############################################################################
20+
##
21+
## Gradle start up script for UN*X
22+
##
23+
##############################################################################
24+
25+
# Attempt to set APP_HOME
26+
# Resolve links: $0 may be a link
27+
PRG="$0"
28+
# Need this for relative symlinks.
29+
while [ -h "$PRG" ] ; do
30+
ls=`ls -ld "$PRG"`
31+
link=`expr "$ls" : '.*-> \(.*\)$'`
32+
if expr "$link" : '/.*' > /dev/null; then
33+
PRG="$link"
34+
else
35+
PRG=`dirname "$PRG"`"/$link"
36+
fi
37+
done
38+
SAVED="`pwd`"
39+
cd "`dirname \"$PRG\"`/" >/dev/null
40+
APP_HOME="`pwd -P`"
41+
cd "$SAVED" >/dev/null
42+
43+
APP_NAME="Gradle"
44+
APP_BASE_NAME=`basename "$0"`
45+
46+
# Add default JVM options here. You can also use JAVA_OPTS and GRADLE_OPTS to pass JVM options to this script.
47+
DEFAULT_JVM_OPTS='"-Xmx64m" "-Xms64m"'
48+
49+
# Use the maximum available, or set MAX_FD != -1 to use that value.
50+
MAX_FD="maximum"
51+
52+
warn () {
53+
echo "$*"
54+
}
55+
56+
die () {
57+
echo
58+
echo "$*"
59+
echo
60+
exit 1
61+
}
62+
63+
# OS specific support (must be 'true' or 'false').
64+
cygwin=false
65+
msys=false
66+
darwin=false
67+
nonstop=false
68+
case "`uname`" in
69+
CYGWIN* )
70+
cygwin=true
71+
;;
72+
Darwin* )
73+
darwin=true
74+
;;
75+
MINGW* )
76+
msys=true
77+
;;
78+
NONSTOP* )
79+
nonstop=true
80+
;;
81+
esac
82+
83+
CLASSPATH=$APP_HOME/gradle/wrapper/gradle-wrapper.jar
84+
85+
86+
# Determine the Java command to use to start the JVM.
87+
if [ -n "$JAVA_HOME" ] ; then
88+
if [ -x "$JAVA_HOME/jre/sh/java" ] ; then
89+
# IBM's JDK on AIX uses strange locations for the executables
90+
JAVACMD="$JAVA_HOME/jre/sh/java"
91+
else
92+
JAVACMD="$JAVA_HOME/bin/java"
93+
fi
94+
if [ ! -x "$JAVACMD" ] ; then
95+
die "ERROR: JAVA_HOME is set to an invalid directory: $JAVA_HOME
96+
97+
Please set the JAVA_HOME variable in your environment to match the
98+
location of your Java installation."
99+
fi
100+
else
101+
JAVACMD="java"
102+
which java >/dev/null 2>&1 || die "ERROR: JAVA_HOME is not set and no 'java' command could be found in your PATH.
103+
104+
Please set the JAVA_HOME variable in your environment to match the
105+
location of your Java installation."
106+
fi
107+
108+
# Increase the maximum file descriptors if we can.
109+
if [ "$cygwin" = "false" -a "$darwin" = "false" -a "$nonstop" = "false" ] ; then
110+
MAX_FD_LIMIT=`ulimit -H -n`
111+
if [ $? -eq 0 ] ; then
112+
if [ "$MAX_FD" = "maximum" -o "$MAX_FD" = "max" ] ; then
113+
MAX_FD="$MAX_FD_LIMIT"
114+
fi
115+
ulimit -n $MAX_FD
116+
if [ $? -ne 0 ] ; then
117+
warn "Could not set maximum file descriptor limit: $MAX_FD"
118+
fi
119+
else
120+
warn "Could not query maximum file descriptor limit: $MAX_FD_LIMIT"
121+
fi
122+
fi
123+
124+
# For Darwin, add options to specify how the application appears in the dock
125+
if $darwin; then
126+
GRADLE_OPTS="$GRADLE_OPTS \"-Xdock:name=$APP_NAME\" \"-Xdock:icon=$APP_HOME/media/gradle.icns\""
127+
fi
128+
129+
# For Cygwin or MSYS, switch paths to Windows format before running java
130+
if [ "$cygwin" = "true" -o "$msys" = "true" ] ; then
131+
APP_HOME=`cygpath --path --mixed "$APP_HOME"`
132+
CLASSPATH=`cygpath --path --mixed "$CLASSPATH"`
133+
134+
JAVACMD=`cygpath --unix "$JAVACMD"`
135+
136+
# We build the pattern for arguments to be converted via cygpath
137+
ROOTDIRSRAW=`find -L / -maxdepth 1 -mindepth 1 -type d 2>/dev/null`
138+
SEP=""
139+
for dir in $ROOTDIRSRAW ; do
140+
ROOTDIRS="$ROOTDIRS$SEP$dir"
141+
SEP="|"
142+
done
143+
OURCYGPATTERN="(^($ROOTDIRS))"
144+
# Add a user-defined pattern to the cygpath arguments
145+
if [ "$GRADLE_CYGPATTERN" != "" ] ; then
146+
OURCYGPATTERN="$OURCYGPATTERN|($GRADLE_CYGPATTERN)"
147+
fi
148+
# Now convert the arguments - kludge to limit ourselves to /bin/sh
149+
i=0
150+
for arg in "$@" ; do
151+
CHECK=`echo "$arg"|egrep -c "$OURCYGPATTERN" -`
152+
CHECK2=`echo "$arg"|egrep -c "^-"` ### Determine if an option
153+
154+
if [ $CHECK -ne 0 ] && [ $CHECK2 -eq 0 ] ; then ### Added a condition
155+
eval `echo args$i`=`cygpath --path --ignore --mixed "$arg"`
156+
else
157+
eval `echo args$i`="\"$arg\""
158+
fi
159+
i=`expr $i + 1`
160+
done
161+
case $i in
162+
0) set -- ;;
163+
1) set -- "$args0" ;;
164+
2) set -- "$args0" "$args1" ;;
165+
3) set -- "$args0" "$args1" "$args2" ;;
166+
4) set -- "$args0" "$args1" "$args2" "$args3" ;;
167+
5) set -- "$args0" "$args1" "$args2" "$args3" "$args4" ;;
168+
6) set -- "$args0" "$args1" "$args2" "$args3" "$args4" "$args5" ;;
169+
7) set -- "$args0" "$args1" "$args2" "$args3" "$args4" "$args5" "$args6" ;;
170+
8) set -- "$args0" "$args1" "$args2" "$args3" "$args4" "$args5" "$args6" "$args7" ;;
171+
9) set -- "$args0" "$args1" "$args2" "$args3" "$args4" "$args5" "$args6" "$args7" "$args8" ;;
172+
esac
173+
fi
174+
175+
# Escape application args
176+
save () {
177+
for i do printf %s\\n "$i" | sed "s/'/'\\\\''/g;1s/^/'/;\$s/\$/' \\\\/" ; done
178+
echo " "
179+
}
180+
APP_ARGS=`save "$@"`
181+
182+
# Collect all arguments for the java command, following the shell quoting and substitution rules
183+
eval set -- $DEFAULT_JVM_OPTS $JAVA_OPTS $GRADLE_OPTS "\"-Dorg.gradle.appname=$APP_BASE_NAME\"" -classpath "\"$CLASSPATH\"" org.gradle.wrapper.GradleWrapperMain "$APP_ARGS"
184+
185+
exec "$JAVACMD" "$@"

examples/demo-apps/android/LlamaDemo/setup.sh

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -18,6 +18,7 @@ cmake . -DCMAKE_INSTALL_PREFIX="${CMAKE_OUT}" \
1818
-DEXECUTORCH_BUILD_EXTENSION_DATA_LOADER=ON \
1919
-DFLATC_EXECUTABLE="${FLATC}" \
2020
-DEXECUTORCH_BUILD_EXTENSION_MODULE=ON \
21+
-DEXECUTORCH_BUILD_OPTIMIZED=ON \
2122
-B"${CMAKE_OUT}"
2223

2324
if [ "$(uname)" == "Darwin" ]; then

extension/android/build.gradle

Lines changed: 0 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -16,14 +16,9 @@ repositories {
1616
mavenCentral()
1717
}
1818

19-
2019
task makeJar(type: Jar) {
21-
compileJava {
22-
excludes = ['**/TensorImageUtils.java']
23-
}
2420
dependencies {
2521
implementation 'com.facebook.fbjni:fbjni-java-only:0.2.2'
2622
implementation 'com.facebook.soloader:nativeloader:0.10.5'
2723
}
2824
}
29-

0 commit comments

Comments
 (0)