Skip to content

Commit 243fffc

Browse files
digantdesaifacebook-github-bot
digantdesai
authored and
facebook-github-bot
committed
Update executorch-arm-delegate-tutorial.md (pytorch#6040)
Summary: Removing code snippets, given they will get quickly outdated. Instead referring directly to the functions in the `setup.sh` and `run.sh`. We can figure out a way to do some better automation but since what we had was outdated I feel this is the right thing to do in the short term. Pull Request resolved: pytorch#6040 Reviewed By: mergennachin Differential Revision: D64113182 Pulled By: digantdesai fbshipit-source-id: 1c195afc7a26c062f6db8fc44d254c80232a761b
1 parent 9364c06 commit 243fffc

File tree

1 file changed

+14
-173
lines changed

1 file changed

+14
-173
lines changed

‎docs/source/executorch-arm-delegate-tutorial.md

Lines changed: 14 additions & 173 deletions
Original file line numberDiff line numberDiff line change
@@ -48,14 +48,9 @@ In the following sections we will walk through the steps to download each of the
4848

4949
## Set Up the Developer Environment
5050

51-
In this section, we will do a one-time setup, like downloading and installing necessary software, for the platform support files needed to run ExecuTorch programs in this tutorial. There are two approaches available:
51+
In this section, we will do a one-time setup, like downloading and installing necessary software, for the platform support files needed to run ExecuTorch programs in this tutorial.
5252

53-
1. Method 1: Use the `examples/arm/setup.sh` script to pull each item in an automated fashion (recommended). It is recommended to run the script in a conda environment. Upon successful execution, you can directly go to [the next step](#convert-the-pytorch-model-to-the-pte-file).
54-
2. Method 2: Follow the guide step by step to understand all the components and the logic of the script. You may want to use this method if you intend to change the behavior of the flow significantly.
55-
56-
```{tip}
57-
In the ExecuTorch repository we have a functioning script which follows the exact same steps to speed things up. It is located at `examples/arm/setup.sh`. Feel free to use that instead if it is convenient, or use it as a reference if some of the steps in the manual instruction aren't very clear.
58-
```
53+
For that we will use the `examples/arm/setup.sh` script to pull each item in an automated fashion. It is recommended to run the script in a conda environment. Upon successful execution, you can directly go to [the next step](#convert-the-pytorch-model-to-the-pte-file).
5954

6055
As mentioned before, we currently support only Linux based platforms with x86_64 or aarch64 processor architecture. Let’s make sure we are indeed on a supported platform.
6156

@@ -67,7 +62,7 @@ uname -m
6762
# x86_64 or aarch64
6863
```
6964

70-
Let's create an empty directory, and use this as a top level development directory.
65+
Next we will walk through the steps performed by the `setup.sh` script to better understand the development setup.
7166

7267
### Download and Set Up the Corstone-300 FVP
7368

@@ -77,126 +72,25 @@ Fixed Virtual Platforms (FVPs) are pre-configured, functionally accurate simulat
7772
By downloading and running the FVP software, you will be agreeing to the FVP [End-user license agreement (EULA)](https://developer.arm.com/downloads/-/arm-ecosystem-fvps/eula).
7873
```
7974

80-
To download, we can either download `Corstone-300 Ecosystem FVP` from [here](https://developer.arm.com/downloads/-/arm-ecosystem-fvps). Alternatively, you can download the same version we tested with like this,
81-
82-
```bash
83-
# for aarch64
84-
curl \
85-
--output FVP_cs300.tgz \
86-
'https://developer.arm.com/-/media/Arm%20Developer%20Community/Downloads/OSS/FVP/Corstone-300/FVP_Corstone_SSE-300_11.22_35_Linux64_armv8l.tgz?rev=b083dc5ac9c546899fbb7ccd67b74c17&hash=BFE589289ECF12B07192636382C15C01'
87-
88-
# for x86_64
89-
curl \
90-
--output FVP_cs300.tgz \
91-
'https://developer.arm.com/-/media/Arm%20Developer%20Community/Downloads/OSS/FVP/Corstone-300/FVP_Corstone_SSE-300_11.22_20_Linux64.tgz?rev=018659bd574f4e7b95fa647e7836ccf4&hash=22A79103C6FA5FFA7AFF3BE0447F3FF9'
92-
```
93-
94-
Now, extract the `FVP_cs300.tgz` file in a new dir, and run the provided script which will install the FVP.
95-
96-
```bash
97-
./FVP_Corstone_SSE-300.sh \
98-
--i-agree-to-the-contained-eula \
99-
--force \
100-
--destination ./ \
101-
--quiet \
102-
--no-interactive
103-
```
104-
105-
Once successful, let's make sure the FVP simulator is available on the PATH for later use.
106-
107-
```bash
108-
# for x86-64 hosts
109-
export PATH=${PATH}:<install_dir>/FVP/models/Linux64_GCC-9.3
110-
# for aarch64 hosts
111-
export PATH=${PATH}:<install_dir>/FVP/models/Linux64_armv8l_GCC-9.3/
112-
113-
hash FVP_Corstone_SSE-300_Ethos-U55 # To make sure we are ready to use
114-
```
75+
To download, we can either download `Corstone-300 Ecosystem FVP` from [here](https://developer.arm.com/downloads/-/arm-ecosystem-fvps). or `setup.sh` script will does that for you under `setup_fvp` function.
11576

11677
### Download and Install the Arm GNU AArch32 Bare-Metal Toolchain
11778

11879
Similar to the FVP, we would also need a tool-chain to cross-compile ExecuTorch runtime, executor-runner bare-metal application, as well as the rest of the bare-metal stack for Cortex-M55 CPU available on the Corstone-300 platform.
11980

120-
These toolchains are available [here](https://developer.arm.com/downloads/-/arm-gnu-toolchain-downloads). We will be using GCC 12.3 targeting `arm-none-eabi` here for our tutorial. Just like FVP, to download the same version as we tested with in the top-level development dir,
121-
122-
```bash
123-
# for aarch64
124-
curl \
125-
--output gcc.tar.xz \
126-
'https://armkeil.blob.core.windows.net/developer/Files/downloads/gnu/12.3.rel1/binrel/arm-gnu-toolchain-12.3.rel1-aarch64-arm-none-eabi.tar.xz'
127-
128-
# for x86_64
129-
curl \
130-
--output gcc.tar.xz \
131-
'https://armkeil.blob.core.windows.net/developer/Files/downloads/gnu/12.3.rel1/binrel/arm-gnu-toolchain-12.3.rel1-x86_64-arm-none-eabi.tar.xz'
132-
```
133-
134-
Once downloaded, you can extract its contents in a new dir. Then, let's make sure the toolchain is available on the PATH for later use.
135-
136-
```bash
137-
export PATH=${PATH}:/<install_dir>/arm-gnu-toolchain-12.3.rel1-x86_64-arm-none-eabi/bin
138-
export PATH=${PATH}:/<install_dir>/arm-gnu-toolchain-12.3.rel1-aarch64-arm-none-eabi/bin
139-
140-
hash arm-none-eabi-gcc # To make sure we are ready to use
141-
```
81+
These toolchains are available [here](https://developer.arm.com/downloads/-/arm-gnu-toolchain-downloads). We will be using GCC 12.3 targeting `arm-none-eabi` here for our tutorial. Just like FVP, `setup.sh` script will down the toolchain for you. See `setup_toolchain` function.
14282

14383
### Setup the Arm Ethos-U Software Development
14484

145-
This git repository is the root directory for all Arm Ethos-U software. It is to help us download required repositories and place them in a tree structure. In the top-level devlopment dir,
146-
147-
```bash
148-
# Download the repo
149-
git clone https://review.mlplatform.org/ml/ethos-u/ethos-u
150-
cd ethos-u
151-
152-
# To align with the version we have tested
153-
git reset --hard 0995223100e3da8011700f58e491f1bf59511e3c
154-
155-
# Download the necessary repos and properly install them
156-
./fetch_externals.py fetch
157-
158-
# Download the Vela compiler
159-
cd .. # To the top-level development dir
160-
git clone https://review.mlplatform.org/ml/ethos-u/ethos-u-vela
161-
```
85+
This git repository is the root directory for all Arm Ethos-U software. It is to help us download required repositories and place them in a tree structure. See `setup_ethos_u` function of the setup script for more details.
16286

16387
Once this is done, you should have a working FVP simulator, a functioning toolchain for cross compilation, and the Ethos-U software development setup ready for the bare-metal developement.
16488

165-
#### Applying Local Patches
166-
Since this is under active development, we have some patches for the Arm Ethos-u software development kit. Let's apply them on the download SDK and the Vela compiler.
167-
168-
```bash
169-
cd ethos-u # this is the top level Ethos-U software directory
170-
171-
# Let's patch core_platform repo
172-
cd core_platform
173-
git reset --hard 204210b1074071532627da9dc69950d058a809f4
174-
git am -3 <path_to>/executorch/examples/arm/ethos-u-setup/core_platform/patches/*.patch
175-
cd ../.. # To the top-level development dir
176-
```
177-
17889
### Install the Vela Compiler
179-
Once the patching is done, let's finish the setup by installing the Vela compiler.
180-
181-
```bash
182-
cd ethos-u-vela
183-
pip install .
184-
```
90+
Once this is done, the script will finish the setup by installing the Vela compiler for you, details are in `setup_vela` function.
18591

18692
### Install the TOSA reference model
187-
```bash
188-
git clone https://review.mlplatform.org/tosa/reference_model -b v0.80
189-
cd reference_model
190-
git submodule update --init --recursive
191-
mkdir -p build
192-
cd build
193-
cmake ..
194-
n=$(nproc)
195-
make -j"$((n - 5))"
196-
cd reference_model # Within the build directory
197-
# Add tosa_reference_model to the path
198-
export PATH=${PATH}:`pwd`
199-
```
93+
This is the last step of the setup process, using `setup_tosa_reference_model` function `setup.sh` script will install TOSA reference model for you.
20094

20195
At the end of the setup, if everything goes well, your top level devlopement dir might look something like this,
20296

@@ -334,7 +228,7 @@ Before generating the `.pte` file for delegated quantized networks like MobileNe
334228
SITE_PACKAGES="$(python3 -c 'from distutils.sysconfig import get_python_lib; print(get_python_lib())')"
335229
CMAKE_PREFIX_PATH="${SITE_PACKAGES}/torch"
336230

337-
cd $et_root_dir
231+
cd <executorch_root_dir>
338232
mkdir -p cmake-out-aot-lib
339233
cmake -DCMAKE_BUILD_TYPE=Release \
340234
-DEXECUTORCH_BUILD_XNNPACK=OFF \
@@ -367,11 +261,7 @@ Now let's try to run these `.pte` files on a Corstone-300 platform in a bare-met
367261

368262
## Getting a Bare-Metal Executable
369263

370-
In this section, we will go over steps that you need to go through to build the runtime application. This then run on the target device.
371-
372-
```{tip}
373-
In the executorch repository we have a functioning script which does the exact same steps. It is located at `executorch/examples/arm/run.sh`. Feel free to use that instead if it is convenient, or use it as a reference if some of the steps in the manual instruction aren't very clear.
374-
```
264+
In this section, we will go over steps that you need to go through to build the runtime application. This then run on the target device. In the executorch repository we have a functioning script which does the exact same steps. It is located at `executorch/examples/arm/run.sh`. We will use that to build necessary pieces and finally run the previously generated PTE file on an FVP.
375265

376266
Also before we get started, make sure that you have completed ExecuTorch cmake build setup, and the instructions to setup the development environment described [earlier](#set-up-the-developer-environment).
377267

@@ -394,68 +284,19 @@ To run a `.pte` file with the Arm backend delegate call instructions, we will ne
394284
- `libexecutorch_delegate_ethos_u.a`
395285

396286

397-
To generate these libraries, use following commands,
287+
These libraries are generated in `build_executorch` function of the `run.sh` script.
398288

399-
```bash
400-
# Empty and already created
401-
cd <executorch_source_root_dir>
402-
403-
# Use provided cmake toolchain for bare-metal builds
404-
toolchain_cmake=<executorch_source_root_dir>/examples/arm/ethos-u-setup/arm-none-eabi-gcc.cmake
405-
406-
cmake \
407-
-DCMAKE_INSTALL_PREFIX=<executorch_build_dir> \
408-
-DEXECUTORCH_BUILD_EXECUTOR_RUNNER=OFF \
409-
-DCMAKE_BUILD_TYPE=Release \
410-
-DEXECUTORCH_ENABLE_LOGGING=ON \
411-
-DEXECUTORCH_BUILD_ARM_BAREMETAL=ON \
412-
-DEXECUTORCH_BUILD_EXTENSION_RUNNER_UTIL=ON \
413-
-DFLATC_EXECUTABLE="$(which flatc)" \
414-
-DCMAKE_TOOLCHAIN_FILE="${toolchain_cmake}" \
415-
-B<executorch_build_dir> \
416-
<executorch_source_root_dir>
417-
418-
cmake --build <executorch_build_dir> --target install --config Release
419-
420-
cmake \
421-
-DCMAKE_INSTALL_PREFIX=<executorch_build_dir> \
422-
-DCMAKE_BUILD_TYPE=Release \
423-
-DEXECUTORCH_SELECT_OPS_LIST="aten::_softmax.out" \
424-
-DCMAKE_TOOLCHAIN_FILE="${toolchain_cmake}" \
425-
-B<executorch_build_dir>/examples/arm \
426-
<executorch_source_root_dir>/examples/arm
427-
428-
cmake --build <executorch_build_dir>/examples/arm --config Release
429-
430-
```
289+
In this function, `EXECUTORCH_SELECT_OPS_LIST` will decide the number of portable operators included in the build and are available at runtime. It must match with `.pte` file's requirements, otherwise you will get `Missing Operator` error at runtime.
431290

432-
`EXECUTORCH_SELECT_OPS_LIST` will decide the number of portable operators included in the build and are available at runtime. It must match with `.pte` file's requirements, otherwise you will get `Missing Operator` error at runtime.
433-
434-
For example, here in the command line above, to run SoftmaxModule, we only included the softmax CPU operator. Similarly, to run AddModule in a non-delegated manner you will need add op and so on. As you might have already realized, for the delegated operators, which will be executed by the Arm backend delegate, we do not need to include those operators in this list. This is only for *non-delegated* operators.
291+
For example, there in the command line above, to run SoftmaxModule, we only included the softmax CPU operator. Similarly, to run AddModule in a non-delegated manner you will need add op and so on. As you might have already realized, for the delegated operators, which will be executed by the Arm backend delegate, we do not need to include those operators in this list. This is only for *non-delegated* operators.
435292

436293
### Building the executor_runner Bare-Metal Application
437294

438295
The SDK dir is the same one prepared [earlier](#setup-the-arm-ethos-u-software-development). And, we will be passing the `.pte` file (any one of them) generated above.
439296

440297
Note, you have to generate a new `executor-runner` binary if you want to change the model or the `.pte` file. This constraint is from the constrained bare-metal runtime environment we have for Corstone-300 platform.
441298

442-
```bash
443-
444-
cd <executorch_source_root_dir>
445-
cd examples/arm/executor_runner
446-
447-
cmake \
448-
-DCMAKE_TOOLCHAIN_FILE="${toolchain_cmake}" \
449-
-DTARGET_CPU=cortex-m55 \
450-
-B build \
451-
-DETHOS_SDK_PATH:PATH=<ethos-u_clone_directory> \
452-
-DET_DIR_PATH:PATH=<executorch_source_root_dir> \
453-
-DET_BUILD_DIR_PATH:PATH=<executorch_build_dir> \
454-
-DET_PTE_FILE_PATH:PATH=<path_to_pte_file_of_choice> \
455-
-DPYTHON_EXECUTABLE=$(which python3)
456-
457-
cmake --build build -- arm_executor_runner
458-
```
299+
This is performed by the `build_executorch_runner` function in `run.sh`.
459300

460301
## Running on Corstone-300 FVP Platform
461302

0 commit comments

Comments
 (0)