Add documentation

backends/mediatek/scripts/README.md → backends/mediatek/README.md

@@ -1,28 +1,35 @@
-# Build Instructions
+# MediaTek Backend on ExecuTorch
+MediaTek backend empowers ExecuTorch to speed up PyTorch models on edge devices that equips with MediaTek Neuron Processing Unit (NPU). This document offers a step-by-step guide to set up the build environment for the MediaTek ExecuTorch libraries.
 
-This document provides a step-by-step guide to set up the build environment for the MediaTek ExercuTorch libraries.
+## Supported Chips
 
-## Prerequisites
+The examples provided in this repository are tested and supported on the following MediaTek chip:
+
+- MediaTek Dimensity 9300 (D9300)
+
+## Build Instructions
+
+### Prerequisites
 
 Before you begin, ensure you have the following prerequisites installed and configured:
 
-### 1. Buck2 Build Tool
+#### 1. Buck2 Build Tool
 
 - **Download Buck2**: Obtain Buck2 from the official [releases page](https://github.com/facebook/buck2/releases/tag/2024-02-01).
 - **Add to PATH**: Extract the downloaded file and add the directory to your system's `$PATH` environment variable.
    ```bash
    export PATH=<path_to_buck>:$PATH
    ```
 
-### 2. Android NDK
+#### 2. Android NDK
 
 - **Download Android NDK**: Acquire the Android NDK version 26.3.11579264 from the [Android developer site](https://developer.android.com/ndk/downloads).
 - **Set NDK Path**: Ensure that the `$ANDROID_NDK` environment variable is set to the path where the NDK is located.
    ```bash
    export ANDROID_NDK=<path_to_android_ndk>
    ```
 
-### 3. MediaTek ExercuTorch Libraries
+#### 3. MediaTek ExecuTorch Libraries
 
 Download [NeuroPilot Express SDK](https://neuropilot.mediatek.com/resources/public/npexpress/en/docs/npexpress) from MediaTek's NeuroPilot portal:
 
@@ -31,11 +38,11 @@ Download [NeuroPilot Express SDK](https://neuropilot.mediatek.com/resources/publ
 - `mtk_converter-8.8.0.dev20240723+public.d1467db9-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl`: This library preprocess the model into a MediaTek representation.
 - `mtk_neuron-8.2.2-py3-none-linux_x86_64.whl`: This library converts the model to binaries.
 
-## Setup
+### Setup
 
 Follow the steps below to setup your build environment:
 
-1. **Setup ExercuTorch Environment**: Refer to the [Setting up ExercuTorch](https://pytorch.org/executorch/stable/getting-started-setup) guide for detailed instructions on setting up the ExercuTorch environment.
+1. **Setup ExecuTorch Environment**: Refer to the [Setting up ExecuTorch](https://pytorch.org/executorch/stable/getting-started-setup) guide for detailed instructions on setting up the ExecuTorch environment.
 
 2. **Setup MediaTek Backend Environment**
 - Install the dependent libs. Ensure that you are inside backends/mediatek/ directory
@@ -52,18 +59,21 @@ Follow the steps below to setup your build environment:
    export NEURON_BUFFER_ALLOCATOR_LIB=<path_to_buffer_allocator>
    ```
 
-## Build
+### Build
+1. Navigate to `scripts/` directory.
 
-1. **Build MediaTek Backend**: Once the prerequisites are in place, run the `mtk_build.sh` script to start the build process, MediaTek backend will be built under `cmake-android-out/backends/` as `libneuron_backend.so`
+2. **Build MediaTek Backend**: Once the prerequisites are in place, run the `mtk_build.sh` script to start the build process, MediaTek backend will be built under `cmake-android-out/backends/` as `libneuron_backend.so`
 
    ```bash
    ./mtk_build.sh
    ```
 
-## Run
+### Run
 
-1. **Push MediaTek universal SDK and MediaTek backend to the device**: push `libneuronusdk_adapter.mtk.so` and `libneuron_backend.so` to the phone and export it to the `$LD_LIBRARY_PATH` environment variable before executing ExercuTorch with MediaTek backend.
+1. **Push MediaTek universal SDK and MediaTek backend to the device**: push `libneuronusdk_adapter.mtk.so` and `libneuron_backend.so` to the phone and export it to the `$LD_LIBRARY_PATH` environment variable before executing ExecuTorch with MediaTek backend.
 
    ```bash
    export LD_LIBRARY_PATH=<path_to_usdk>:<path_to_neuron_backend>:$LD_LIBRARY_PATH
    ```
+
+Please refer to `executorch/examples/mediatek/` for export and execution examples of various of models.

docs/source/build-run-mediatek-backend.md

@@ -0,0 +1,94 @@
+# Building and Running ExecuTorch with MediaTek Backend
+
+MediaTek backend empowers ExecuTorch to speed up PyTorch models on edge devices that equips with MediaTek Neuron Processing Unit (NPU). This document offers a step-by-step guide to set up the build environment for the MediaTek ExecuTorch libraries.
+
+::::{grid} 2
+:::{grid-item-card}  What you will learn in this tutorial:
+:class-card: card-prerequisites
+* How to export and lower a PyTorch model ahead of time with ExecuTorch for MediaTek devices.
+* How to build MediaTek backend and examples.
+* How to deploy the exported models on device with ExecuTorch runtime.
+:::
+:::{grid-item-card}  Tutorials we recommend you complete before this:
+:class-card: card-prerequisites
+* [Introduction to ExecuTorch](intro-how-it-works.md)
+* [Setting up ExecuTorch](getting-started-setup.md)
+* [Building ExecuTorch with CMake](runtime-build-and-cross-compilation.md)
+:::
+::::
+
+
+## Prerequisites (Hardware and Software)
+
+### Host OS
+- Linux operating system
+
+### Supported Chips:
+- MediaTek Dimensity 9300 (D9300)
+
+### Software:
+
+- [NeuroPilot Express SDK](https://neuropilot.mediatek.com/resources/public/npexpress/en/docs/npexpress) is a lightweight SDK for deploying AI applications on MediaTek SOC devices.
+
+## Setting up your developer environment
+
+Follow the steps below to setup your build environment:
+
+1. **Setup ExecuTorch Environment**: Refer to the [Setting up ExecuTorch](https://pytorch.org/executorch/stable/getting-started-setup) guide for detailed instructions on setting up the ExecuTorch environment.
+
+2. **Setup MediaTek Backend Environment**
+- Install the dependent libs. Ensure that you are inside `backends/mediatek/` directory
+   ```bash
+   pip3 install -r requirements.txt
+   ```
+- Install the two .whl downloaded from NeuroPilot Portal
+   ```bash
+   pip3 install mtk_neuron-8.2.13-py3-none-linux_x86_64.whl
+   pip3 install mtk_converter-8.9.1+public-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
+   ```
+- Set evironment variables for building backend
+   ```bash
+   export NEURON_BUFFER_ALLOCATOR_LIB=<path_to_buffer_allocator.so>
+   ```
+
+## Build
+
+### Ahead of time:
+
+**Exporting a PyTorch Model for MediaTek Backend**:
+1. Lower and export the `.pte` file for on-device execution. The export script samples are povided under `example/mediatek/`. For example, the following commnad exports the `.pte` using the scripts provided.
+```bash
+cd executorch
+
+./examples/mediatek/shell_scripts/export_oss.sh mobilenetv3
+```
+
+2. Find the `.pte` files under the directory named as same as the model.
+
+### Runtime:
+
+**Build MediaTek Backend for ExecuTorch Runtime**
+1. Navigate to `backends/mediatek/scripts/` directory.
+
+2. **Build MediaTek Backend**: Once the prerequisites are in place, run the `mtk_build.sh` script to start the build process:
+   ```bash
+   ./mtk_build.sh
+   ```
+
+3. MediaTek backend will be built under `cmake-android-out/backends/` as `libneuron_backend.so`.
+
+**Build a runner to execute the model on the device**:
+1. Build the runners and the backend by exedcuting the script:
+```bash
+./mtk_build_examples.sh
+```
+
+2. The runners will be built under `cmake-android-out/examples/`
+
+## Deploying and running on a device
+
+1. **Push MediaTek universal SDK and MediaTek backend to the device**: push `libneuronusdk_adapter.mtk.so` and `libneuron_backend.so` to the phone and export it to the `$LD_LIBRARY_PATH` environment variable before executing ExecuTorch with MediaTek backend.
+
+   ```bash
+   export LD_LIBRARY_PATH=<path_to_usdk>:<path_to_neuron_backend>:$LD_LIBRARY_PATH
+   ```

docs/source/index.rst

@@ -111,6 +111,7 @@ Topics in this section will help you get started with ExecuTorch.
       customcarditem entries below.
    executorch-arm-delegate-tutorial
    build-run-coreml
+   build-run-mediatek-backend
    build-run-mps
    build-run-qualcomm-ai-engine-direct-backend
    build-run-xtensa
@@ -331,6 +332,13 @@ ExecuTorch tutorials.
    :link: build-run-coreml.html
    :tags: Export,Backend,Delegation,CoreML
 
+.. customcarditem::
+   :header: Building and Running ExecuTorch with MediaTek Backend
+   :card_description: A tutorial that walks you through the process of building ExecuTorch with MediaTek Backend
+   :image: _static/img/generic-pytorch-logo.png
+   :link: build-run-mediatek-backend.html
+   :tags: Export,Backend,Delegation,MediaTek
+
 .. customcarditem::
    :header: Building and Running ExecuTorch with MPS Backend
    :card_description: A tutorial that walks you through the process of building ExecuTorch with MPSGraph Backend

examples/mediatek/README.md

@@ -72,12 +72,6 @@ bash shell_scripts/export_oss.sh <model_name>
     - `model_name`: deeplabv3/edsr/inceptionv3/inceptionv4/mobilenetv2/mobilenetv3/resnet18/resnet50
 
 # Runtime
-## Supported Chips
-
-The examples provided in this repository are tested and supported on the following MediaTek chip:
-
-- MediaTek Dimensity 9300 (D9300)
-
 ## Environment Setup
 
 To set up the build environment for the `mtk_executor_runner`: