|
1 | 1 | # MediaTek Backend
|
2 | 2 |
|
3 |
| -Placeholder |
| 3 | +MediaTek backend empowers ExecuTorch to speed up PyTorch models on edge devices that equips with MediaTek Neuron Processing Unit (NPU). This document offers a step-by-step guide to set up the build environment for the MediaTek ExecuTorch libraries. |
| 4 | + |
| 5 | +::::{grid} 2 |
| 6 | +:::{grid-item-card} What you will learn in this tutorial: |
| 7 | +:class-card: card-prerequisites |
| 8 | +* How to export and lower a PyTorch model ahead of time with ExecuTorch for MediaTek devices. |
| 9 | +* How to build MediaTek backend and examples. |
| 10 | +* How to deploy the exported models on device with ExecuTorch runtime. |
| 11 | +::: |
| 12 | +:::{grid-item-card} Tutorials we recommend you complete before this: |
| 13 | +:class-card: card-prerequisites |
| 14 | +* [Introduction to ExecuTorch](intro-how-it-works.md) |
| 15 | +* [Setting up ExecuTorch](getting-started-setup.md) |
| 16 | +* [Building ExecuTorch with CMake](runtime-build-and-cross-compilation.md) |
| 17 | +::: |
| 18 | +:::: |
| 19 | + |
| 20 | + |
| 21 | +## Prerequisites (Hardware and Software) |
| 22 | + |
| 23 | +### Host OS |
| 24 | +- Linux operating system |
| 25 | + |
| 26 | +### Supported Chips: |
| 27 | +- MediaTek Dimensity 9300 (D9300) |
| 28 | + |
| 29 | +### Software: |
| 30 | + |
| 31 | +- [NeuroPilot Express SDK](https://neuropilot.mediatek.com/resources/public/npexpress/en/docs/npexpress) is a lightweight SDK for deploying AI applications on MediaTek SOC devices. |
| 32 | + |
| 33 | +## Setting up your developer environment |
| 34 | + |
| 35 | +Follow the steps below to setup your build environment: |
| 36 | + |
| 37 | +1. **Setup ExecuTorch Environment**: Refer to the [Setting up ExecuTorch](https://pytorch.org/executorch/stable/getting-started-setup) guide for detailed instructions on setting up the ExecuTorch environment. |
| 38 | + |
| 39 | +2. **Setup MediaTek Backend Environment** |
| 40 | +- Install the dependent libs. Ensure that you are inside `backends/mediatek/` directory |
| 41 | + ```bash |
| 42 | + pip3 install -r requirements.txt |
| 43 | + ``` |
| 44 | +- Install the two .whl downloaded from NeuroPilot Portal |
| 45 | + ```bash |
| 46 | + pip3 install mtk_neuron-8.2.13-py3-none-linux_x86_64.whl |
| 47 | + pip3 install mtk_converter-8.9.1+public-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl |
| 48 | + ``` |
| 49 | +- Set evironment variables for building backend |
| 50 | + ```bash |
| 51 | + export NEURON_BUFFER_ALLOCATOR_LIB=<path_to_buffer_allocator.so> |
| 52 | + ``` |
| 53 | + |
| 54 | +## Build |
| 55 | + |
| 56 | +### Ahead of time: |
| 57 | + |
| 58 | +**Exporting a PyTorch Model for MediaTek Backend**: |
| 59 | +1. Lower and export the `.pte` file for on-device execution. The export script samples are povided under `example/mediatek/`. For example, the following commnad exports the `.pte` using the scripts provided. |
| 60 | +```bash |
| 61 | +cd executorch |
| 62 | + |
| 63 | +./examples/mediatek/shell_scripts/export_oss.sh mobilenetv3 |
| 64 | +``` |
| 65 | + |
| 66 | +2. Find the `.pte` files under the directory named as same as the model. |
| 67 | + |
| 68 | +### Runtime: |
| 69 | + |
| 70 | +**Build MediaTek Backend for ExecuTorch Runtime** |
| 71 | +1. Navigate to `backends/mediatek/scripts/` directory. |
| 72 | + |
| 73 | +2. **Build MediaTek Backend**: Once the prerequisites are in place, run the `mtk_build.sh` script to start the build process: |
| 74 | + ```bash |
| 75 | + ./mtk_build.sh |
| 76 | + ``` |
| 77 | + |
| 78 | +3. MediaTek backend will be built under `cmake-android-out/backends/` as `libneuron_backend.so`. |
| 79 | + |
| 80 | +**Build a runner to execute the model on the device**: |
| 81 | +1. Build the runners and the backend by exedcuting the script: |
| 82 | +```bash |
| 83 | +./mtk_build_examples.sh |
| 84 | +``` |
| 85 | + |
| 86 | +2. The runners will be built under `cmake-android-out/examples/` |
| 87 | + |
| 88 | +## Deploying and running on a device |
| 89 | + |
| 90 | +1. **Push MediaTek universal SDK and MediaTek backend to the device**: push `libneuronusdk_adapter.mtk.so` and `libneuron_backend.so` to the phone and export it to the `$LD_LIBRARY_PATH` environment variable before executing ExecuTorch with MediaTek backend. |
| 91 | + |
| 92 | + ```bash |
| 93 | + export LD_LIBRARY_PATH=<path_to_usdk>:<path_to_neuron_backend>:$LD_LIBRARY_PATH |
| 94 | + ``` |
0 commit comments