Skip to content

Commit 35db16d

Browse files
guangy10facebook-github-bot
authored andcommitted
Fix xnnpack demo (#122)
Summary: Pull Request resolved: #122 Fixed several issues with the XNNPACK demo: - broken buck build in OSS (see errors in P813564245) - `buck2 build //examples/backend:xnn_executor_runner` doesn't build - `buck2 build //examples/backend:xnnpack_examples` doesn't build - broken `README` file, e.g. missing `.md` so no format at all ([see the file in OSS](https://github.com/pytorch/executorch/blob/main/examples/backend/README)) - broken generated file names, e.g. ugly name like `mv2_xnnpack_.pte` or `mv2_xnnpack__quantize.pte` Reviewed By: huydhn, kirklandsign Differential Revision: D48661299 fbshipit-source-id: d9fd89f6ffe565023c0f9a9a9ccf1415a3c59024
1 parent 3794945 commit 35db16d

File tree

5 files changed

+61
-38
lines changed

5 files changed

+61
-38
lines changed

examples/backend/README

Lines changed: 0 additions & 22 deletions
This file was deleted.

examples/backend/README.md

Lines changed: 33 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,33 @@
1+
This README gives some examples on backend-specific model workflow.
2+
3+
# XNNPACK Backend
4+
5+
[XNNPACK](https://github.com/google/XNNPACK) is a library of optimized of neural network inference operators for ARM and x86 platforms. Our delegate lowers models to run using these highly optimized CPU operators. You can try out lowering and running some example models using the following commands:
6+
7+
## XNNPACK delegation-only
8+
9+
The following command will produce an floating-point XNNPACK delegated model `mv2_xnnpack_fp32.pte` that can be run using XNNPACK's operators. It will also print out the lowered graph, showing what parts of the models have been lowered to XNNPACK via `executorch_call_delegate`.
10+
11+
```bash
12+
# For MobileNet V2
13+
python3 -m examples.backend.xnnpack_examples --model_name="mv2" --delegate
14+
```
15+
16+
Once we have the model binary (pte) file, then let's run it with Executorch runtime using the `xnn_executor_runner`.
17+
18+
```bash
19+
buck2 run examples/backend:xnn_executor_runner -- --model_path ./mv2_xnnpack_fp32.pte
20+
```
21+
22+
## XNNPACK quantization + delegation
23+
The following command will produce an XNNPACK quantized and delegated model `mv2_xnnpack_q8.pte` that can be run using XNNPACK's operators. It will also print out the lowered graph, showing what parts of the models have been lowered to XNNPACK via `executorch_call_delegate`.
24+
25+
```bash
26+
python3 -m examples.backend.xnnpack_examples --model_name="mv2" --quantize --delegate
27+
```
28+
29+
Once we have the model binary (pte) file, then let's run it with Executorch runtime using the `xnn_executor_runner`.
30+
31+
```bash
32+
buck2 run examples/backend:xnn_executor_runner -- --model_path ./mv2_xnnpack_q8.pte
33+
```

examples/backend/TARGETS

Lines changed: 3 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -1,19 +1,8 @@
1-
load("@fbsource//xplat/executorch/build:runtime_wrapper.bzl", "runtime")
1+
# Any targets that should be shared between fbcode and xplat must be defined in
2+
# targets.bzl. This file can contain fbcode-only targets.
3+
24
load(":targets.bzl", "define_common_targets")
35

46
oncall("executorch")
57

68
define_common_targets()
7-
8-
runtime.python_binary(
9-
name = "xnnpack_examples",
10-
main_src = "xnnpack_examples.py",
11-
deps = [
12-
"//caffe2:torch",
13-
"//executorch/backends/xnnpack:xnnpack_preprocess",
14-
"//executorch/backends/xnnpack/partition:xnnpack_partitioner",
15-
"//executorch/examples/models:models",
16-
"//executorch/examples/quantization:quant_utils",
17-
"//executorch/exir/backend:backend_api",
18-
],
19-
)

examples/backend/targets.bzl

Lines changed: 23 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,29 @@ def define_common_targets():
77
TARGETS and BUCK files that call this function.
88
"""
99

10-
# executor runner for XNNPACK Backend and portable kernels.
10+
runtime.python_binary(
11+
name = "xnnpack_examples",
12+
main_module = "executorch.examples.backend.xnnpack_examples",
13+
deps = [
14+
":xnnpack_examples_lib",
15+
],
16+
)
17+
18+
runtime.python_library(
19+
name = "xnnpack_examples_lib",
20+
srcs = [
21+
"xnnpack_examples.py",
22+
],
23+
deps = [
24+
"//executorch/backends/xnnpack/partition:xnnpack_partitioner",
25+
"//executorch/examples/models:models",
26+
"//executorch/examples/quantization:quant_utils",
27+
"//executorch/exir:lib",
28+
"//executorch/exir/backend:backend_api",
29+
],
30+
)
31+
32+
# executor_runner for XNNPACK Backend and portable kernels.
1133
runtime.cxx_binary(
1234
name = "xnn_executor_runner",
1335
srcs = [],

examples/backend/xnnpack_examples.py

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -85,7 +85,8 @@
8585

8686
exec_prog = edge.to_executorch()
8787
buffer = exec_prog.buffer
88-
quant_tag = "_quantize" if args.quantize else ""
88+
89+
quant_tag = "q8" if args.quantize else "fp32"
8990
filename = f"{args.model_name}_xnnpack_{quant_tag}.pte"
9091
logging.info(f"Saving exported program to {filename}.")
9192
with open(filename, "wb") as f:

0 commit comments

Comments
 (0)