@@ -19,11 +19,15 @@ Options:
19
19
To build runner-aoti, run the following commands * from the torchchat root directory*
20
20
21
21
```
22
- cmake -S ./runner-aoti -B ./runner-aoti/cmake-out -G Ninja -DCMAKE_PREFIX_PATH=`python3 -c 'import torch;print(torch.utils.cmake_prefix_path)'`
23
- cmake --build ./runner-aoti/cmake-out
22
+ # Pull submodules (re2, abseil) for Tiktoken
23
+ git submodule sync
24
+ git submodule update --init
25
+
26
+ cmake -S . -B ./cmake-out -G Ninja -DCMAKE_PREFIX_PATH=`python3 -c 'import torch;print(torch.utils.cmake_prefix_path)'`
27
+ cmake --build ./cmake-out --target et_run
24
28
```
25
29
26
- After running these, the runner-aoti binary is located at ./runner-aoti/ cmake-out/run .
30
+ After running these, the runner-aoti binary is located at ./cmake-out/aoti_run .
27
31
28
32
Let us try using it with an example.
29
33
We first download stories15M and export it to AOTI.
@@ -37,7 +41,7 @@ We can now execute the runner with:
37
41
38
42
```
39
43
wget -O ./tokenizer.bin https://github.com/karpathy/llama2.c/raw/master/tokenizer.bin
40
- ./runner-aoti/ cmake-out/run ./model.so -z ./tokenizer.bin -i "Once upon a time"
44
+ ./cmake-out/aoti_run ./model.so -z ./tokenizer.bin -i "Once upon a time"
41
45
```
42
46
43
47
## Building and running runner-et
@@ -47,12 +51,16 @@ Before building runner-et, you must first setup ExecuTorch by following [setup E
47
51
To build runner-et, run the following commands * from the torchchat root directory*
48
52
49
53
```
54
+ # Pull submodules (re2, abseil) for Tiktoken
55
+ git submodule sync
56
+ git submodule update --init
57
+
50
58
export TORCHCHAT_ROOT=${PWD}
51
- cmake -S ./runner-et -B ./runner-et /cmake-out -G Ninja
52
- cmake --build ./runner-et/ cmake-out
59
+ cmake -S . -B ./cmake-out -G Ninja
60
+ cmake --build ./cmake-out --target et_run
53
61
```
54
62
55
- After running these, the runner-et binary is located at ./runner-et/ cmake-out/run .
63
+ After running these, the runner-et binary is located at ./cmake-out/et_run .
56
64
57
65
Let us try using it with an example.
58
66
We first download stories15M and export it to ExecuTorch.
@@ -66,5 +74,5 @@ We can now execute the runner with:
66
74
67
75
```
68
76
wget -O ./tokenizer.bin https://github.com/karpathy/llama2.c/raw/master/tokenizer.bin
69
- ./runner-et/ cmake-out/run ./model.pte -z ./tokenizer.bin -i "Once upon a time"
77
+ ./cmake-out/et_run ./model.pte -z ./tokenizer.bin -i "Once upon a time"
70
78
```
0 commit comments