Skip to content

Commit e0117e5

Browse files
byjlwMichael Gschwind
authored andcommitted
add ET Runner instructions (#691)
Co-authored-by: Michael Gschwind <[email protected]>
1 parent 2995984 commit e0117e5

File tree

1 file changed

+16
-0
lines changed

1 file changed

+16
-0
lines changed

README.md

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -199,6 +199,22 @@ export TORCHCHAT_ROOT=${PWD}
199199
./scripts/install_et.sh
200200
```
201201

202+
### Test it out using our Executorch runner
203+
Build the runner
204+
```bash
205+
scripts/build_native.sh et
206+
```
207+
208+
**Get a PTE file if you don't have one already**
209+
```
210+
python3 torchchat.py export llama3 --quantize config/data/mobile.json --output-pte-path llama3.pte
211+
```
212+
213+
Execute using the runner
214+
```bash
215+
cmake-out/et_run llama3.pte -z ~/.torchchat/model-cache/meta-llama/Meta-Llama-3-8B-Instruct/tokenizer.model -i "Once upon a time"
216+
```
217+
202218
### Export for mobile
203219
The following example uses the Llama3 8B Instruct model.
204220

0 commit comments

Comments
 (0)