File tree Expand file tree Collapse file tree 1 file changed +14
-3
lines changed Expand file tree Collapse file tree 1 file changed +14
-3
lines changed Original file line number Diff line number Diff line change @@ -91,7 +91,18 @@ If you want to deploy and run a smaller model for educational purposes. From `ex
91
91
python -m examples.models.llama2.tokenizer.tokenizer -t tokenizer.model -o tokenizer.bin
92
92
```
93
93
94
- ## Step 3: Run on your computer to validate
94
+ ## Step 3: Evaluate model accuracy
95
+
96
+ > Forewarning: Model evaluation without a GPU may take a long time, especially on larger models.
97
+
98
+ Using the same arguments from above
99
+ ```
100
+ python -m examples.models.llama2.eval_llama -c <checkpoint.pth> -p <params.json> -t <tokenizer.model> -d fp32 --max_seq_len <max sequence length > --limit <number of samples >
101
+ ```
102
+
103
+ The Uncyclotext results generated above used: `{max_seq_len: 2048, limit: 1000}`
104
+
105
+ ## Step 4: Run on your computer to validate
95
106
96
107
1. Build executorch with XNNPACK enabled. Build options available [here](https://github.com/pytorch/executorch/blob/main/CMakeLists.txt#L59).
97
108
```
@@ -127,13 +138,13 @@ If you want to deploy and run a smaller model for educational purposes. From `ex
127
138
cmake-out/examples/models/llama2/llama_main --model_path=<model pte file> --tokenizer_path=<tokenizer.bin> --prompt=<prompt>
128
139
```
129
140
130
- ## Step 4 : Run benchmark on Android phone
141
+ ## Step 5 : Run benchmark on Android phone
131
142
132
143
1. Build llama runner binary for Android
133
144
134
145
2. Run on Android via adb shell
135
146
136
- ## Step 5 : Build iOS and/or Android apps
147
+ ## Step 6 : Build iOS and/or Android apps
137
148
138
149
TODO
139
150
You can’t perform that action at this time.
0 commit comments