Skip to content

Commit d56330c

Browse files
Jack-Khuufacebook-github-bot
authored andcommitted
Adding instructions for generating model accuracy (#2855)
Summary: Title Created from CodeHub with https://fburl.com/edit-in-codehub Reviewed By: kimishpatel, lucylq Differential Revision: D55775529
1 parent f64130e commit d56330c

File tree

1 file changed

+14
-3
lines changed

1 file changed

+14
-3
lines changed

examples/models/llama2/README.md

Lines changed: 14 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -91,7 +91,18 @@ If you want to deploy and run a smaller model for educational purposes. From `ex
9191
python -m examples.models.llama2.tokenizer.tokenizer -t tokenizer.model -o tokenizer.bin
9292
```
9393
94-
## Step 3: Run on your computer to validate
94+
## Step 3: Evaluate model accuracy
95+
96+
> Forewarning: Model evaluation without a GPU may take a long time, especially on larger models.
97+
98+
Using the same arguments from above
99+
```
100+
python -m examples.models.llama2.eval_llama -c <checkpoint.pth> -p <params.json> -t <tokenizer.model> -d fp32 --max_seq_len <max sequence length> --limit <number of samples>
101+
```
102+
103+
The Uncyclotext results generated above used: `{max_seq_len: 2048, limit: 1000}`
104+
105+
## Step 4: Run on your computer to validate
95106
96107
1. Build executorch with XNNPACK enabled. Build options available [here](https://github.com/pytorch/executorch/blob/main/CMakeLists.txt#L59).
97108
```
@@ -127,13 +138,13 @@ If you want to deploy and run a smaller model for educational purposes. From `ex
127138
cmake-out/examples/models/llama2/llama_main --model_path=<model pte file> --tokenizer_path=<tokenizer.bin> --prompt=<prompt>
128139
```
129140
130-
## Step 4: Run benchmark on Android phone
141+
## Step 5: Run benchmark on Android phone
131142
132143
1. Build llama runner binary for Android
133144
134145
2. Run on Android via adb shell
135146
136-
## Step 5: Build iOS and/or Android apps
147+
## Step 6: Build iOS and/or Android apps
137148
138149
TODO
139150

0 commit comments

Comments
 (0)