Skip to content

Commit df56c5d

Browse files
lucylqfacebook-github-bot
authored andcommitted
Llama stories example
Summary: Add llama stories example to readme Differential Revision: D53827636
1 parent 1689ed8 commit df56c5d

File tree

1 file changed

+32
-2
lines changed

1 file changed

+32
-2
lines changed

examples/models/llama2/README.md

Lines changed: 32 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -26,8 +26,38 @@ This example tries to reuse the Python code, with modifications to make it compa
2626

2727

2828
# Instructions:
29+
### Setup
2930
1. Follow the [tutorial](https://pytorch.org/executorch/stable/getting-started-setup) to set up ExecuTorch
3031
2. `cd examples/third-party/llama`
3132
3. `pip install -e .`
32-
4. Go back to `executorch` root, run `python3 -m examples.models.llama2.export_llama`. The exported program, llama2.pte would be saved in current directory using the dummy checkpoint.
33-
5. Llama2 pretrained parameters can be downloaded [here](https://ai.meta.com/resources/models-and-libraries/llama-downloads/) and run with `python3 -m examples.models.llama2.export_llama --checkpoint <checkpoint.pth> --params <params.json>`.
33+
4. Go back to `executorch` root, run `bash examples/models/llama2/install_requirements.sh`.
34+
35+
### Export llama2 models
36+
2. From `executorch` root, run `python3 -m examples.models.llama2.export_llama`. The exported program, llama2.pte would be saved in current directory using the dummy checkpoint.
37+
3. Llama2 pretrained parameters can be downloaded [here](https://ai.meta.com/resources/models-and-libraries/llama-downloads/) and run with `python3 -m examples.models.llama2.export_llama --checkpoint <checkpoint.pth> --params <params.json>`.
38+
39+
### Export and run stories110M model
40+
41+
1. Download `stories110M.pt` and `tokenizer.model` from Github.
42+
```
43+
wget "https://huggingface.co/karpathy/tinyllamas/resolve/main/stories110M.pt"
44+
wget "https://raw.githubusercontent.com/karpathy/llama2.c/master/tokenizer.model"
45+
```
46+
2. Create params file.
47+
```
48+
echo '{"dim": 768, "multiple_of": 32, "n_heads": 12, "n_layers": 12, "norm_eps": 1e-05, "vocab_size": 32000}' > params.json
49+
```
50+
3. Export model. Export options available [here](https://github.com/pytorch/executorch/blob/main/examples/models/llama2/export_llama_lib.py#L161).
51+
```
52+
python3 -m examples.models.llama2.export_llama -c stories110M.pt -p params.json
53+
```
54+
4. Create tokenizer.bin.
55+
```
56+
buck2 run examples/models/llama2/tokenizer:tokenizer_py -- -t tokenizer.model -o tokenizer.bin
57+
```
58+
5. Run model. Run options available [here](https://github.com/pytorch/executorch/blob/main/examples/models/llama2/main.cpp#L13).
59+
```
60+
buck2 run examples/models/llama2:main -- --model_path=llama2.pte --tokenizer_path=tokenizer.bin --prompt="Once"
61+
```
62+
63+
See test script [here](https://github.com/pytorch/executorch/blob/main/.ci/scripts/test_llama.sh).

0 commit comments

Comments
 (0)