Skip to content

Commit 3945100

Browse files
iseeyuanfacebook-github-bot
authored andcommitted
Update README.md with TorchTune reference (#2873)
Summary: We'd estimate communities like to try ExecuTorch with their own tuned models. Add a reference to TorchTune so that users can use all PyTorch native support in the entire flow. Pull Request resolved: #2873 Reviewed By: mergennachin, cccclai Differential Revision: D55807346 Pulled By: iseeyuan fbshipit-source-id: e848b2638922e97c6e267ee71bf77a7c7b23e771
1 parent 6219367 commit 3945100

File tree

1 file changed

+23
-0
lines changed

1 file changed

+23
-0
lines changed

examples/models/llama2/README.md

Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -89,6 +89,29 @@ If you want to deploy and run a smaller model for educational purposes. From `ex
8989
python -m examples.models.llama2.tokenizer.tokenizer -t tokenizer.model -o tokenizer.bin
9090
```
9191
92+
93+
## (Optional) Finetuning
94+
95+
If you want to finetune your model based on a specific dataset, PyTorch provides [TorchTune](https://github.com/pytorch/torchtune) - a native-Pytorch library for easily authoring, fine-tuning and experimenting with LLMs.
96+
97+
Once you have [TorchTune installed](https://github.com/pytorch/torchtune?tab=readme-ov-file#get-started) you can finetune Llama2 7B model using LoRA on a single GPU, using the following command. This will produce a checkpoint where the LoRA weights are merged with the base model and so the output checkpoint will be in the same format as the original Llama2 model.
98+
99+
```
100+
tune run lora_finetune_single_device \
101+
--config llama2/7B_lora_single_device \
102+
checkpointer.checkpoint_dir=<path_to_checkpoint_folder> \
103+
tokenizer.path=<path_to_checkpoint_folder>/tokenizer.model
104+
```
105+
106+
To run full finetuning with Llama2 7B on a single device, you can use the following command.
107+
108+
```
109+
tune run full_finetune_single_device \
110+
--config llama2/7B_full_single_device \
111+
checkpointer.checkpoint_dir=<path_to_checkpoint_folder> \
112+
tokenizer.path=<path_to_checkpoint_folder>/tokenizer.model
113+
```
114+
92115
## Step 3: Evaluate model accuracy
93116
94117
> Forewarning: Model evaluation without a GPU may take a long time, especially on larger models.

0 commit comments

Comments
 (0)