Skip to content

Commit 64dbbb6

Browse files
kimishpatelmalfet
authored andcommitted
Fix quant doc 2 (#643)
Summary: Generate step did not specify model name args. Names llama3. Fix that.
1 parent 6c0b25f commit 64dbbb6

File tree

1 file changed

+2
-2
lines changed

1 file changed

+2
-2
lines changed

docs/quantization.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -69,13 +69,13 @@ python3 generate.py [--compile] llama3 --prompt "Hello, my name is" --quantize '
6969
```
7070
python3 torchchat.py export llama3 --quantize '{"embedding": {"bitwidth": 4, "groupsize":32}, "linear:int4": {"groupsize" : 256}}' --output-dso-path llama3.dso
7171
72-
python3 generate.py --dso-path llama3.dso --prompt "Hello my name is"
72+
python3 generate.py llama3 --dso-path llama3.dso --prompt "Hello my name is"
7373
```
7474
### ExecuTorch
7575
```
7676
python3 torchchat.py export llama3 --dtype fp32 --quantize '{"embedding": {"bitwidth": 4, "groupsize":32}, "linear:a8w4dq": {"groupsize" : 256}}' --output-pte-path llama3.pte
7777
78-
python3 generate.py --pte-path llama3.pte --prompt "Hello my name is"
78+
python3 generate.py llama3 --pte-path llama3.pte --prompt "Hello my name is"
7979
```
8080

8181
## Model precision (dtype precision setting)

0 commit comments

Comments
 (0)