Skip to content

Commit aba0679

Browse files
yanbing-jJack-Khuu
andauthored
Update docs for max-autotune usage (#1405)
Co-authored-by: Jack-Khuu <[email protected]>
1 parent 326c1fe commit aba0679

File tree

2 files changed

+5
-0
lines changed

2 files changed

+5
-0
lines changed

docs/ADVANCED-USERS.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -251,6 +251,8 @@ To improve performance, you can compile the model with `--compile`
251251
trading off the time to first token processed with time per token. To
252252
improve performance further, you may also compile the prefill with
253253
`--compile-prefill`. This will increase further compilation times though.
254+
For CPU, you can use `--max-autotune` to further improve the performance
255+
with `--compile` and `compile-prefill`. See [`max-autotune on CPU tutorial`](https://pytorch.org/tutorials/prototype/max_autotune_on_CPU_tutorial.html).
254256

255257
Parallel prefill is not yet supported by exported models, and may be
256258
supported in a future release.

docs/model_customization.md

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -34,6 +34,9 @@ prefill with `--compile_prefill`.
3434

3535
To learn more about compilation, check out: https://pytorch.org/get-started/pytorch-2.0/
3636

37+
For CPU, you can use `--max-autotune` to further improve the performance with `--compile` and `compile-prefill`.
38+
39+
See [`max-autotune on CPU tutorial`](https://pytorch.org/tutorials/prototype/max_autotune_on_CPU_tutorial.html).
3740

3841
## Model Precision
3942

0 commit comments

Comments
 (0)