Skip to content

Commit 2e2929f

Browse files
committed
create dir on download
1 parent daac3e3 commit 2e2929f

File tree

2 files changed

+1
-1
lines changed

2 files changed

+1
-1
lines changed

README.md

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -112,7 +112,6 @@ Designed for interactive graphical conversations using the familiar web browser
112112

113113
Quantization is the process of converting a model into a more memory-efficient representation. Quantization is particularly important for accelerators -- to take advantage of the available memory bandwidth, and fit in the often limited high-speed memory in accelerators – and mobile devices – to fit in the typically very limited memory of mobile devices.
114114

115-
116115
Depending on the model and the target device, different quantization recipes may be applied. Torchchat contains two example configurations to optimize performance for GPU-based systems `config/data/cuda.json` , and mobile systems `config/data/mobile.json`. The GPU configuration is targeted towards optimizing for memory bandwidth which is a scarce resource in powerful GPUs (and to a less degree, memory footprint to fit large models into a device's memory). The mobile configuration is targeted towards optimizing for memory fotoprint because in many devices, a single application is limited to as little as GB or less of memory.
117116

118117
You can use the quantization recipes in conjunction with any of the `chat`, `generate` and `browser` commands to test their impact and accelerate model execution. You will apply these recipes to the export comamnds below, to optimize the exported models. To adapt these recipes or wrote your own, please refer to the [quantization overview](docs/quantization.md).

download.py

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -102,6 +102,7 @@ def download_and_convert(
102102
# overwriting if necessary.
103103
if os.path.isdir(model_dir):
104104
shutil.rmtree(model_dir)
105+
os.makedirs(model_dir, exist_ok=True)
105106
shutil.move(temp_dir, model_dir)
106107

107108
finally:

0 commit comments

Comments
 (0)