Convert safetensors model into gguf (aka. quantization)

1. First, go to the Models tab and locate the model you want to convert.

   

2. Select a target Type from the dropdown.

   

3. Right-click on the model and choose "Convert model to %s gguf format." The application will ask you to confirm the conversion with the selected quantization settings.

4. The converted model file will follow this naming format:

   model_original_name.safetensors -> model_original_name_F16.gguf

   If a file with this name already exists, the application will prompt you to confirm if you'd like to overwrite it.

5. Compatibility Check: If the source model is compatible with sd.cpp, the conversion should proceed without issues. If it’s not supported (e.g., "unsupported dtype 'F64'"), you’ll need to first convert the model to F16 or F32 before it can be used. For converting this type of models, you can use this tool: sdc

6. If an error occurs during conversion, check the app.log file for details.