gguf : fix strings to not be null-terminated

[ggerganov]

close #2836

[ggerganov]

cc @philpax @slaren

[slaren]

Maybe this one needs to be updated too?

https://github.com/ggerganov/llama.cpp/blob/34e5c9afe5816368ad75fdebacffc172cb863a7e/ggml.c#L20232

[slaren]

I am trying to compare a f16 model converted with convert.py, then taking that the same model and "quantizing" to f16. The result should be the same, right? But I am seeing differences in the headers. It looks like there is a lot more padding in the file created by quantize. Left is from quantize, right is from convert.pty:

I'll try re-converting the file, it may have been created with an outdated version of convert.py.

[ggerganov]

The left one is GGUFv2 and the right one is GGUFv1 - see the 5th byte

[slaren]

Yes, after reconverting the file the headers look almost identical. Still, the files have a different hash. It's probably not an issue, but I'll see if I can figure what exactly are the differences.

[ggerganov]

Yes, we should figure out where this difference comes from

[slaren]

Ok, the only difference is the metadata general.quantization_version. I commented it from llama.cpp, and the results are identical now, exact same md5sum.

[ggerganov]

Looks like this change is backwards compatible, so no need for re-quantizing models in general, correct?

[slaren]

In practice, I doubt this would cause any issues, but the strings read from the python library will have a trailing NUL character. However, it should be easy to modify the library to remove them, there is no reason to have strings with NULs anyway.

[philpax]

All seems to work well, thanks!