llama2c : fix segfault and alloc-dealloc-mismatch

[cebtenzzre]

When the vocab file was not found:

$ gdb --args ./convert-llama2c-to-ggml --llama2c-model stories15M.bin
<snip>
Assuming llama2.c vocabulary since models/7B/ggml-model-f16.gguf is not a gguf file

Program received signal SIGSEGV, Segmentation fault.
__GI__IO_fread (buf=0x7fffffffd4f4, size=4, count=1, fp=0x0) at iofread.c:37
37        _IO_acquire_lock (fp);
>>> bt
#0  __GI__IO_fread (buf=0x7fffffffd4f4, size=4, count=1, fp=0x0) at iofread.c:37
#1  0x0000555555561623 in llama_file::read_raw (this=0x7fffffffd5a0, ptr=0x7fffffffd4f4, size=4)
    at examples/convert-llama2c-to-ggml/convert-llama2c-to-ggml.cpp:500
#2  0x000055555556182f in llama_file::read_u32 (this=0x7fffffffd5a0)
    at examples/convert-llama2c-to-ggml/convert-llama2c-to-ggml.cpp:511
#3  0x000055555555f5da in load_vocab (filename=0x555555645dd7 "models/7B/ggml-model-f16.gguf", config=0x7fffffffd660, 
    vocab=0x7fffffffd680) at examples/convert-llama2c-to-ggml/convert-llama2c-to-ggml.cpp:600
#4  0x000055555556105c in main (argc=3, argv=0x7fffffffd988)
    at examples/convert-llama2c-to-ggml/convert-llama2c-to-ggml.cpp:930

When doing exit cleanup:

==114607==ERROR: AddressSanitizer: alloc-dealloc-mismatch (operator new [] vs operator delete) on 0x7fb510ed7800
    #0 0x7fb5166e2b8a in operator delete(void*) /usr/src/debug/gcc/gcc/libsanitizer/asan/asan_new_delete.cpp:152
    #1 0x55fbe2d76b3c in free_weights(TransformerWeights*) examples/convert-llama2c-to-ggml/convert-llama2c-to-ggml.cpp:177
    #2 0x55fbe2d71b2a in main examples/convert-llama2c-to-ggml/convert-llama2c-to-ggml.cpp:960
    #3 0x7fb515e23ccf in __libc_start_call_main ../sysdeps/nptl/libc_start_call_main.h:58
    #4 0x7fb515e23d89 in __libc_start_main_impl ../csu/libc-start.c:360
    #5 0x55fbe2d73d34 in _start (/home/cebtenzzre/src/forks/llama.cpp/convert-llama2c-to-ggml+0x2dd34) (BuildId: 8dfd479c1bd88031c1afb94dc85b78069e3fcc93)

0x7fb510ed7800 is located 0 bytes inside of 36864000-byte region [0x7fb510ed7800,0x7fb5131ff800)
allocated by thread T0 here:
    #0 0x7fb5166e2182 in operator new[](unsigned long) /usr/src/debug/gcc/gcc/libsanitizer/asan/asan_new_delete.cpp:98
    #1 0x55fbe2d74e82 in malloc_weights(TransformerWeights*, Config*, bool) examples/convert-llama2c-to-ggml/convert-llama2c-to-ggml.cpp:104
    #2 0x55fbe2d714c4 in main examples/convert-llama2c-to-ggml/convert-llama2c-to-ggml.cpp:928
    #3 0x7fb515e23ccf in __libc_start_call_main ../sysdeps/nptl/libc_start_call_main.h:58

SUMMARY: AddressSanitizer: alloc-dealloc-mismatch /usr/src/debug/gcc/gcc/libsanitizer/asan/asan_new_delete.cpp:152 in operator delete(void*)

[cebtenzzre]

I fixed these memory leaks too:

Details

==123842==ERROR: LeakSanitizer: detected memory leaks

Direct leak of 36864000 byte(s) in 1 object(s) allocated from:
    #0 0x7f4c960e2182 in operator new[](unsigned long) /usr/src/debug/gcc/gcc/libsanitizer/asan/asan_new_delete.cpp:98
    #1 0x55695da9f103 in malloc_weights(TransformerWeights*, Config*, bool) examples/convert-llama2c-to-ggml/convert-llama2c-to-ggml.cpp:104
    #2 0x55695daa950b in main examples/convert-llama2c-to-ggml/convert-llama2c-to-ggml.cpp:928
    #3 0x7f4c95823ccf in __libc_start_call_main ../sysdeps/nptl/libc_start_call_main.h:58

Direct leak of 5308416 byte(s) in 1 object(s) allocated from:
    #0 0x7f4c960e2182 in operator new[](unsigned long) /usr/src/debug/gcc/gcc/libsanitizer/asan/asan_new_delete.cpp:98
    #1 0x55695da9ff2c in malloc_weights(TransformerWeights*, Config*, bool) examples/convert-llama2c-to-ggml/convert-llama2c-to-ggml.cpp:125
    #2 0x55695daa950b in main examples/convert-llama2c-to-ggml/convert-llama2c-to-ggml.cpp:928
    #3 0x7f4c95823ccf in __libc_start_call_main ../sysdeps/nptl/libc_start_call_main.h:58

Direct leak of 5308416 byte(s) in 1 object(s) allocated from:
    #0 0x7f4c960e2182 in operator new[](unsigned long) /usr/src/debug/gcc/gcc/libsanitizer/asan/asan_new_delete.cpp:98
    #1 0x55695daa0430 in malloc_weights(TransformerWeights*, Config*, bool) examples/convert-llama2c-to-ggml/convert-llama2c-to-ggml.cpp:131
    #2 0x55695daa950b in main examples/convert-llama2c-to-ggml/convert-llama2c-to-ggml.cpp:928
    #3 0x7f4c95823ccf in __libc_start_call_main ../sysdeps/nptl/libc_start_call_main.h:58

Direct leak of 5308416 byte(s) in 1 object(s) allocated from:
    #0 0x7f4c960e2182 in operator new[](unsigned long) /usr/src/debug/gcc/gcc/libsanitizer/asan/asan_new_delete.cpp:98
    #1 0x55695daa01ae in malloc_weights(TransformerWeights*, Config*, bool) examples/convert-llama2c-to-ggml/convert-llama2c-to-ggml.cpp:128
    #2 0x55695daa950b in main examples/convert-llama2c-to-ggml/convert-llama2c-to-ggml.cpp:928
    #3 0x7f4c95823ccf in __libc_start_call_main ../sysdeps/nptl/libc_start_call_main.h:58

Direct leak of 1990656 byte(s) in 1 object(s) allocated from:
    #0 0x7f4c960e2182 in operator new[](unsigned long) /usr/src/debug/gcc/gcc/libsanitizer/asan/asan_new_delete.cpp:98
    #1 0x55695da9fce6 in malloc_weights(TransformerWeights*, Config*, bool) examples/convert-llama2c-to-ggml/convert-llama2c-to-ggml.cpp:122
    #2 0x55695daa950b in main examples/convert-llama2c-to-ggml/convert-llama2c-to-ggml.cpp:928
    #3 0x7f4c95823ccf in __libc_start_call_main ../sysdeps/nptl/libc_start_call_main.h:58

Direct leak of 1990656 byte(s) in 1 object(s) allocated from:
    #0 0x7f4c960e2182 in operator new[](unsigned long) /usr/src/debug/gcc/gcc/libsanitizer/asan/asan_new_delete.cpp:98
    #1 0x55695da9f6cb in malloc_weights(TransformerWeights*, Config*, bool) examples/convert-llama2c-to-ggml/convert-llama2c-to-ggml.cpp:113
    #2 0x55695daa950b in main examples/convert-llama2c-to-ggml/convert-llama2c-to-ggml.cpp:928
    #3 0x7f4c95823ccf in __libc_start_call_main ../sysdeps/nptl/libc_start_call_main.h:58

Direct leak of 1990656 byte(s) in 1 object(s) allocated from:
    #0 0x7f4c960e2182 in operator new[](unsigned long) /usr/src/debug/gcc/gcc/libsanitizer/asan/asan_new_delete.cpp:98
    #1 0x55695da9f8d4 in malloc_weights(TransformerWeights*, Config*, bool) examples/convert-llama2c-to-ggml/convert-llama2c-to-ggml.cpp:116
    #2 0x55695daa950b in main examples/convert-llama2c-to-ggml/convert-llama2c-to-ggml.cpp:928
    #3 0x7f4c95823ccf in __libc_start_call_main ../sysdeps/nptl/libc_start_call_main.h:58

Direct leak of 1990656 byte(s) in 1 object(s) allocated from:
    #0 0x7f4c960e2182 in operator new[](unsigned long) /usr/src/debug/gcc/gcc/libsanitizer/asan/asan_new_delete.cpp:98
    #1 0x55695da9fadd in malloc_weights(TransformerWeights*, Config*, bool) examples/convert-llama2c-to-ggml/convert-llama2c-to-ggml.cpp:119
    #2 0x55695daa950b in main examples/convert-llama2c-to-ggml/convert-llama2c-to-ggml.cpp:928
    #3 0x7f4c95823ccf in __libc_start_call_main ../sysdeps/nptl/libc_start_call_main.h:58

Direct leak of 6912 byte(s) in 1 object(s) allocated from:
    #0 0x7f4c960e2182 in operator new[](unsigned long) /usr/src/debug/gcc/gcc/libsanitizer/asan/asan_new_delete.cpp:98
    #1 0x55695da9f4d6 in malloc_weights(TransformerWeights*, Config*, bool) examples/convert-llama2c-to-ggml/convert-llama2c-to-ggml.cpp:110
    #2 0x55695daa950b in main examples/convert-llama2c-to-ggml/convert-llama2c-to-ggml.cpp:928
    #3 0x7f4c95823ccf in __libc_start_call_main ../sysdeps/nptl/libc_start_call_main.h:58

Direct leak of 6912 byte(s) in 1 object(s) allocated from:
    #0 0x7f4c960e2182 in operator new[](unsigned long) /usr/src/debug/gcc/gcc/libsanitizer/asan/asan_new_delete.cpp:98
    #1 0x55695da9f2ea in malloc_weights(TransformerWeights*, Config*, bool) examples/convert-llama2c-to-ggml/convert-llama2c-to-ggml.cpp:107
    #2 0x55695daa950b in main examples/convert-llama2c-to-ggml/convert-llama2c-to-ggml.cpp:928
    #3 0x7f4c95823ccf in __libc_start_call_main ../sysdeps/nptl/libc_start_call_main.h:58

Direct leak of 1152 byte(s) in 1 object(s) allocated from:
    #0 0x7f4c960e2182 in operator new[](unsigned long) /usr/src/debug/gcc/gcc/libsanitizer/asan/asan_new_delete.cpp:98
    #1 0x55695daa0626 in malloc_weights(TransformerWeights*, Config*, bool) examples/convert-llama2c-to-ggml/convert-llama2c-to-ggml.cpp:134
    #2 0x55695daa950b in main examples/convert-llama2c-to-ggml/convert-llama2c-to-ggml.cpp:928
    #3 0x7f4c95823ccf in __libc_start_call_main ../sysdeps/nptl/libc_start_call_main.h:58

SUMMARY: AddressSanitizer: 60766848 byte(s) leaked in 11 allocation(s).

They were happening when e.g. you tried to open a truncated llama2c model file.