llama : add llama_model_load_from_splits

[ngxson]

Some downstream program may want to use non-conventional file name. For example, ollama is using SHA256 as file name. This can make adding support for multi-splits GGUF become tricky.

This PR adds a new API llama_model_load_from_splits that allow user to manually specify a list of GGUF files:

    // Load the model from multiple splits (support custom naming scheme)
    // The paths must be in the correct order
    LLAMA_API struct llama_model * llama_model_load_from_splits(
                             const char ** paths,
                                 size_t    n_paths,
              struct llama_model_params    params);

[ggerganov]

This can be static function in the source file only - no need to add it in the header.

There is also an existing a llama_split_ prefix which seems suitable to use for this function: llama_split_get_list()

[ngxson]

Ah yeah, I wanted to use this in llama.cpp but decided not to do that in the end. Forgot to delete it in the header file.

It should be fixed with 49822ba

[jianlinshi]

How to use it in command line?

[ngxson]

I don't add this to CLI for now, as this is only useful for downstream users.

For llama-cli we simply support <name>-0000x-of-0000x.gguf file name for now. Not sure in which cases you want to have different naming?

[jianlinshi]

It would be very helpful to start with:
llama-serve with the first split file -0001-of-000x.gguf
I often had issues with merging, and that process will need extra space. I am working on a controlled device, the space is often an issue.
Thank you!