Bug fixed with n_ctx=0

[DanieleMorotti]

If the n_ctx parameter is set to 0 the function should use the maximum context length of the selected model, but it didn't work. There was a problem with the initialization of this parameter and a related problem with n_batch.

I know that the code is not the best one, but in order to get the model information about the context I needed to add it after the creation of the model instance at line 923 of the file llama.py.

self._model = _LlamaModel(
    path_model=self.model_path, params=self.model_params, verbose=self.verbose
)

Unfortunately, different objects were already initialized, therefore in the fix I had to change the n_ctx, self.n_batch, self.context_params.n_ctx and self.context_params.n_batch variables even if they already had a value.

Tell me if you find a smarter or more elegant solution to change the code and i will implement it.

This change should also fix #988.

Thank you

[DanieleMorotti]

Another approach can be to use the code in the llama.cpp repo to read the metadata from the gguf file before loading the model. In the json format the context length is related to the key llama.context_length.

[abetlen]

@DanieleMorotti does the metadata value differ from n_ctx_train? I think this approach is good though, maybe worth changing to the default in a future major release.

[DanieleMorotti]

I think n_ctx_train is the same value that we can read from the metadata. Tell me if I should try to implement the other approach, we would need to add gguf-py directory of llama.cpp in this repo and call the function that returns the json with the information.
Thanks