Migrate inference to llama_batch and llama_decode api (ggml-org#795)

* Add low-level batching notebook

* fix: tokenization of special characters: (ggml-org#850)

It should behave like llama.cpp, where most out of the box usages
treat special characters accordingly

* Update CHANGELOG

* Cleanup

* Fix runner label

* Update notebook

* Use llama_decode and batch api

* Support logits_all parameter

---------

Co-authored-by: Antoine Lizee <[email protected]>

Author: abetlen