Skip to content

Commit 60c9029

Browse files
authored
docs : bring llama-cli conversation/template docs up-to-date (#12426)
1 parent b1b132e commit 60c9029

File tree

1 file changed

+36
-5
lines changed

1 file changed

+36
-5
lines changed

examples/main/README.md

Lines changed: 36 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -27,12 +27,24 @@ Once downloaded, place your model in the models folder in llama.cpp.
2727
##### Input prompt (One-and-done)
2828

2929
```bash
30-
./llama-cli -m models/gemma-1.1-7b-it.Q4_K_M.gguf --prompt "Once upon a time"
30+
./llama-cli -m models/gemma-1.1-7b-it.Q4_K_M.gguf -no-cnv --prompt "Once upon a time"
3131
```
3232
##### Conversation mode (Allow for continuous interaction with the model)
3333

3434
```bash
35-
./llama-cli -m models/gemma-1.1-7b-it.Q4_K_M.gguf -cnv --chat-template gemma
35+
./llama-cli -m models/gemma-1.1-7b-it.Q4_K_M.gguf --chat-template gemma
36+
```
37+
38+
##### Conversation mode using built-in jinja chat template
39+
40+
```bash
41+
./llama-cli -m models/gemma-1.1-7b-it.Q4_K_M.gguf --jinja
42+
```
43+
44+
##### One-and-done query using jinja with custom system prompt and a starting prompt
45+
46+
```bash
47+
./llama-cli -m models/gemma-1.1-7b-it.Q4_K_M.gguf --jinja --single-turn -sys "You are a helpful assistant" -p "Hello"
3648
```
3749

3850
##### Infinite text from a starting prompt (you can use `Ctrl-C` to stop it):
@@ -44,12 +56,24 @@ Once downloaded, place your model in the models folder in llama.cpp.
4456

4557
##### Input prompt (One-and-done)
4658
```powershell
47-
./llama-cli.exe -m models\gemma-1.1-7b-it.Q4_K_M.gguf --prompt "Once upon a time"
59+
./llama-cli.exe -m models\gemma-1.1-7b-it.Q4_K_M.gguf -no-cnv --prompt "Once upon a time"
4860
```
4961
##### Conversation mode (Allow for continuous interaction with the model)
5062

5163
```powershell
52-
./llama-cli.exe -m models\gemma-1.1-7b-it.Q4_K_M.gguf -cnv --chat-template gemma
64+
./llama-cli.exe -m models\gemma-1.1-7b-it.Q4_K_M.gguf --chat-template gemma
65+
```
66+
67+
##### Conversation mode using built-in jinja chat template
68+
69+
```powershell
70+
./llama-cli.exe -m models\gemma-1.1-7b-it.Q4_K_M.gguf --jinja
71+
```
72+
73+
##### One-and-done query using jinja with custom system prompt and a starting prompt
74+
75+
```powershell
76+
./llama-cli.exe -m models\gemma-1.1-7b-it.Q4_K_M.gguf --jinja --single-turn -sys "You are a helpful assistant" -p "Hello"
5377
```
5478

5579
#### Infinite text from a starting prompt (you can use `Ctrl-C` to stop it):
@@ -77,6 +101,8 @@ The `llama-cli` program provides several ways to interact with the LLaMA models
77101

78102
- `--prompt PROMPT`: Provide a prompt directly as a command-line option.
79103
- `--file FNAME`: Provide a file containing a prompt or multiple prompts.
104+
- `--system-prompt PROMPT`: Provide a system prompt (will otherwise use the default one in the chat template (if provided)).
105+
- `--system-prompt-file FNAME`: Provide a file containing a system prompt.
80106
- `--interactive-first`: Run the program in interactive mode and wait for input right away. (More on this below.)
81107

82108
## Interaction
@@ -89,7 +115,10 @@ In interactive mode, users can participate in text generation by injecting their
89115

90116
- `-i, --interactive`: Run the program in interactive mode, allowing users to engage in real-time conversations or provide specific instructions to the model.
91117
- `--interactive-first`: Run the program in interactive mode and immediately wait for user input before starting the text generation.
92-
- `-cnv, --conversation`: Run the program in conversation mode (does not print special tokens and suffix/prefix, use default chat template) (default: false)
118+
- `-cnv, --conversation`: Run the program in conversation mode (does not print special tokens and suffix/prefix, use default or provided chat template) (default: true if chat template found)
119+
- `-no-cnv`: Disable conversation mode (default: false)
120+
- `-st, --single-turn`: Only process a single conversation turn (user input) and then exit.
121+
- `--jinja`: Enable jinja chat template parser, will use the model's built-in template or a user-provided one (default: false)
93122
- `--color`: Enable colorized output to differentiate visually distinguishing between prompts, user input, and generated text.
94123

95124
By understanding and utilizing these interaction options, you can create engaging and dynamic experiences with the LLaMA models, tailoring the text generation process to your specific needs.
@@ -125,6 +154,8 @@ When --in-prefix or --in-suffix options are enabled the chat template ( --chat-t
125154

126155
Example usage: `--chat-template gemma`
127156

157+
`--chat-template-file FNAME`: Load a custom jinja chat template from an external file, useful if the model contains outdated or incompatible template, some examples can be found in models/templates. Up-to-date chat templates can be downloaded from Hugging Face using scripts/get_chat_template.py
158+
128159
## Context Management
129160

130161
During text generation, LLaMA models have a limited context size, which means they can only consider a certain number of tokens from the input and generated text. When the context fills up, the model resets internally, potentially losing some information from the beginning of the conversation or instructions. Context management options help maintain continuity and coherence in these situations.

0 commit comments

Comments
 (0)