How to use Prompt template in llama.cpp?

[Folko-Ven]

Hello, could you please tell me how to use Prompt template
(like
You are a helpful assistant
USER: prompt goes here
ASSISTANT:
)
in llama.cpp? Usually i use this parameters
(--color --instruct --temp 0.8 --top_k 40 --top_p 0.95 --ctx_size 2048 --n_predict -1 --keep -1)

Do i need use --interactive or --interactive-first? Do i need to use Prompt template in every question or only at first?
Please share your experience.

[ghost]

There's a few ways for using a prompt template:

1. Use the -p parameter like this:
   ./main --color --instruct --temp 0.8 --top_k 40 --top_p 0.95 --ctx_size 2048 --n_predict -1 --keep -1 -i -r "USER:" -p "You are a helpful assistant. USER: prompt goes here ASSISTANT:"

2. Save the template in a .txt file, and then load it with the -f parameter, like this:
   ./main --color --instruct --temp 0.8 --top_k 40 --top_p 0.95 --ctx_size 2048 --n_predict -1 --keep -1 -i -r "USER:" -f ~/path/to/PromptTemplate.txt

Don't use --interactive-first for this situation. Yes, --interactive (-i) is required.

[Folko-Ven]

First of all - thanks for the reply!
I'm currently using a slightly different method than the one you wrote.
To keep the dialogue structure i use --prompt 'You are a helpful assistant' --in-prefix 'USER: ' --in-suffix 'ASSISTANT: '

[ghost]

Cool, I'll try it out. Thanks for sharing.

[ghost]

Can you provide more information as to how you're using these parameters?

I'm trying this..
./main -m ~/llama.cpp/models/wizardlm-7b-v1.0-uncensored.ggmlv3.q4_0.bin --color -c 2048 --keep -1 -t 4 -b 10 -i --prompt 'You are a helpful assistant.' --in-prefix 'USER: ' --in-suffix 'ASSISTANT:'

Llama.cpp starts, I see the prompt, but then it just auto-chats.

[Folko-Ven]

Here's what I'm using now: ./main/main --model ./models/airoboros-65B-gpt4-1.2.ggmlv3.q5_K_S.bin --threads 7 --color --instruct --temp 0.8 --top_k 40 --top_p 0.95 --ctx_size 2048 --n_predict -1 --keep -1 --n-gpu-layers 8 --file ./prompts/airoboros.txt --in-prefix 'USER: ' --in-suffix 'ASSISTANT: '

I think you use -i for --interactive, and i use --instruct

[ghost]

Yup, I switched from interactive to instruct and it's working as expected. Thanks for that.

[ghost]

I've been testing and found --instruct works best with Alpaca models.

An alternative method is with --interactive. Here's an example template: A chat between a curious user and an artificial intelligence assistant. The assistant gives helpful, detailed, and polite answers to the user's questions. USER: Hi, how are you? ASSISTANT:

./main -m ~/7B.ggmlv3.q4_1.bin -i -r "USER:" --in-prefix " " --in-suffix "ASSISTANT:" -p "A chat between a curious user and an artificial intelligence assistant. The assistant gives helpful, detailed, and polite answers to the user's questions. USER: Hi, how are you? ASSISTANT:"

--in-prefix " " is from the ReadMe.