Skip to content

Commit a4e63b7

Browse files
authored
Add instruction for using Alpaca (#240)
1 parent 9e17072 commit a4e63b7

File tree

1 file changed

+39
-9
lines changed

1 file changed

+39
-9
lines changed

README.md

Lines changed: 39 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -176,21 +176,51 @@ In this mode, you can always interrupt generation by pressing Ctrl+C and enter o
176176

177177
Here is an example few-shot interaction, invoked with the command
178178
```
179-
./main -m ./models/13B/ggml-model-q4_0.bin -n 256 --repeat_penalty 1.0 --color -i -r "User:" \
180-
-p \
181-
"Transcript of a dialog, where the User interacts with an Assistant named Bob. Bob is helpful, kind, honest, good at writing, and never fails to answer the User's requests immediately and with precision.
182-
183-
User: Hello, Bob.
184-
Bob: Hello. How may I help you today?
185-
User: Please tell me the largest city in Europe.
186-
Bob: Sure. The largest city in Europe is Moscow, the capital of Russia.
187-
User:"
179+
./main -m ./models/13B/ggml-model-q4_0.bin -n 256 --repeat_penalty 1.0 --color -i -r "User:" -f prompts/chat-with-bob.txt
188180
189181
```
190182
Note the use of `--color` to distinguish between user input and generated text.
191183

192184
![image](https://user-images.githubusercontent.com/1991296/224575029-2af3c7dc-5a65-4f64-a6bb-517a532aea38.png)
193185

186+
### Instruction mode with Alpaca
187+
188+
First, download the `ggml` Alpaca model into the `./models` folder:
189+
190+
```
191+
# use one of these
192+
# NOTE: these are copied from the alpaca.cpp repo - not sure how long these will work
193+
# TODO: add a script to simplify the download
194+
curl -o ggml-alpaca-7b-q4.bin -C - https://gateway.estuary.tech/gw/ipfs/QmQ1bf2BTnYxq73MFJWu1B7bQ2UD6qG7D7YDCxhTndVkPC
195+
curl -o ggml-alpaca-7b-q4.bin -C - https://ipfs.io/ipfs/QmQ1bf2BTnYxq73MFJWu1B7bQ2UD6qG7D7YDCxhTndVkPC
196+
curl -o ggml-alpaca-7b-q4.bin -C - https://cloudflare-ipfs.com/ipfs/QmQ1bf2BTnYxq73MFJWu1B7bQ2UD6qG7D7YDCxhTndVkPC
197+
```
198+
199+
Now run the `main` tool like this:
200+
201+
```
202+
./main -m ./models/ggml-alpaca-7b-q4.bin --color -f ./prompts/alpaca.txt -ins
203+
```
204+
205+
Sample run:
206+
207+
```
208+
== Running in interactive mode. ==
209+
- Press Ctrl+C to interject at any time.
210+
- Press Return to return control to LLaMa.
211+
- If you want to submit another line, end your input in '\'.
212+
213+
Below is an instruction that describes a task. Write a response that appropriately completes the request.
214+
215+
> How many letters are there in the English alphabet?
216+
There 26 letters in the English Alphabet
217+
> What is the most common way of transportation in Amsterdam?
218+
The majority (54%) are using public transit. This includes buses, trams and metros with over 100 lines throughout the city which make it very accessible for tourists to navigate around town as well as locals who commute by tram or metro on a daily basis
219+
> List 5 words that start with "ca".
220+
cadaver, cauliflower, cabbage (vegetable), catalpa (tree) and Cailleach.
221+
>
222+
```
223+
194224
### Android
195225

196226
You can easily run `llama.cpp` on Android device with [termux](https://play.google.com/store/apps/details?id=com.termux).

0 commit comments

Comments
 (0)