Request for detailed walkthrough for laypeople

[initiator1]

I've managed to get far enough to download three models, and run one model using llama.cpp, but then the model is just chatting with itself in Git Bash (the only way I could figure out how to stop it from talking to itself was to close the window) and ./server -m [model] gives me "No such file or directory" and in another attempt to clone it followed by "&& make -j4" it populated the files in the Windows directory but I get "make: command not found." I have near-zero experience with programming or with Git and I've been using ChatGPT to learn and troubleshoot and try to figure out next steps, but it's more difficult and leading to dead ends now that they took away the browsing feature. Is there any chance someone would be willing and able to put together (or point me to) an extremely dumbed-down and ridiculously detailed and smooth step-by-step walkthrough for an average Windows user re: what applications to use, what keys to stroke, etc. to get any LLMs running locally on a fairly strong Windows 11 laptop and/or a fairly new and powerful Android device? My sincerest thanks in advance for any help, and apologies if I missed it while searching these discussions unsuccessfully.

[ghost]

Hi, I can't assist with Windows, but here's Android instructions:

Use F-Droids Termux.

1. a. Ensure Termux is up to date:

apt update && apt upgrade
pkg install make clang git

1. b. Clone the repo:

$HOME

git clone https://github.com/ggerganov/llama.cpp

1. c. Move your model to the proper directory, for example:

mv ~/storage/downloads/7b-ggml-q4_0.bin ~/

**Note: The model doesn't have be inside llama.cpp directories, but it does need to be moved to Termux $HOME path.

1. d. Build llama.cpp:

$HOME

cd llama.cpp

make

Usage example:

cd llama.cpp

./main -m ~/7b-ggml-q4_0.bin --color -c 2048 --keep -1 -t 4 -b 7 -i -ins

By following these instructions precisely, then you'll be chatting with your model.

By moving the model to the correct directory, then ./server will work as expected. For example, ./server -m ~/open-llama-7b-v2-open-instruct.ggmlv3.q4_0.bin -c 2048 -t 4 -b 7

Performance may improve by modifying the -t or --threads parameter depending on your device. -t 4 best for my device, but sometimes less, or more is better for other devices - common advise is to match -t # to the # of physical cores in your device.

Let me know how it goes.

[BetaDoggo]

On Windows:
If you just want to do basic instruct you can just stick the following in a bat file within the llama.cpp directory alongside the executable:
main.exe -m "<model>" --temp 0.7 -t 4 -c 2048 -n -1 --instruct --reverse-prompt "##"
Replace with the path to your model. You get quickly get the path of the models by doing a SHIFT + Right-Click then clicking on "Copy as Path".

[SlyEcho]

You don't really need to compile anything, because we have already compiled executables available on the releases page.